Hey, @ozgurgul! Thanks for reaching out.
Depending on the operator you are using, there could be a xcom_push parameter associated an operator’s
__init__ method. As far as I know, BashOperator is the only operator with that parameter in the past.
On a side note, it looks like even that parameter is on it’s way out in favour for
do_xcom_push, which will be universal for all operators who are a child of BaseOperator.
1.10.9 and including, BashOperator returned the output if the
xcom_push is set as True.
1.10.10, the conditional is removed since the same logic can be found in TaskInstance since
For clarification from the official airflow documentation, this interaction is described under the XCom section and it has been this way as early as
In addition, if a task returns a value (either from its Operator’s
execute() method, or from a PythonOperator’s
python_callable function), then an XCom containing that value is automatically pushed.
Back to your example, PostgresOperator has never had any parameters associated with xcom until
do_xcom_push was added as a parameters for
BaseOperator. Even then,
PostgresOperator does not have a return statement meaning it is impossible for any values to be pushed to XCom as is.
On the other hand, there are various routes you can take to communicate the result of a select query through XCOM. One easy way is to use a PythonOperator since it is one of the most flexible operators that allows you to do practically anything in python. An example is provided in the XCom section of the documentation and I have provided a more tailored example below.
from airflow.operators.python_operator import PythonOperator
from airflow.hooks.postgres_hook import PostgresHook
pg_hook = PostgresHook(postgres_conn_id='pg_conn')
records = pg_hook.get_records(sql=sql)
action = PythonOperator(
'sql': 'select * from table;'
Please let me know if you still have questions about XCom! I am happy to explain in further details