Xcom_push will be deprecated in Airflow version 2.0

Hi there, I noticed xcom_push in operators like PostgresOperator is deprecating in Airflow version 2.0.

What is your take on this? How do we suppose to exchange information by the result set of select query with its downstream tasks where we want to use this result set (xcom) variable to do business intelligence/ETL. Am I missing something?

[2020-06-08 10:25:19,259] {taskinstance.py:900} INFO - Executing <Task(PythonOperator): extract_last_batch_date> on 2020-06-07T11:00:00+00:00
/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/postgres_operator.py:54: PendingDeprecationWarning: Invalid arguments were passed to PostgresOperator (task_id: current_batch_date). Support for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'postgress_conn_id': 'va_target', 'xcom_push': True}
  super(PostgresOperator, self).__init__(*args, **kwargs)

Many thanks,


Hey, @ozgurgul! Thanks for reaching out.

Depending on the operator you are using, there could be a xcom_push parameter associated an operator’s __init__ method. As far as I know, BashOperator is the only operator with that parameter in the past.

On a side note, it looks like even that parameter is on it’s way out in favour for do_xcom_push, which will be universal for all operators who are a child of BaseOperator.

Pre 1.10.9 and including, BashOperator returned the output if the xcom_push is set as True.

In 1.10.10, the conditional is removed since the same logic can be found in TaskInstance since 1.10.5.

For clarification from the official airflow documentation, this interaction is described under the XCom section and it has been this way as early as 1.8.1.

In addition, if a task returns a value (either from its Operator’s execute() method, or from a PythonOperator’s python_callable function), then an XCom containing that value is automatically pushed.

Back to your example, PostgresOperator has never had any parameters associated with xcom until 1.10.5 when do_xcom_push was added as a parameters for BaseOperator. Even then, PostgresOperator does not have a return statement meaning it is impossible for any values to be pushed to XCom as is.

On the other hand, there are various routes you can take to communicate the result of a select query through XCOM. One easy way is to use a PythonOperator since it is one of the most flexible operators that allows you to do practically anything in python. An example is provided in the XCom section of the documentation and I have provided a more tailored example below.

from airflow.operators.python_operator import PythonOperator
from airflow.hooks.postgres_hook import PostgresHook

def query_and_push(sql):
    pg_hook = PostgresHook(postgres_conn_id='pg_conn')
    records = pg_hook.get_records(sql=sql)
    return records

action = PythonOperator(
        'sql': 'select * from table;'

Please let me know if you still have questions about XCom! I am happy to explain in further details :smiley:


Great answer @Alan :rocket: and welcome to the forum @ozgurgul


Hi Alan, thanks so much for your clarification. It makes perfect sense.