PythonVirtualenvOperator connections

Airflow
1

Hi Guys,
It’s posible passing connections when using PythonVirtualenvOperator ?
In the actuallity utilizamos PythonVirtualenvOperator, but we have not been able to access the connections already configured.

Tranks for your help.

2

Hey @alexisaraya

I know its late, but for everyone’s benefit going to try and answer your question.

If I understand correctly, you want to use the Airflow Connections you have created from within a PythonVirtualenvOperator. Short answer is yes!

The reason is, PythonVirtualenvOperator accepts a python function that will perform the action within the venv created by this operator. And you can call the connections and use them via Hooks within this python_callable.

For example:

from airflow.models import Connection

def my_venv_python_fn():
    # test the connection
    conn = Connection('my_conn')
    conn.test_connection()

    # Directly get the connection
    Connection.get_connection_from_secrets('my_conn')

    # use the connection in a hook
    conn = PostgresHook(postgres_conn_id='pg_conn')
    conn.copy_expert("COPY my_table FROM STDIN WITH CSV HEADER DELIMITER ','", "/usr/local/airflow/include/my_table.csv")

Then you can pass this function in python_callable of PythonVirtualenvOperator. See how to use this module in Astronomer registry.

Thanks
Manmeet