I want to install some libraries for DAG creation, some open source not required for Airflow and some custom made to reuse code that I already have. I tried copying the packages to the path where Airflow installed the libraries, but it didn’t work. I saw import errors. What is the best practice to handle this usecase? My DAG creation is likely to be complicated and I will require to arrange it in packages in DAGS home.
Hey @aircolleague - can you post the import errors you’re seeing?
Generally speaking, you can put any python packages in the requirements.txt file generated by our CLI’s astro airflow init command and all os packages in the packages.txt file generated by that same command- more on that here.
If the packages you’re trying to install aren’t publicly available, you can follow the steps that @paola outlined in this forum post and associated doc.
1 Like
Pete,
Thanks for your reply with all the details. Will try these and get back to you.