If we want to use pycharm + docker + astro CLI for local development, is there a recommended way to do this?
Astro CLI (at least 0.7.5.2) uses compose under the hood but does seem to write out the file for us.
This presents two problems when you try to use interactive python console.
- while the CLI provides a means for loading environment variables, we either have to do copy paste in pycharm or create our own compose file – neither of which is super desirable.
- If we just reference the docker image directly, we don’t get the out-of-the-box bind mounts. So again, we either have to go fiddling around with pycharm, or make our own compose file.
Both of these issues seem to undercut the potential for smoothness of dev experience with the astro cli. Are there some tricks I’m missing?
Thanks in advance
I tried out this compose file for pycharm console:
version: '3.4' services: python_console: image: <your image> environment: - AIRFLOW__CORE__EXECUTOR=LocalExecutor - AIRFLOW__CORE__FERNET_KEY=d6Vefz3G9U_ynXB3cr7y_Ak35tAHkEGAVxuz_B-jzWw= - AIRFLOW__CORE__LOAD_EXAMPLES=False - AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql://postgres:postgres@postgres:5432 env_file: - .env volumes: - ./include:/usr/local/airflow/include entrypoint: python
This seems to work. I grabbed the fernet key from environment of running container after
astro airflow start. It looks like this is hardcoded into the CLI, so that we could commit this to the repo and it would still work on different developer machines etc.
The question still remains though: what’s the best way to set up for local development, using pycharm including interactive console, while remaining fully consistent with standard astronomer standards for workflow and structure?