First I want to commend y’all on some fantastic repos under your org. I’m currently leveraging the 0.19.0 helm chart and trying to find the most stable ^2.0.0 image to use for both local (docker-compose) and remote (k8s) development.
Recently I’ve run into some issues regarding 2.0.0 (post 1 through 5) where the docker compose is failing during start-up due to an issue with the utils/db.py run and I’m trying to avoid going too deep down the rabbit hole triage wise. It looks like the migration is failing part of the way through (stack trace at the bottom of the post) and I’m curious if this is a known issue or if the docker-compose.yml generated from the 0.23.4 CLI cut is out of date now for these images?
Secondly it looks like 2.0.2-* is not compatible with the 0.19.0 chart and I’m curious if there is a branch cut I can be leveraging that fixes that? Scouring the repo last night I didn’t see anything promising and was this close to forking and testing the chart pointing the default version to 2.0.2-* just in case this was a migrations incompatibility between 2.0.0 and 2.0.2.
Lastly, I’m trying to leverage poetry for our artifact build and I’m running into an interesting solver conflict with putting astronomer_certified in:
Because astronomer-certified (2.0.0.post5) depends on apache-airflow (1) and apache-airflow-providers-databricks (1.0.1) depends on apache-airflow (>=2.0.0), astronomer-certified (2.0.0.post5) is incompatible with apache-airflow-providers-databricks (1.0.1). So, because airflow-libs depends on both apache-airflow-providers-databricks (1.0.1) and astronomer_certified (2.0.0-5), version solving failed.
I’m curious if the transitive bind for ^2.0.0 is intentionally set to 1.0.0 for some reason I’m unaware of or if this is just a global repo compatibility concern? Either way right now I’m forced to swap to apache-airflow for the reference but it would be nice to leverage the correct pip so that the site installs match what is in the image.
Stack trace for 2.0.0-5:
scheduler_1 | Traceback (most recent call last): scheduler_1 | File "/usr/local/bin/airflow", line 8, in <module> scheduler_1 | sys.exit(main()) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/__main__.py", line 40, in main scheduler_1 | args.func(args) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command scheduler_1 | return func(*args, **kwargs) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 89, in wrapper scheduler_1 | return f(*args, **kwargs) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/db_command.py", line 48, in upgradedb scheduler_1 | db.upgradedb() scheduler_1 | File "/usr/local/lib/python3.7/site-packages/astronomer/airflow/version_check/plugin.py", line 29, in run_before scheduler_1 | fn(*args, **kwargs) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/utils/db.py", line 684, in upgradedb scheduler_1 | command.upgrade(config, 'heads') scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 294, in upgrade scheduler_1 | script.run_env() scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 481, in run_env scheduler_1 | util.load_python_file(self.dir, "env.py") scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/util/pyfiles.py", line 97, in load_python_file scheduler_1 | module = load_module_py(module_id, path) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 182, in load_module_py scheduler_1 | spec.loader.exec_module(module) scheduler_1 | File "<frozen importlib._bootstrap_external>", line 728, in exec_module scheduler_1 | File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 108, in <module> scheduler_1 | run_migrations_online() scheduler_1 | File "/usr/local/lib/python3.7/site-packages/airflow/migrations/env.py", line 102, in run_migrations_online scheduler_1 | context.run_migrations() scheduler_1 | File "<string>", line 8, in run_migrations scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/runtime/environment.py", line 813, in run_migrations scheduler_1 | self.get_context().run_migrations(**kw) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/runtime/migration.py", line 548, in run_migrations scheduler_1 | for step in self._migrations_fn(heads, self): scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/command.py", line 283, in upgrade scheduler_1 | return script._upgrade_revs(revision, rev) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 361, in _upgrade_revs scheduler_1 | for script in reversed(list(revs)) scheduler_1 | File "/usr/local/lib/python3.7/contextlib.py", line 130, in __exit__ scheduler_1 | self.gen.throw(type, value, traceback) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/script/base.py", line 194, in _catch_revision_errors scheduler_1 | compat.raise_(util.CommandError(resolution), from_=re) scheduler_1 | File "/usr/local/lib/python3.7/site-packages/alembic/util/compat.py", line 294, in raise_ scheduler_1 | raise exception scheduler_1 | alembic.util.exc.CommandError: Can't locate revision identified by '2e42bb497a22'