Problem
I’m using the Kubernetes Decorator with Airflow 2.4.2 to run tasks with the KubernetesPodOperator and more easily define them with: @task.kubernetes()
.
But I consistently get this error:
{taskinstance.py:1851} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/decorators/kubernetes.py", line 95, in execute
write_python_script(jinja_context=jinja_context, filename=script_filename)
File "/usr/local/lib/python3.9/site-packages/airflow/providers/cncf/kubernetes/python_kubernetes_script.py", line 79, in write_python_script
template = template_env.get_template('python_kubernetes_script.jinja2')
File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 1010, in get_template
return self._load_template(name, globals)
File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 969, in _load_template
template = self.loader.load(self, name, self.make_globals(globals))
File "/usr/local/lib/python3.9/site-packages/jinja2/loaders.py", line 126, in load
source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python3.9/site-packages/jinja2/loaders.py", line 218, in get_source
raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: python_kubernetes_script.jinja2
Root Cause
To use the Kubernetes decorator, python_kubernetes_script.jinja2
is required but didn’t get packaged in the providers package.
Workaround/Fix
This issue is generally expected to be fixed in Airflow 2.4.4 or Airflow 2.5 with this PR.
In the meantime, add python_kubernetes_script.jinja2
to your Astro project.