I have installed airflow on K8S and with airflow image.
Question is how to create a custom operator as a pip module so that I can run install that while creating image ?
any reference/sample will be useful
Hi @VivekPemawat!
Im not sure if you have looked into the Plugins functionality that is available in stock Airflow. Plugins are a way to integrate external features such as your custom operators, which would be the most straightforward and preferred method of customizing your Airflow installation.
If you are deploying Airflow on Kubernetes, then that would mean you need to bake the additional pieces into the airflow image itself with the files in your $AIRFLOW_HOME/plugins folder.
Another way mentioned in the documentation is using the plugin as python packages. You can follow the instructions to see if that is better suited for your installation.
I have also seen deployments where the wheel is self-hosted in a PyPI server to be downloaded and moved into the plugins directory, which is a blend of both methods.