We’re trying to trigger a DAG from an external source. I know that Airflow has an API that we can use to do so, but is there a best practice around this that you know of other than using a sensor?
Airflow does have a REST API being developed for external triggering but it’s still considered to be in the “experimental” stage (defined by the core Airflow contributors). For that reason, we wouldn’t recommend it as a production solution at the moment.
We’d suggest either creating a DAG that runs at a more frequent interval (possibly what the poke is set at) and skips downstream tasks if no file is found. If you need absolute real-time, it might be a job better suited for a lambda function.