What is the best practice of having a unique identifier in Airflow? For example I have ID’s:
ids = [‘A’, ‘B’, ‘C’] right now.
I will process each ID in a series of task. How should I name the tasks? Should I use for loop? Should I give the dag an ID?
every tasks requires a unique task_id. if you are using a loop to create tasks, you want to make sure you somehow include something to make each task_id unique. so if you are looping with an integer, you could name your tasks task_1, task_2, task_3. all dag’s also require a unique id defined when you define your dag. Avoid using . in your dag and task names bc it can cause some weird UI issues in 1.10.3