How to customize the Airflow job status colors based on Java job result

Airflow
1

I am planning to map the Talend job(or any normal java program) return status code to Airflow Status UI.
Can you please how to map it?

By default Airflow will show job status in some color like Success(Green),Failure(Red) and up_for_retry(Orange) based on the job result.But how to show the status based on Java job return code on UI.

Thanks in advance.

2

Hey, @RRR! Can you provide some more detail as to what sort of operator that you are using to run the java job?

Some past examples I have seen are custom operators leveraging the Bash Operator to run the java command and obtaining the exit status of the last command executed through the script below.

echo $?

If that is the way you are passing the return status of the java process to airflow, you can simply control the output in the java program itself. An output of 0 generally means success and any non-zero output is considered an error. Airflow will set the task status to success with a return value of 0. Otherwise, if there are retries set and the task failed then the task will be marked as up_for_retry or if no retries are set it is set as failed.

3

Thank you @Alan for quick response.I am using BashOperator and invoking jar as below .I wanna customized the java logic to return the status code as my business standards.In this scenario,I need logic how we can map the java return code to Airflow job status to reflect in UI.

t2 = BashOperator(
task_id = ‘t2’,
dag = dag,
bash_command = 'java -jar /home/ryada/airflow/talendjobs/AirflowAPIConnecter.jar’

)
Thanks in advance.

4

As described above, the exit code has to be communicated through airflow via the script listed above. It should look something like this.

from airflow.operators.bash_operator import BashOperator

t2 = BashOperator(
    task_id='t2',
    bash_command='java -jar /home/ryada/airflow/talendjobs/AirflowAPIConnecter.jar; echo $?',
    dag=dag
)

The actual logic is available in the execute function of the Bash Operator. As you can see, if the return value is 0 it will fail the conditional and won’t raise an AirflowException. Otherwise, any none zero return values will raise the exception causing the task to fail.

As for how you control the exit code as your program reaches various end states, the Java code below could be used to set the return value as described here.

System.exit(n);

Let me know if that resolves your query or if I need to elaborate the solution.

5

Thank you @Alan …Actually we are able to print the exit code in logs without $echo .But my question is how to made handshake this status code in Airflow env.(Example I wanna customize the Airflow status UI as per the status code returning back from java code) .Pls let me know What kind of Python changes has to be done for above requirements.

6

I see. This is not possible to my knowledge. The state of the task is solely dependent on whether the execute function succeed or not, which is dependent on whether an exception was raised. For BashOperators, it is using the return code as the determining factor.

The closest way to replicate the behaviour your are describing would be to utilize the how the conditional is setup. 0 for success and any non-zero values would mean failure.

If you are able to figure out a way to map the output to the status of a task with more flexibility, please let me know! I would love to document this behaviour.