Env :
Ray 2.1.0
Airflow - 2.5.1
Python - 3.8.15
I am trying to execute a single task on a Ray remote cluster as part of an Airflow DAG but it keeps failing with this error - .
TypeError: Object of type PlainXComArg is not JSON serializable
The task runs locally without any issues but fails to run on any remote instance. Also, the task is executing remotely without any issues if triggered outside of Airflow.
I have followed instructions mentioned here but without much help. - GitHub - anyscale/airflow-provider-ray: Ray provider for Apache Airflow
Any pointers to troubleshoot this ?