Airflow DAG on Ray Cluster

Env :
Ray 2.1.0
Airflow - 2.5.1
Python - 3.8.15

I am trying to execute a single task on a Ray remote cluster as part of an Airflow DAG but it keeps failing with this error - .

TypeError: Object of type PlainXComArg is not JSON serializable

The task runs locally without any issues but fails to run on any remote instance. Also, the task is executing remotely without any issues if triggered outside of Airflow.
I have followed instructions mentioned here but without much help. - GitHub - anyscale/airflow-provider-ray: Ray provider for Apache Airflow

Any pointers to troubleshoot this ?