I tried connecting my on prem server to GCS via airflow astronomer.
I used ssh generated set of public and private key pair, Yet it throws Authentication Failed error. Is there a way to debug and figure out the issue for the same
I would start with just ensuring you can access the SFTP server locally. Try it in your terminal.
From there, maybe try to just move exactly what was in your local terminal and working into a BashOperator - everything should still be working, and if not you should be able to determine what is different between the environments.
After the BashOperator is working, you should easily be able to move that into your desired operator.