Ah, my fault. A few of us were talking and somehow I thought we were only dealing with the port.
Under the hood the CLI is using docker-compose type functionality through a library called libcompose. After some googling, it looks like docker-compose supports passing an IP through when you bind the port. I found this explaination - https://www.reddit.com/r/docker/comments/731cop/docker_compose_change_default_ip_from_0000/.
If libcompose works the same way you might be able to change your port setting to be something like:
webserver:
port: 0.0.0.0:8081
For reference, that part of the CLI is here: https://github.com/astronomer/astro-cli/blob/master/airflow/include/composeyml.go#L81-L82