If I had my way both Python runners and the runner webapp would not be long for this world.
I will say, though: the runner webapp MVP exists to do basically what you're describing (and keeps an internal queue). Yatai is architected so that the runner instances run on a separate Kubernetes cluster to the pre- and post-processing code that that's run in the main webapp itself, and can be scaled separately.
You can absolutely mount a BentoML service into your own Starlette (or any ASGI framework); `svc.asgi_app` is all you'd need.
Instantiating and using a runner can be done anywhere with `init_local`, but it's really the runner ASGI app that does the work of queuing and batching. We've thought about allowing users to spin that app up separately but it's not a focus right now; instead we're trying to ensure that the system is as easy to use for data scientists as possible and have that workflow fully ironed out before we support the more advanced use-cases.
The whole runner situation is quite complex because we wanted to support user-created runners in the nicest way possible, and also leave the space open for non-python runners (and service app) in the future.
Syncthing is less purely peer-to-peer. It's easy enough set up a central server and have your devices only sync to that server because of the way connections are made.
I will say, though: the runner webapp MVP exists to do basically what you're describing (and keeps an internal queue). Yatai is architected so that the runner instances run on a separate Kubernetes cluster to the pre- and post-processing code that that's run in the main webapp itself, and can be scaled separately.