Show HN: Truss – Serve any ML model without boilerplate code(github.com)
github.com
Show HN: Truss – Serve any ML model without boilerplate code
https://github.com/basetenlabs/truss
6 comments
Superb product and team.
Worth looking into if you’ve done any engineering work around deploying ML models as a or within a service.
Worth looking into if you’ve done any engineering work around deploying ML models as a or within a service.
Looks great! What is the argument to use this over MLFlow model packaging and serving?
Truss contributor here
MLFlow is awesome; it's powerful and flexible and extends through multiple valuable portions of the ML cycle.
Truss aims to be simpler to use in both straightforward and complex cases, requiring fewer lines of code but more importantly coming in later in the modeling process. Truss is opinionated and focused.
But interoperability is important to us and we are looking into ways to connect Truss and MLFlow.
MLFlow is awesome; it's powerful and flexible and extends through multiple valuable portions of the ML cycle.
Truss aims to be simpler to use in both straightforward and complex cases, requiring fewer lines of code but more importantly coming in later in the modeling process. Truss is opinionated and focused.
But interoperability is important to us and we are looking into ways to connect Truss and MLFlow.
This is likely to share its name with the next Prime Minister of the UK...
Looks interesting, what if I need to write some logic (pre/post prediction) in the prediction server?
Every time that you make a truss to package your model, you can add custom pre/post processing to every request/response. We've done our best to document it.
https://truss.baseten.co/reference/structure#model
https://truss.baseten.co/develop/processing
Take a look at `{truss_name}/model/model.py` to add the code.
Please open up an issue or PR if you think we can improve on the documentation!
Take a look at `{truss_name}/model/model.py` to add the code.
Please open up an issue or PR if you think we can improve on the documentation!
In this category I’m a big fan of https://github.com/bentoml/BentoML
What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.
I have no affiliation with them whatsoever, just an admirer.
What I like about it is their idiomatic developer experience. It reminds me of other Pythonic frameworks like Flask and Django in a good way.
I have no affiliation with them whatsoever, just an admirer.
Big fan of projects like BentoML, the whole ML space is powered by some truly great projects. What we've tried to do with truss is create an experience that is optimizing for simplicity and universality. Our roadmap begins to deviate away from projects like BentoML, and so creating truss was very much motivated by the need to continue this mindset as we march down through that roadmap.
context: I am one of the contributors
context: I am one of the contributors
For instance in a Rails app the ML model services would probably be served as a completely external service API generated with something like Truss wrapped in a service class that just exposes the outputs and handles errors/input validation!