Zollerboy1

9 dagen geledendiscuss

Basically, this is the end goal :) TBH, there are still some major hurdles to overcome to get to this point though. We are working on some parts of this pipeline already (e.g. schematic creation or distributor/EDA integration), but other pieces are still quite far away. Especially component placement and routing are still largely unsolved problems and are also beyond the intelligence of today's LLMs.

Zollerboy1

vorig jaardiscuss

Thanks!

> I'm assuming by the pricing it's "serverless" inference, what's the cold-start time like?

Yeah, you could probably call it serverless inference. However, due to the fact that all fine-tuned models are trained on the same base model(s), we have some interesting optimizations we can apply over standard "serverless" model deployment. The biggest is that we can keep the base model loaded in VRAM and only swap the trained weight deltas per request. This gives us sub-second cold-start times for inference in the average case.

> Any idea on inference costs?

Right now, we’re pricing inference at $0.5/M input tokens, $2.5/M output tokens. That’s in a similar price range but a bit lower than gpt-4o/Claude 3.5, which we consider the main models we’re "competing" with. As it’s our goal to democratize access to models/agents in the long run, we hope that we can drop the prices for inference further, which should be enabled by some other optimizations we’re currently planning.

Zollerboy1

vorig jaardiscuss

Wow! Thanks for taking the time to think through it. Yes, you are exactly right! I couldn’t have described Augento better than this myself. We actually want to make writing a reward function completely optional and build some RLHF (Reinforcement Learning from Human Feedback) loop soon. One of our long-term goals is to bring the cost of RL down so the barrier of entry to fine-tuning big models is not as high as it currently is.

Zollerboy1

vorig jaardiscuss

We could open-source (parts of) our platform. What specifically would you like to see open-sourced? We thought about developing this into a piece of software you can run in your own cloud (for compliance and security) but at the moment this makes the GPU economics really difficult and would probably be only interesting/relevant to big enterprises. Anyway, we're definitely curious to hear if anyone has interesting applications for an open-source version of Augento!

Zollerboy1

vorig jaardiscuss

A C only kernel build without CONFIG_RUST will not build a single line of Rust code nor will it touch anything in the rust subdirectory. If you don’t want to deal with rust, you don’t have to at all.

Zollerboy1

2 jaar geledendiscuss

> Is this using WiFi only?

Currently, our software stack only supports WiFi and Ethernet at the data link layer, but that’s mainly because these are easy to set up with linux/batman, not because of an architectural problem.

Other technologies are not only possible but also on our roadmap. Just recently, for example, I’ve been taking a first look into getting LoRa working with hyveOS, but this would probably be a more long-term project.

comments