I have been working on a project that is built off of Cloudflare's pingora project - https://sbproxy.dev. This provides all of the features of a LiteLLM, but can also support your API traffic too. I worked on the edge infrastructure for Walmart, and took those learnings and build a fully open source gateway.
Take a look - should be able to address your needs.
Another reason golang is interesting for the gateway is having clear control of the supply chain at compile time. Tools like LiteLLM the supply chain attacks can have more impact at runtime, where the compiled binary helps.
SBproxy (https://sbproxy.dev) - An API and AI gateway as a single binary. Sits between your apps and LLM providers to give you more control. Think Cloudflare meets LiteLLM.
After the LiteLLM supply chain hack last month I feel pretty good about that choice. Your LLM gateway holds every provider API key you have. That probably shouldn’t be a pip install.
Worked on API gateway for a major retailer and incorporated my learnings into this platform.
Entropy is real. Microsoft has lessened the friction of purchasing vs their competitors. Public Sector this may be their only choice because they set the groundwork decades ago to do business with most organizations. It may not be the best solution, but it offers the least resistance to getting something up and running.
The supply chain impact on manufacturers in the US on contracts is overlooked in this tariff fiasco. For example - auto manufacturers enter into long term agreements as a fixed price for a product. The only negotiation point they may have to reset that price would be a government action, which a tariff checks the box. The genie is out of the bottle, lots of contracts will be negotiated and prices will go up.
I would like to think this is some kind of 4-d chess game to avoid rate hikes and to devalue the dollar, but on implementation it will accomplish none of the above with a sprinkle of recession.
This was a great throw back. My interest in Electrical Engineering was traced back to building out simple circuits on this device. Little did I know my day to day at school was going to involve even more complicated circuits on a bread board - I was in heaven!
This is an area that government subsidies could really influence change in urban planning and cutting oil demand. If there was a similar subsidy on bikes as there are on electric cars, I would expect the push back against bike infrastructure would become less. Right now in the Bay Area through poor design and aging infrastructure there is push back on bike lanes. An example is the Richmond bridge, which has a protected bike lane taking a 3rd lane of traffic that could see a larger number of riders if more ebikes become used. Likewise for the Bay Bridge, whose bike lane is a ghost town in the mornings when commute traffic is worst. This would be less of a problem if the lane went entirely to the city.
This is a major pain point moving customers from Oracle to cloud services. PL/SQL encapsulates a lot of logic in legacy enterprise applications that handcuffs customers to the platform similarly to mainframe systems. Migration tools can get part of the way there but still requires a strangler type pattern when migrating.
You should look at Lucene core - they have incorporated vector embeddings in 9.4.x and it could provide you better scale than Redis with durability as well.
Word processors in the 70s had this as well. I remember the CPT 8100 having this as a selling point in their literature. There was no inline correction. Run the program, correct your issues then continue!
Anyone else remember Shift+F7? Bring out your Wordperfect templates.
Have your process regularly update the CDB file from a blob store like S3. Any deltas can be pulled from S3 or you can use a message bus if the changes are small. Every so often pull the latest CDB down and start aggregating deltas again.
It frustrates my partner to no end. I always ask for a discount. Worst case you end up paying the amount you expect, best case you open up a window to negotiations of a discount. It never hurts to ask.
I have been on a similar trend lately. Trying to first focus on low gluten/no gluten and nothing with processed sugar in it. Often hard to eliminate it all but even a dramatic cut back in processed foods has made a difference on my sleep.