The most interesting findings (at least from my perspective)
- OpenAI measured "honesty" that could become an interesting model eval others should adopt.
- Using instruction heirarchy to _avoid_ prompt injections
- When pushed to pursue misaligned goals, it tried to disable oversight in 5% of tests!
I've been thinking about AI guardrails in production for almost 7 months now, and I'm convinced that using them to "control", "secure", or "block" LLM outputs is not the best use for them .
Using them to "guide", "accelerate", and "empower" your AI development process? Now we're talking.
Let me explain why I believe this shift in perspective could change how we build AI...
Most people see AI guardrails as training wheels. But what if we saw them as a nitrous boost for our AI hot rods instead?
When you're not worried about your AI going off the rails, you can push it to its limits. You can experiment with wilder ideas, test bolder hypotheses, and innovate at lightning speed.
Want to train on spicy data? Guardrails have your back.
Thinking of an aggressive model architecture? Guardrails keep you in bounds.
Controversial prompt? Guardrails ensure you don't cross the line.
Am I in the right direction, or am I just going off the rails?
Being one of the most downloaded datasets on Huggingface, I was a little bit surprised by how dirty this dataset was. Plus it had very limited information and some incorrect classifications as well.
For an internal experiment on building a "Truthful Evaluator", we picked up this dataset and tried fine-tuning a model on these 8000 odd examples.
Realised that it needed:
1. Cleaning up
2. Some reclassification
But, most importantly - it lacked context data. It only had a link pointing to the source which was also absent for a few rows.
We scraped the internet for the link in the dataset, matched it to the question and narrowed down on a small context to be added to the main dataset.
Releasing it publicly so that someone else may avoid the 2-3 days of pain of wrangling with this data.
tl;dr:
- Keep prompts short, combine prompts or make more detailed prompts but go to a smaller model
- Simple and semantic cache lookups
- Classify tasks and route to the best LLM using an AI gateway
At Portkey, this is a problem we deal with quite a bit. Also the reason that Datadog and the traditional observability vendors did not work for LLM use cases since they're not built to handle large volumes of data.
We've done this through a careful combination of Clickhouse + MinIO for fast retrieval of log items + selected retrieval from the MinIO buckets.
Cost becomes a very big factor when managing, filtering and searching through TBs of data even for fairly small use cases.
One thing we lost in the process is full-text search over the request & response pairs and while we try to intelligently add metadata to requests to make searching easier, it isn't the complete experience yet. Still WIP as a problem statement to solve and maybe the last straw here. Any suggestions?
Individual subreddits have usually been my go-to place to discuss and understand the nuances of academic papers. Love LocalLLama for discussions on generative AI papers.
You beat us to posting Portkey to HN first! Thanks @gaocegege.
Having built LLM systems for the past 3 years, we painfully realised the need of DevOps for LLM systems, now emerging as the LLMOps/FMOps category. We're dead focused on the production side of things.
@HN - when building with LLMs, are you looking for LLMOps tools? How do you choose? What's most critical?