While I'm here - I recently optimized our PR review skill to make better use of the KV cache.
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.
Is it on topic to complain about the various claude-isms in this article? I don't know any actual humans that write titles like "Two floors the rate card hides".
I find my brain disengages once I suspect something of being written by an LLM. If the author didn't put much effort into writing it, should I expect them to have put much effort into fact-checking it?
Edit: this specific title has been deleted from the article. That was not my point! Please put in more effort into writing things that you want others to read! Rather than putting in low effort but being better at hiding it.
A family member stayed in a hotel in Dubai recently and on returning said how incredible the staff were and willing they were to help them, and my response was "no shit"
Nothing has changed, just OP found the mirror of the main repo with the ee/ folder removed. That repo has been up for most of the lifetime of the company.
That's valid criticism, I kinda hand-waved the "months" part. I read everything I could about parsers while building this (I have a CS background but hadn't thought about parsers in a long time) and came across this blog post https://lakesail.com/blog/sql-parser-in-one-week/ which talked about building a toy parser in a week, so I scaled that up to months for a production one.
The previous parser is mostly a declarative grammar file, which is extremely readable. It codegens a C++ parser, which is hard to read. It depends which of those you count as the previous parser's source code!
In the future, we'd make changes by modifying the ANTLR parser first, then using the same approach as in the blog post to get the new parser to parity. We have no plans to get rid of the C++ parser as an oracle!
Yeah, one of the interesting parts to me while working on this is that the breakpoint for when it's worth writing your own parser vs accepting ANTLR's slowness has shifted massively. Previously it would have been someone's full-time job to maintain. Now with this approach you can get the best of both worlds.
In what way? This was a geometric mean of the improvements from a small test corpus. In production, where it only parses longer SQL that didn't hit the parser cache, the mean parse time went down by 454x, across millions of parses.
Ha I did consider that! But 70x is plenty fast enough (we still have to query an actual database!) and the parser runs in a shared process on untrusted input, so it wasn't worth the security risk
Our SQL is very similar to ClickHouse SQL, in that we used ClickHouse SQL as a starting point as that's what our underlying DB is. We needed to have our own parser so that we could add additional language features on top.
I did that first, which didn't work for a couple of reasons. Many "big and tall" stores are the intersection only, and many of the stores didn't quite have the right vibe. I do have some of my wardrobe from places like 2tall.com but I was looking for something very silly for an in-joke for a friends vacation.
Undecided. One of the sites didn't have an LT but the LLM flagged that chest dimensions on their large were narrower than others, so could be worth trying.
Not super relevant to the Googlebook ad, but in case the perspective is interesting to you: I'm quite tall (194cm) but not very wide, so I usually struggle with buying clothes online. I used AI to scrape a bunch of clothing stores to see whether they sold a men's shirt with an LT or slim fit size, in stock, and matching a particular vibe.
Previously, it loaded up the diff, persona and prompt, and wrote those into the first message for each of 6 sub-agent reviewers. The prompt was templated with the persona name, so was slightly different for each reviewer.
My optimized version had a common first message with diff + prompt, and a script to run to atomically claim a persona. It also runs the first reviewer before the rest to warm the KV cache, and the agent doesn't launch the rest of the subagents until the first persona has been claimed (which means that the LLM is running, and therefore the KV cache is warm). The agent has a script it runs which does the waiting for it.
Agents 2-6 only run once the KV cache is warm, and because the intro message including the diff is shared, it's there in the cache. Only the persona file is different.
In my testing this brought down the cost of a review by ~half, though of course this depends on how big the diff is, how many agent are launched, and what model is used.