Wasn’t that almost the norm for both new OS drivers and subsequently GPU SoC firmware of ‘gamer cards’ to literally optimize for the latest AAA titles? I feel in that case, the interests aligned - in a world where I had a chance to actually play Crysis, I wouldn’t care why the frame rate was decent, no?
“Today, the NASDAQ fell by 18% moments after Simon Willison published his latest blog evaluating the Legend 7.4 model that rendered an animated SVG of a pelican on a bicycle and the pelican fell off”
Agree directionally - even back in sonnet 3.5 days, I was helping some friends by showing them how to create intermediate representations for SVG building blocks mapping to parametrizable functions that can be used to make interactive SVG-rendered visualizations for various medical needs.
Not an expert on this, so won’t speculate regarding what traits would create robustness specifically attributed to SVG visual representation capability, but felt you might find this paper on reasoning models trained on physical world video data becoming better at general reasoning interesting:
What if they’re just unsuccessful at it and something about a model grokking how to create a realistic visual representation of a pelican on a bicycle ends being key to the next OOM of capability unhobbling.
And since we have established this silly routine once, I must keep going and ask - yes, you have quite a collection of bona fide pelicans you’ve seen and photographed.
Have you physically seen all the other animals you’ve evaluated as well?
I was born in Soviet Russia, so trust but verify and if still around, maybe there pelican make LLM draw Simon ride bicycle and notice Python code improve.
Simon - thank you for not dismissing it (and surviving the text that came before the question).
I asked because I genuinely feel that the % of people working on some of the most important technology these days - things such as these 'strangely shaped tools' (to borrow from nearcyan) - large language models - the younger generation (folks in their early/mid 20s) - it is not unlikely that they have not physically seen the meatspace version of whatever digital correspondence of it that is being packed into latent space.
After all, why waste time going to the SF or Oakland zoo? One can just check Simon's latest pelican blog post and skip the zoo trip - the harnesses are waiting.
Simon - has no one told you about the Willison-Pelican Scaling Law?
```
if is_willison_pelican_blog_post:
[redacted]
```
You haven't seen their final form [1]
[1] final form is a frontend/react/let's not talk about it, library - it caused a great deal of PTSD to me and my previous company's team due to its dogmatic preference for "we use these axioms, end of story", over practical utility - so it was quite challenging to do state of the art tasks such as nested form fields (e.g. 'user.address.personal.line-1'). The PTSD it caused made us all block out the memories, I suppose. But - it had zero dependencies. That is what mattered. It kept us going. We weren't reaching for more. We had plenty of time.
And thank god for that. Because I'd forgotten my watch in California - and this was in Tokyo [2]
[2] a joke within a joke about Jensen's Kyoto gardener story. Beautiful story, drowned out by WatchGate memes. Why can't jokes have layers? Models have trillions. If you miss 100% of the jokes you don't make, make all the jokes. Someone will laugh (eventually, maybe?) Even if it's: "this person + comedy club = full secret service detail". If someone laughs at that - at my own expense? I don't mind. They laughed. I know this is a gibberish, off-topic message - it's also a human message. I just felt we need more such things in our lives these days.
PS: have you physically seen a pelican in real life? (not a joke)
My best learnings came from mistakes - mine, my team's, my company's, other companies - but almost always, it was when those mistakes were promptly followed up with a thoughtful incident report.
For example, when I launched what is now Uber One, two days later, I got a call that I broke something. It turned out I had forgotten an edge case in a completely different service. Yes, this was 2016 Uber, 3000 microservices, but I don't think the users who didn't get their discount really cared.
The takeaway I still remember from that today is about always looking for "works by coincidence"; I often ask myself about the preconditions that must be in place for some class of mistake to be impossible.
> I highly recommend talking to strangers! People are lovely. Go out and try it.
I’ve been here since 2009 and this is one of the loveliest comments I’ve read.
At face value, it may seem ‘duh!’.. but there’s a distinct aesthetic to it that resonated with me.
Perhaps the best analogy I can think of is Asimov’s philosophy about writing.
‘I want the reader to forget they are reading as if my thoughts are being transmitted directly from my brain into theirs’ [sic]
Recently, a research publication demonstrated that an LLM.. nah, not today. Sometimes knowing the underlying theory and deciding to disengage from it and just appreciate the moment is fine.
Because I can go outside my apartment here in Tokyo right now and try it. I already do, but each of us has our own unique loveliness. So I’ll keep trying. Just because.
Push notifications and mental real estate by being “an app” are the primary business reason (based on both statsig experiments I’ve seen across my career as well as some intuition about behavioral psychology regarding the app mental real estate bit).
I personally I prefer #3 for its explicitness - you can actually review the code it generates for a new/changed endpoint. It does come w/ downside of more code + as codebase gets larger you start to need a cache to not regenerate the entire API every little change.
Overall, I find the explicit approach to be worth it, because, in my experience, it saves days/weeks of eng hours later on in large production codebases in terms of not chasing down server/client validation quirks.
Tangential, but thought I'd share since validation and API calls go hand-in-hand: I'm personally a fan of using `ts-rest` for the entire stack since it's the leanest of all the compile + runtime zod/json schema-based validation sets of libraries out there. It lets you plug in whatever HTTP client you want (personally, I use bun, or fastify in a node env). The added overhead is totally worth it (for me, anyway) for shifting basically all type safety correctness to compile time.
Curious what other folks think and if there are any other options? I feel like I've searched pretty exhaustively, and it's the only one I found that was both lightweight and had robust enough type safety.
Back in 2015ish Uber we liked to emphasize everyone building at the company to never forget about the long tail [distribution].
When your N crosses gets above N per day, even the 0.1% edge cases happen several days times a day. When this has real world implications, even a single instance can matter a lot.
Cool project! I built a similar tool [0] last year, but:
1. Targeting fbt (Meta's internal i18n tool)
2. Used CST (<3 ast-grep) instead of AST - really useful here IMO esp. for any heuristic-based checks.
3. Fun fact: this was made entirely on my phone (~2.5h) while I was walking around Tokyo. Voice prompting + o1-pro. Why? My friend was working on porting fbt to TS and said he was planning to build this. I wanted to one-up him + convince him to start using LLMs =)
One thing you should be aware of is that for at least Japanese, localization is far from just translating the text. There are lots and lots of Japan-specific cultural nuances you have to take into account for web users and even down to actually just having an entirely different design for your landing page often because those you'll find those just convert better when you know certain things are done that are typically not done for you know non-Japan websites.
Notta (multi-lingual meeting transcriptions + reports) is a great example if you compare their Japanese [1] and English [2] landing pages.
Note how drastically different the landing pages are. Furthermore, even linguistically, Japanese remains a challenge for proper context-dependent interpretation. Gemini 2.5 actually likely performs best for this thanks to Shane Gu [3], who's put in tons of work into having it perform well for Japanese (as well as other "tough" languages)
I spent several hours searching online for existing solutions - couldn't find anything (even when exploring the idea of stitching together multiple different tools, each in a different programming language).
This took me ~3-4 hours end-to-end. I haven't seen any other OSS code that is able to handle converting unstructured JSON into normalized, structured JSON with a schema, while also using a statistical sampling sliding window method for handling for all these:
- speculative SIMD prediction of end of current JSON entry
- distinguishing whether two "similar" looking objects represent the same model or not
- normalizing entities based on how often they're referenced
- ~5-6 GB/s throughput on a Macbook M4 Max 24GB
- arbitrary horizontal scaling (though shared entity/normalization resource contention may eventually become an issue)
I didn't write this code. I didn't even come up with all of these ideas in this implementation. I initially just thought "2NF"/"BNF" probably good, right? Not for multi-TB files.
This was spec'd out by chatting with Sonnet for ~1.5 hours. It was the one that suggested statistical normalization. It suggested using several approaches for determining whether two objects are the same schema (that + normalization were where most of the complexity decided to live).
I did this all on my phone. With my voice.
I hope more folks realize this is possible. I strongly encourage you and others reconsider this assumption!
Cite things from ID based specs. You’re facing a skill issue. The reason most people don’t see it as such is because an LLM doesn’t just “fail to run” here. If this was code you wrote in a compiled language, would you post and say the language infuriates you because it won’t compile your syntax errors? As this kind of dev style becomes prevalent and output expectation adjust, work performance review won’t care that you’re mad. So my advice is:
1. Treat it like regular software dev where you define tasks with ID prefixes for everything, acceptance criteria, exceptions. Ask LLM to reference them in code right before impl code
2. “Debug” by asking the LLM to self reflect on its decision making process that caused the issue - this can give you useful heuristics o use later to further reduce the issues you mentioned.
“It” happening is a result of your lack of time investment into systematically addressing this.
_You_ should have learned this by now. Complain less, learn more.