Yeah. Great callout. You basically need signal all the way to production. Did it get nuked during a code review? Is it still running 2 mo later? How many incidents were caused by this code?
This is tough though because enterprises go absolutely ballistic over “training on our data” - which is understandable, but will also hold us back.
This reduces down to the problem of summarization - a quite difficult one. At commit time it’s difficult to know what questions readers will have. You can get close but never all the way there.
Pre AI when engineers couldn’t find the answer in commit messages or documentation they would ask the author “why” and that human would “compute” the summary on demand.
I think that’s what I expect to do with these agent sessions - I don’t want more markdown, I want to ask it questions on demand. Git AI (https://github.com/git-ai-project/git-ai) uses the prompts that way. I think that model will win out. Save sessions. Read/ask questions relevant to the current agent’s work.
On asking peers. This is regrettably on the way out today - I’ll ask engineers about complex code they generated and they can’t give good answers. I think it’s because it all happened so fast — they didn’t sit with the problem for 48 hours. So even if they steered the agent thoughtfully it’s hard to remember all the decisions they made a week later.
love the shout but git-ai is decidedly not trying to replace the SCMs. there are teams building code review tools (commercial and internal) on top of the standard and I don't think it'll be long before GitHub, GitLab and the usual suspects start supporting it since folks the community have already been hacking it into Chrome extensions - this one got play on HN last week https://news.ycombinator.com/item?id=46871473
I met Marshall a few times. He was a good teacher and someone who had a positive impact on several successive classes of students who wanted to start companies and build meaningful products + technologies on that campus.
And I trust (quite a bit) that whatever he brought to light should be followed up on - if no other reason than to respect his memory. I hope it is taken seriously and those who retaliated find themselves w/o their positions of responsibility and power over other faculty.
Useful here-say from some investors at the last few demo days: not all of the companies that are "copiers" apply + are accepted with the "copying" idea. Many founding teams end up pivoting during the batch and scramble to get proof points on the board before demo day. They're most likely to end up pivoting to well-known problems, therefore the clustering around a few common themes. It doesn't explain all of the data, but it's a big part of it. When you have to come up with a fundable new idea in a week and prove it out in a month this can happen...
Printed this out and pasted it into my journal. Going to come back to it in a few years. This touches on something important I can’t quite put into words yet. Some fundamental piece of consciousness that is hard to replicate - desire maybe
I heard inside info from the hill. Apparently a few weeks ago when TikTok got thousands of kids to call their congresspeople, it tipped a ton of the people on the fence to the “hell yes” category. Many offices got more calls than they do in a month that one day. TikTok showed how powerful it is, right there, right then.
If China / TikTok hadn’t overplayed the hand, they wouldn’t lose this weapon. Now they likely will (good thing)
This clicked for me when someone explained that fault != responsibility.
It unlocked the whole thing. I could take responsibility for getting the life I wanted, but that did not mean the bad things that came my way were my fault. Bad shit always comes, not always as a result of my actions, but it's always my responsibility to change the situation.
Hey HN - I’ve been working on adding support for natural language rules to Optic’s API linting + testing tool [0].
Instead of generating API designs, we’re using LLMs to check if a team’s API is “good” by their own standards.
Many large companies that use OpenAPI have custom Spectral + Optic rules to enforce API security practices, consistent styles, versioning policies, SLA adherence, and to prevent breaking change. These tests are hard to write and cannot verify a lot of the more abstract aspects of an API design. It’s really hard to write code to check if an OpenAPI operation allows batch creation, but when you give an LLM YAML and the rule “No endpoint should allow users to create multiple resources at once” — it does a surprisingly good job at testing the design and writing a nice error message when it fails.
The hardest technical challenge has been:
- figuring out how to fit OpenAPI files (sometimes 10k lines or more) into the context window. Optic already breaks up API specs into their parts: ie parameters, response bodies, headers, etc. We’ve been putting just the relevant lines of the OpenAPI file into the LLMs and resolving all their dependencies beforehand so the model has all the context it needs.
- Minimizing API calls. The first time you run LintGPT it is pretty slow because it has to run every rule across every part of the API specification (1000s of calls). But we shouldn’t have to repeat that work. Most of the time parameters, properties, etc don’t change and neither do the rules. We’re building caching into our web app to make this fast / save $ for end users.
Happy to answer any questions. I really think there’s a huge use case here for linting all kinds of code, config, database schemas, policies in ways that were never possible before. And personally, I like the idea of having these smart tools guiding me towards making my work better vs generating it all for me — idk something about that just feels good.
Ok then -- so these forward passes should take about the same amount of time per inference of the next token. That matches my intuition. When does the model know to stop? Is there a stop token?
Also aside -- totally crazy this thing can run a full forward-pass many times a second. I naively would have bet each pass token a lot longer even on powerful GPUs
This is tough though because enterprises go absolutely ballistic over “training on our data” - which is understandable, but will also hold us back.