This is something I've been working on exposing to AI labs through my startup LatentEvals[1], and found similar results in other industries from lending to insurance claims.
Happy to share some sample reports if anyone is interested!
There are multiple "sandbox backends"[1] available besides Vercel currently. I'm sure more will follow, and it's open-source so you could always implement your own.
I'm the creator of hty, as part of my work for building custom eval suites at LatentEvals.com I found that AI agents struggled with interactive CLI / TUIs and would resort to setting flags that may be outdated or not exist at all.
I needed a way for AI agents to use all the software that humans do, things like htop, k9s, create-next-app, and even nethack.
How it works: hty wraps any interactive cli in a persistent PTY session. Your agent "sees" the rendered terminal the way you do and types the way you would. I've had a lot of success with agents being able to get through interactive npx setups, play terminal games, and create logical commits from a bunch of work in an long claude code session by using git add -p for interactively patching the git stage.
It's available to use as a simple skill so it works for almost all agents, but you can also use it as the human via the CLI. It's written in zig and built on top of libghostty.
I've been doing bias and misaligned behavior research, creating custom private eval suites to test and compare models. Claude Opus 4.7 is heavily biased and presents clear regulatory and reputational risk.
It seems the initial product footprint tries to sidestep this problem by not giving the agents control on who to lend to or which applications to approve. Even so I think it's quite an optimistic read on their end. Happy to share reports to anyone who's interested ([email protected]), especially if you work at a frontier model lab and are interested in plugging my evals into your RL systems!
Author here, I built this because my agents could not use TUI or interactive CLI software. Think puppeteer for TUIs.
If you've ever asked Claude Code, codex, opencode, etc... to make logical commits from a bunch of changes you might have seen them struggle with moving files around, deleting files when a simple `git add -p` would have let them work through the changes interactivly. ht (the binary name) allows them to spawn their own terminal and drive it similar to puppeteer, sending keys, viewing a snapshot of the terminal and making a decision.
Some other use cases are also solved from CI/CD to making nice demos (it can export images or the format used by asciicast for making videos of the session.
It's powered by libghostty so it has support for all the stuff a modern terminal can do (full color, mouse clicks, etc...) and I've had a lot of fun letting claude run wild and even play games like nethack.
Happy to answer questions, and of course let me know if you run into any issues!
It seems they nerf it, then release a new version with previous power. So they can do this forever without actually making another step function model release.
A little off topic, but did Anthropic distill from an older OpenAI model? All the sudden over the last few days I'm getting a ton of em dashes in claude code responses!
It gets really fun when you prompt them to update their BOT.md, also they get access to previous run results so they continually "learn" from mistakes or investigate changes.
https://github.com/montanaflynn