I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.
Thanks. That’s a fair comparison, but I’m thinking a bit broader than model serving alone.
I’m less interested in building a generic inference backend and more focused on how the system behaves over time — things like agent lifecycles, state, memory, and coordination. Inference is part of it, but not the whole thing.
Correctness, performance, and privacy matter because the system is long-lived and stateful, not just because it’s pushing tokens around.
Yes — definitely interested in the client side, especially where it intersects with core system design rather than just presentation.
Both native markdown rendering and non-linear / branching conversation models are areas I’ve been thinking about as well, particularly as systems move beyond linear chat and into more exploratory or agent-driven workflows.
Would be curious to hear more about what you experimented with and what tradeoffs you ran into, especially on the data model and UI side.
I too had a similar situation with PayPal. When I filed a complaint they rejected it because the fake seller was able to produce shipping information. They send a letter (or empty package) to some address in your city but not your actual address so that the item shows as delivered. If you look closely you will see that your actual address is never shown in the details.
I thought that too(just for gamers) except I think Apple has some early examples of AR up its sleeve that have convinced it that it is the next big thing. Should be fun to see.
I don’t think you can support something like this if it’s a total black box. If behavior drifts, users need some way to see what state the system thinks it’s in and why it did what it did, otherwise support becomes guesswork.
On updates, I’m pretty cautious about changing behavior silently. I’d expect versioned behavior, explicit migrations, and being careful about what auto-updates versus what users opt into.
And if people are paying over time, updates need to be real improvements they can understand, not just maintenance or regressions.
This feels more like a design problem than a testing problem.