Similar issues arose with automated learning companions for XR. Mismatches between a learner's mental model of the companion, and its actual capabilities and limitations, can cause so many problematic misunderstandings. So one idea, is to emphasize non-human non-social sources of behavior - to set up alternative narratives for "why did it do that?". The companion might be "Sparky, the glitching robot dog". So forgetting things, getting them wrong, spacing out, being unhelpful, confused, distracted, missing social cues, giving poor cues, etc... that's just glitchy robot dog glitching, not social signal. Maybe - speculative.
> the hidden state for different layers carry meaningful self-awareness signal for various situations.
Is it plausible to wonder if some developer judgement feels, like maybe "the code I just wrote is clean/crufty", or "things came together smoothly/janky", might have extractable signals in some models?
If so, might one create a shopping list of desired signals to check for in a model, as with activation steering concepts, where one checks whether and how hard each concept can usefully be nudged?
Meta thought: Having recently learned of WeChat Groups in China, I'm now repeatedly noticing situations where familiar alternatives are higher friction.
AI as salonnière. It's not enough to get lots of interesting people in a room (as at a meetup). That merely gets you to "there are so many valuable conversations latent in this room... and basically none of them will happen".
Also beyond the room. AI as ambient conversational matchmaking serendipity, weaving connections - "btw, that person in line with you also loves foul-weather sailing".
Years back, I was expecting this to rapidly become a pervasive app category. I'm unclear on why it hasn't.
In interactive lecturing (clickers) with peer instruction, there's an idea that people log in with seat location, and based on who understands or is confused about what, and having a record of past interactions, the system optimizes the fruitfulness of each peer assignment pairing "Alice, turn and discuss this question with Bob sitting to your rear left".
> To build the dataset, we keep only sketches that have been recognized by the neural network in the QuickDraw game. To prevent imbalance from overly represented countries (e.g., the US), we down-sample the data by focusing on the 100 most prominent countries and capping the number of drawings per category–country pair at 10,000. [p19]
>> children figure out what they want to do and the learning comes along with it; the model views adults wanting children to learn something specific as generally counterproductive
> One thing AI is uniquely good at is meeting learners where they are
What if the AI wanted children to learn something specific? Able to patiently await an opportune moment. Able to blend it invisibly into other material. Able to subtly check and correct understanding.
Long term, one possibility is AI enabling a massive implicit curriculum. "[A]dults wanting children to learn" about say street crossing might be counterproductive... but funny how, at opportune times, some random stories just happen to include crossing a street, and do formative assessment, and happen to, quietly and eventually, cover and reinforce the associated learning objectives.
Take How the Piloses Evolved Skinny Noses, a children's picture book inoculation against natural selection misconceptions. It could merely be a book on a shelf. Or the AI might introduce or provide the book at an opportune moment. Or the book's approach itself might be dissolved and blended into other content.
So some story is explicitly about Fido going for a walk. But implicitly, it covers some aspect of safely crossing a street, and street lights as a communication device, and of concrete crack propagation, and tick precautions, and natural selection, and ...
Science is a richly interwoven tapestry of stories... but we basically never teach it like that. Even if such material was gathered, which pre-AI was absurdly implausibly demanding of domain expertise, it would largely be beyond the capability of an individual tutor to compellingly and adaptively deliver. But with AI?
Yes, it's easy to forget context. Especially the time component. Several 100 k children will be born tomorrow. 100+ M in the coming year. Then in six years, almost all of them will hit schools. Schools which are, pervasively, really really bad.
What can we do to help them? Better teacher training? Better new teachers? Better new teacher colleges? Consider the latencies. A decade cohort is 1+ B kids.
So refining intervention safety and efficacy is nice. Especially for development problems. Less so for an acute humanitarian disaster. For maximizing the golden hour in crisis mass-casualty triage. In some literate and numerate sense, most of this last decade's 1+ B kids did not survive our collective care.
> What happens when the low-hanging fruit is gone?
Depends on why it's gone? For example, before the shocker of Facebook buying Oculus, there was an active community of side-gig businesses selling kits around VR, large touch panels, haptics, and such. What couldn't exist, was scaling that beyond hobby kits. No VARs, no integrators or assemblers, no OEMs, no contract or bespoke manufacturing, no searching for viable niche products. Because "it's merely a kit" was the only way the US patent system allowed the stuff to exist at all as a business. The gap between what you could make, and what you're allowed to sell, is large. Even now, we can't reasonably search for those niche market fits, because such scales are beneath the notice of big patent-holding companies. So absent FRAND reform or some such ("you're required to let me use it and pay you"), no fruit for you.
CA lacking non-competes, "Stealing IP!", grew software fruit in CA. But the US has lacked the growing conditions for Shenzhen fruit. So, what happens when the low-hanging fruit is gone, from the US? Fruit is grown, and enjoyed, elsewhere.
Consider "spec -> plan -> implement" as an unremarkable agentic workflow. TFA suggests that Spec be organized by cross-cutting concerns. This seems a nice thought.
Doubts in comments so far seem around expectations of an AOP-like experience. Mainly that patching aspects/Spec makes happy changes to implementation. I'm not sure TFA intended that use. Nor that current AOP implementations provide the happy. But lets explore.
With a frontier model, a greenfield ends up with, say, some spec and plan docs, and a "woven" implementation. It's reasonable to wonder how well an LLM will then un-and-re-weave code as it makes changes. And how to express join points or equivalent. And how well code review agents will detect/check intended joins continuing to be woven correctly. And whether a baseline of the LLM simply using existing AOP tooling constitutes progress.
Coming from non-frontier models, some of that seems more straightforward. Harnesses use lots of small jobs/tasks, with subagent paperwork and reviews. Historically scarce context, precious context, encouraged tightly optimizing single-task context for each job. Including derived code "views", rather than raw code. And associated LLM-coded tooling around working with not-the-raw-code.
Thus working with name/signature pseudocode outlines. And filtering code/ast. Both get you closer to having unwoven forms, and to deterministic tooling for un/re-weaving. Consider i18n clutter - just to coddle the model, one might strip it (replace it with English), let the model work, and then restore it. Which is equivalent to un/reweaving an i18n aspect. And even read-only views, like function distillations which drop or abstract some specified variables and associated code, make that "check spec against implementation" code review easier. So, even interpreting TFA broadly, it seems at least potentially possible, no?
Big picture, I'd like to get away from traditional "repo as single point in design space, laboriously nudged around", to something which preserves the multiplicity of model/run/prompt ensembles, and allows playing on design space manifolds. "This exact code has been in prod" is useful information, if underutilized, but not so useful as to be worth discarding so many new possibilities.
I once worked on a medical hackathon concept for computer-assisted population screening for cervical cancer in a developing nation. Community health workers take photos. The AI would look at the images, and make a call of "clearly negative" vs "clearly positive" vs "needs (scarce) expert review". But taking good photos is hard, so it's also "photos insufficient" and "worker needs additional mentorship on taking photos". Only by computes reducing all three costs - expert workload, exam success, and quality-control/training - might successful deployment be financially and logistically plausible for that nation.
With direct discussion, the same tendency to harmonize towards groupthink applies.
Aside from the statelessness GP mentioned, one can insert anti-conciliatory intermediation. "I saw a random claim go by, but something about it seems not quite right. What am I missing? They said: [...]." Weaponizing the bias, and orchestrating the discourse from the harness.
While diffusion from joint space is perfusion limited, some work suggests ultrasound might increase that perfusion? By vasodilation and microvascular recruitment, from heating and shear stress on endothelial cells. With bubble vibration and cavitation as one source of stress.
Also, though perhaps not rate limiting, ultrasound might be able to mess with the N2 bubble boundary diffusion rate.
Caveats: Very not my field; no clinical practice; mostly animal studies; mostly musculature and not joint; and under-validated AI.
Meta: But I so very much enjoy surfing literature with AI. Even with AI's rich collection of interesting failure modes. They serve as fun added encouragement to keep you on your toes, and keep clear on the gradients of your confidence.
> Open weights/source doesn't necessarily mean running on local hardware, though.
And we've barely started to scratch the surface on helping open-weight models "be the best they can be", with cloud burst parallel sampling and prompt mutation. Looking for best probabilistic results for a prompt, and looking for best prompt variants for a task. Adaptively scaling computes at generation, not just training.
And speculatively, if agentic coding is naturally a multiplicity, what UX might enable human devs to dance with that quantum superposition? Rather than quickly collapsing to one monkey and its keyboard.