A potential solution would be to have a reputation system, with some light pageranking in it. People who care about their reputation are bound to rate other likewise people higher. I don't think such loyalty exists between vibetalkers.
It's obvious we're not alone, and 2 distinct clusters would form (with people on a spectrum in between)
- rank some first results locally (ask the LLM for relevance, usefulness? ask the human?)
- query in active learning manner; providing positive and negative examples to orient the direction of better results. It'd silence the polluting components from the original query
- iterate
If anyone's hiring for remote roles, I'd love to give that technique a try
Yet another manycore proposal, but I feel each time we're getting closer. Bandwidth is one of the few dimensions still growing, and there's something to unlock by shaping computation by this firehose of bits.
However it seems the software part is always a blocker, and these architecture only address well a handful of program types.
Is this time different? I think it is. The paradigm here is about small threads that yield very often, and we have that way of programming: Erlang, Go, Java's virtual threads.
JAX is designed from the start to fit well with systolic arrays (TPUs, Nvidia's tensor cores, etc), which are extremely energy-efficient. WebGL won't be the tool that connects it on the web, but the generation after WebGPU will.
I believe we could get there eventually. For example for collision there is work to make it differentiable (or use a local surrogate at the collision point):
https://arxiv.org/abs/2207.00669
The robotics will need to connect vision with motors with haptics with 3D modelling. And to propagate gradient seamlessly. For calibrating torque with the the elastic deformation of the material for example. After all matter is not discreet at small scales (staying above the atomic scale)
All this will require all modules to be compatible with differentiability. It'll be expensive at first, but I'm sure some optimizations can get us close to the discreet case.
Also even for meshes there is a lot to gain with trying to go the continuous way:
I had a lot of fun writing the article! And it is only half a joke
My intuition for so-called world models is that we'll have to plug modules, each responsible for a domain (text, video, sound, robot-haptics, physical modelling) It'll require to plug modules in a way that will allow the gradient to propagate. A differentiable architecture. And JAX seems well placed for this by making function manipulation a first citizen. Looking at your testimony comforts me in this view
It is AI generated. Or was written by someone a bit far from the technical advances IMHO. The Johnson-Lindenstrauss Lemma is a very specific and powerful concept, when in the article the QLJ explanation is vacuous. A knowledgeable human would not have left the reader wanting for how that relates to the Lemma.
> the whole process remains differentiable: we can even propagate gradients through the computation itself. That makes this fundamentally different from an external tool. It becomes a trainable computational substrate that can be integrated directly into a larger model.
IMHO the key point at which this technique has an unfair advantage vs a traditional interpreter is here.
How disruptive is it to have differentiability? To me it would mean that some tweaking-around can happen in an LLM-program at train-time; like changing a constant, or switching from a function call to another function. Can we gradient-descent effectively inside this huge space? How different is it from tool-calling from a pool of learned programs (think github but for LLM programs written in classic languages)?
I'm not well versed into RF physics. I had the feeling that light-wave coherency in lasers had to be created at a single source (or amplified as it passes by). That's the first time I hear about phased-array lasers.
A hard problem, especially wrt to transactions on a moving target.
From memory, handful of projects just dedicated to this dimension of databases: Noria, Materialize, Apache Flink, GCP's Continuous Queries, Apache Spark Streaming Tables, Delta Tables, ClickHouse streaming tables, TimescaleDB, ksqlDB, StreamSQL; and dozens more probably. IIRC, since this is about postgres, there is recently created extension trying to deal with this: pg_ivm
On the brand new and shiny 486 running dos and windows 3.1 that my father bought, on the qbasic language. With only a paper book as reference. No llm, no stackoverflow, no pageranked search engine, no internet, and not even Ctrl+F. In these days when you had a bug, you could chew on it for days.
Of note is that once you've got planes, you can define points as intersections of n hyperplanes.
In 2D, 2 intersecting hyperplanes (=lines here) will define a point.
But what if these lines are parallel? Well you just got the "point at infinity" abstraction for free. And if you defined operators on points as intersections of lines they will also work with the points at infinity.
Also: with a few modifications you get conformal geometry as well; with everything being defined as intersections of spheres. After all, what is a plane but a sphere that has its center at infinity?
> > Just intuitively, in such a high dimensional space, two random vectors are basically orthogonal.
> What's the intuition here? Law of large numbers?
Yep, the large number being the number of dimensions.
As you add another dimension to a random point on a unit sphere, you create another new way for this point to be far away from a starting neighbor. Increase the dimensions a lot and then all random neighbors are on the equator from the starting neighbor. The equator being a 'hyperplane' (just like a 2D plane in 3D) of dimension n-1, the normal of which is the starting neighbor, intersected with the unit sphere (thus becoming a n-2 dimensional 'variety', or shape, embedded in the original n dimensional space; like the earth's equator is 1 dimensional object).
The mathematical name for this is 'concentration of measure' [1]
It feels weird to think about it, but there's also a unit change in here. Paris is about 1/8 of the circle far away from the north pole (8 such angle segments of freedom). On a circle. But if that's the definition of location of Paris, on the 3D earth there would be an infinity of Paris. There is only one though. Now if we take into account longitude, we have Montreal, Vancouver, Tokyo, etc ; each 1/8 away (and now we have 64 solid angle segments of freedom)
Strongly agree with your comment. I wonder now if this "theory building" can have a grammar, and be expressed in code; be versioned, etc. Sort of like a 5th-generation language (the 4th-generation being the SQL-likes where you let the execution plan be chosen by the runtime).
The closest I can think of:
* UML
* Functional analysis (ie structured text about various stakeholders)
A bit late to the discussion, but this has deep connections. As a programmer, your job is provide business invariants using complexity management techniques. And checking that your state space is small is a tool with gigantic payoff.
Maintaining a small state space it why we want to let it crash. Each program instruction can potentially multiply the number of states possible. Erlang even has this whole "Let It Crash" philosophy as a guideline [1].
Maintaining a small state space is how you tame concurrent programs, where adding one thread can cartesian-product your state space. But there are tools like TLA+ which can help you build proofs over this state space. And build invariants that your threads can use safely. Hre is a visualizer of that state space [2]. Notice any resemblance to the graphs you just saw in the video?
Programming sometimes feel like this "Rush Hour" puzzle.
A potential solution would be to have a reputation system, with some light pageranking in it. People who care about their reputation are bound to rate other likewise people higher. I don't think such loyalty exists between vibetalkers.
It's obvious we're not alone, and 2 distinct clusters would form (with people on a spectrum in between)