Quanta Magazine consistently explains mathematics/physics for an advanced lay audience in ways that don't terribly oversimplify / still expose you to the true ideas. It's really nice! I don't know of any other sources like this.
(1) This is awesome. Feels like this wraps enough complexity that it won't just be a toy / for prototyping.
(2) When a schema is provided, is it fully enforced? Is there a way to do migrations?
Migrations are the only remaining challenge I can think of that could screw up this tool long-term unless a good approach gets baked in early. (They're critically important + very often done poorly or not supported.) When you're dealing with a lot of data in a production app, definitely want some means of making schema changes in a safe way. Also important for devex when working on a project with multiple people — need a way to sync migrations across developers.
Stuff like scalability — not worried about that — this tool seems fundamentally possible to scale and your team is smart :) Migrations though... hope you focus on it early if you haven't yet!
I haven’t seen a better solution than remolacha’s #2 (create separate temporary state for the form).
Forms just inherently can have partially-finished/invalid states, and it feels wrong to try and kraal model objects into carrying intermediary/invalid data for them (and in some cases won’t work at all, eg if a single form field is parsed into structured data in the model)
In all the cases in the article it looks like shame plays a big role. I wonder if hikikomori is caused by a loop of [adverse circumstances that cause the person to feel shame] -> withdrawal to avoid shame -> being ashamed of having withdrawn [loop]
How’s PostgreSQL’s code quality? If projects have tons of technical debt or poor abstractions it can often be hard to make significant changes. Is that the case here, or no?
It was certainly an unresolved question before they did this work!
Naively, it seems reasonable to believe that if you adjust all the weights of a neural net towards the behavior you want via SFT and RLHF, that it would compete with/mute/obscure undesired behavior like a back door. But it seems not to be so… Indeed the cute mask does not cover the entire shoggoth— it may still have tentacles (https://images.app.goo.gl/YW9g3BvwGqGwYTgd6)
An LLM can be executed in an “OODA” loop like in AutoGPT and given a goal towards which it takes agentic actions, especially if the LLM is fine-tuned for function calling. So, it can be the main component of an agent that does have goals/de-facto motives! The wrapper code can just be a couple hundred lines.
AutoGPT itself is pretty weak, but it’s possible to write wrapper code that leads to stronger agency. Also, agents formed this way with GPT4 are way stronger than with GPT3.5… so expect this trend to continue.
> But the ways in which a LLM can “talk about” the universe (and everything it contains) are limited to the ways in which humans have previously talked about the universe.
This is often said, but it isn't so.
The task of predicting the next token in human speech really well requires immense intelligence — potentially far more intelligence than possessed by the original speaker! Imagine yourself engaging in the task of listening to someone who isn't that smart speak and then trying to figure out what they'll say next — in doing so, you might make all sorts of extrapolations about the person, their motivations, their manner, their dialect, etc — calling on all sorts of internal models that you've built up about people over time. This is what models are being trained to do when we train them on predicting tokens.
There are concrete examples of models inventing new ways of thinking that are not described in their training set. For example, when training a transformer from scratch to perform addition mod P (and having no training data other than examples of addition mod P), the transformer was able to discover the use of discrete fourier transforms and trigonometric identities [1]. As we can see, neural nets can build all sorts of internal mental models that no one explained to them beforehand. These internal mental models can then be elicited and used for other purposes by e.g. fine-tuning.
I think a good mental model for transformers/neural nets is that they're automatic scientists. They figure out ways of modeling things in order to predict the output from the input — which is what scientists do! As part of this, they can de-facto discover new theories, and come to rely on the theories that prove useful in their prediction task.
Also, not all tokens in the training set are from human speech, so models are being trained to model all manner of data-generating processes.
It's true that many of the people on Jan 6 didn't view themselves as extreme and didn't realize they were being fascist — they probably mostly truly did believe they were protecting democracy — however this does not preclude them being de-facto fascists.
Caro understands his subjects so so well that you feel you are inside their heads.
Caro’s works have spoiled me — other biographies feel out-of-focus and surface-level and speculative.
(And all this despite the fact that Caro isn’t even really focusing directly on his subjects, because actually his books are about power and its dynamics.)