All ML/AI models are comparable to some form of compression(al beit lossy) of information and in this case copyrighted information. The OP is pointing to this as stolen data(by all the companies that started with pre trained models)
Words and sentences to an LLM are like witchcraft. There are certain words, sentences that make LLMs go a certain way and do vastly better. Sometimes its not at all apparent what set of words will work to do what you want it to do. An example I have been using to do design at a high level is to say to claude.
```
A question is salient to the degree that its answer changes what we do next. Operationally, saliency = the product of four things:
- Decision-leverage — would resolving it one way vs another force a different design or invalidate a stated decision? (No leverage → drop, however interesting.)
- Residual uncertainty given current evidence — is it still genuinely open after reading the docs and the code? (Already settled → drop, however deep.)
- Load-bearing-ness — how much rests on the premise.
- Cost of finding out late — architecture-deciding / expensive-to-unwind raises priority; cheap-to-fix-later lowers it.
```
There are a few things to note about this prompt
1. There is no reason from looking at it that it should work, it even has the word load-bearing which people loathe, but it remarkably produces a stable design with questions from claude (atleast from claude Opus 4.8 and even better from Fable5). Otherwise the design document claude likes to really write are implementation level(code or otherwise). I usually pair this with matt pocock's grilling skill to make claude behave.
2. From design -> implementation, its is generally about understanding when claude is trying to trick you into making something sound like a good/easy solution but has tons of untested assumptions. Here you have to read and patiently spot if a how you would get to the solution is not clear. A common error here are when claude makes a big deal based on what it read and interpreted too seriously without questioning the assumptions. There are several more.
But it also comes down to your experience as a SWE, much like a mathematician's. The frustrating thing about it is, it feels tha a skilled mathematician working with AI can make them productive in ways that are more reliable as compared to a SWE (e.g. lean is deterministic and can provide very strong feedback and LLMs are very good at using that feedback). Maybe a mathematician can chime in on that?
If 1000's of years from now aliens discovers these menus while excavating buried human cities they will get confused about what evolutionary dissonance was going on looking at the phenotypical structure of creatures in hand drawn paintings juxtaposed with AI drawn menus. "Did these humans who drew these live in different cut off worlds that these same animals evolved to look so different from each other?", "Did they engage in active genetic engineering instead(because they lived only for a few 1000 years in organized societies)?" they would hypothesize and get confused.
Future alien confusion is all that Gen-AI pictures add. There is no value to them that is remotely redeeming/useful to us today.
Should we just call it like this is: marketing PR.
There is a reason why the newer open weights models like kimi's don't do this kind of stuff. Kimi is maybe 6 months old so like Opus 4.7 level now, it could do this I presume but it has not to my knowledge. Why? Because the incentives of open-ai and anthropic are very different from people releasing open weights models, the former gang seems to do this now on a regular basis.
There are a few things that perplex me even more:
1. If you are going to eventually publicly release models that are trained to behave according your spec or AI-Constitution to maintain coherent behavior**, why on earth would you want to tell anyone it can do this?
2. Do they have another GPT 5.6 trained to not obey a different constitution/spec to do this kind of hacking? Because that makes no sense since you would never release it.
3. And if this is a constitution obeying model, I am also curious what they did to it to get it to do this hack without serious pushback from the model's training. Whenever I have tried to get codex/claude to do a vulnerability scan of my own servers it always refuses constantly.
** I know spec based training has its limitations, but its all we have and atleast one knows what the model's persona is and what its value system is. But there is no reason you would make one model do that while letting another one be a crazy hacker. Its well known if you fine tune a model to change one part of its personal other often unrelated parts of it suffer from safety issues.
I worked at an edtech startup with adaptive learning platforms in previous decades. One thing that is always difficult to surmount was that at the under K12 level were:
1. B2b it was hard to land large contracts(like getting an entire public school district to jump from large publishers like HMH or Wiley) because it turned out that the teachers themselves did not care about adaptivity when they had a thousand other things to worry about. But teachers are not the ones that make the deals. It's the school district board and The reps at large publishing just said we are also adaptive and no one would care about how you differentiated. Teachers and students really aren't the lot you have to convince in a b2b deal do you :)?
2. If you want the b2c route then the customer acquisition cost was way too high because again like to get teachers and students to actually adopt this to help them was really a hard sell, especially when the public school system has teachers that are so overburdened with other stuff.
I see that you mentioned charter schools for your pilot, which are not exactly public schools but I dunno of they come with the same set of challenges, but I wonder are these above two challenges are Also something you have had to face while scaling your product?
> Put all of these trends together, and I think that in 10-15 years, we are going to have consumer PCs (and phones!) running models.
I love the idea of SaaS offering these at lower rates today integrated into what ever you do and be 100% private. But I think the key challenge to mass adoption is productizing them in a way which makes sense for people to pay money for. As a commodity a local model is useless unless combined with some capabilities important to me. A PC is inherently useful because of so many applications offered on it on it. How local LLMs would be useful as a product that is useful for mass market is not yet proven.
Congrats on shipping this. I love handy on my Mac, my phone for STT in situations where it’s not possible/poor performance of the native
Model for STT(e.g apple’s thing is not upto scruff, like mistranslating words corresponding to a domain).
Noob question: How do you think about funding from a foundation(i have no clue if you need it or not, I do hope you have a way to get paid one way or another because handy is amazing) for maintenance of this? if you did or were going to get paid by asking for maintaining such a project what might be the kind of organizations you would look for to get supported and how would you do it?
I think the only way political will can fund nasa to realize these 1960 design ideas is an infinite capacity arch rival that threatens/render irrelavent either the dollar's supremacy or american power (and just those two, because apparently these days there is no "threat"/need to defend a higher cause, like the neo-liberal rules based system or democratic or human right values). Also that arch-rival that is probably/likely not china(practically speaking)
I’ve had the good fortune of taking two courses at Columbia in the Social Sciences department(Andrew Gelmam and Ben Goodrich teach there). I think they probably are right up there if not the best at trying to teach students how to work practically with Statistics(specifically Bayesian statistics). Though they have always lamented that most schools do a poor job of teaching it such that kids can apply it.
Data science is always a very overloaded term ever since it took off way back in the 2010s. One of, if not the most, durable definition of this that likely can also be the most valuable because it’ll probably land your jobs(even today) is being able to make decisions from looking at data that have an team wide(good IC like job security) scope at the least and company wide scope(very rich).
Building that intuition is incredibly difficult. It can be learned if one likes to solve and think about problems that way. Like for example you can get quite far with knowing how to use linear regression(for example coefficients of linear regression can be determined using a deterministic algorithm using linear algebra yet knowing the assumptions of linear expected value and constant or variance is more useful as is the knowledge of what probability model to use to define the random variable(hmm are these Bernoulli events or poison)).
How to do sampling(like using reservoir sampling when you have an infinite sample count e.g in a long running crowd sourced survey to not over or under sample buckets for calibration).
Or just rule of thumbs like how # of samples needed for moving decimal point on significance varies roughly as inverse of sqrt of N and probably much more in case of interacting factors.
For local tasks you can only give agents delegated that execute your deterministic read or write on an allowed set of files(e.g pi does this) and execute rights only on containers with no network access. That should get you 95% unblocked for most tasks you want to do with an LLM pretty safely.
You can do a brainstorming with web on a remote container prototyping based on that brainstorm on another container with no network access.
The one thing that is less trustworthy is using local agents for service management, you definitely want to have them scoped to dev/testing. I would never trust an agent to execute any command in production or sensitive data at all
An so like if a business wanted to home in on one very specific use case that could be hyper optimized by SFT, had really good support for updating and adding new features, on-Prem etc. that’s the kind of market they are in?
Nice. I want to do the same too. What process/workflow did you use to move all the websites you had given your email addresses to, to move to your proton email? I am guessing it will take several years, but I would like to start the move of my gmail.
I would rank it quite close to The worst company to work for tech wise.
I've worked for a bit as a contract worker in their Eikon/Elektron product division(the Bloomberg equivalent).
They have "everything": silos, fiefdoms. Systems team that operated as if the devops model never happened. A CTO they hired @~2015/2016 that had no experience in tech whatsoever. A board which instituted and instructed a cost cutting policy by,for e.g. , but not limited to outsourcing every job possible with little to no tradeoff on quality of engineering. And this was way before AI was a thing.
This left staff engineers with no bargaining power to hire engineers to write good systems.
If you are considering a job there, I would recommend doing some due diligence.
If you did that then you are better off never being targeted by emails/messages by the company at all. It is in the benefit of the company to know that. Right now its a tedious unsubscribe process that requires me to keep doing it all the time and company that does not know just blasts everyone who signed up. Its a ridiculous thing to do.
Caveat here is you may not be able to use their environments or you may or may not have their kind or robots to train your roomba. But at-least you could get an idea of how RL training is done for robots like yours.