> 1. The AI answers many questions, some of which are potentially harmful, and generates first draft answers.
...
> 4. The system repeats this process until it collects a large dataset of first draft answers, and rewritten more-ethical second-draft answers.
> 5. The system trains the AI to write answers that are less like the first drafts, and more like the second drafts.
Actually, there are 2 separate models involved in this finetuning step: AI in steps 1-4 is a pre-existing LLM trained to converse and follow instructions (I call it "advisor"). The second AI starts out as a pre-trained LLM (so only trained to provide likely completions on a general dataset), and is trained in step 5 to generate completions that look more like the dataset constructed in 1-4.
So this process is using the helpfulness in the adviser to bootstrap additional values onto a new model.
> What if you had overseer AIs performing Constitutional AI Feedback on trainee AIs, or otherwise tried to separate out the labor?
That is literally what the paper is about, though its a bit hidden. First paragraph of 4.1 Methods:
"We continue to utilize human feedback labels for helpfulness as in prior work, but replace human feedback
labels with model feedback labels for harmlessness. That is, instead of asking crowdworkers to provide
comparison labels for harmlessness, we simply present the same task to an independent model, called the
feedback model (typically a pretrained LM). Once the desired comparison labels are obtained, the remainder
of the training pipeline (i.e., preference model training and RL) is exactly the same as RLHF."
> Factually, its first draft made a best guess (prediction) of what’s accurate.
A well pre-trained LLM makes every word a good guess of what the authors of its training set would say, following the words that preceded... in a different context, possibly a few years ago, possibly as part of a web-fiction. "accurate" is the wrong word here.
To make an LLM prefer, when answering factual questions, to rely on factual sources and approximate the real world, you have to use an additional process that does so. RLHF and CAI might be part of the solution to that, or not, depending on the feedback they provide.
Dividends lower stock price by moving cash from company to owner, buybacks increase stock prices[1] by de-diluting, so only one of those is evidence of the CEO (whose compensation is often tied to stock price) acting on his own interests. Also taxation is different.
That sounds like an argument about "if one's time is worth less than minimum wage, one cannot find a job". And sure that is true for some people, but saying it benefits _only_ ... sounds like it is never good for any worker, which is way too strong a conclusion, because other situations exist.
For example: hiring one of two candidates, Nick and Joe, would be profitable under 30$ per hour. Nick asks for 15$, its Joe's first job so he's willing to take 10$, I go to Nick and say, "look Joe will do it for 10, so I really can't justify paying more, but I'd prefer to pay you those 10, since you're experienced". If Nick has a better offer elsewhere, he has no problem. If not, Nick gets just 10.
If the state says 15 is the minimum wage, not only Nick but also Joe must get 15, so my best move is to take Nick at 15. Clearly, a higher minimum wage _can_ benefit the worker, and not only the very weakest one.
Capitalism allows each participant to seek only the best available deal that is agreeable to them, sure, but availability is subject to negotiating power, and that is distributed very unevenly.
PhD in Machine Learning, on O-1 based on research. EB1B got denied (I have enough accomplishments, company accomplishments excluded for weird reasons). O-1 up for renewal next year.
1. How often are O1 extensions approved?
2. Am working for the same company, which does not publish its research, does that hurt chances of extension?
3. How much harder is EB1A standard than EB1B?
Of course the coverage by libraries, accessibility of varied example code have improved tremendously, as have version control (es. distributed).
But I would say that the amount of code to put something on the screen, speed of compilation and running (non-cpu bounded) code, quality of debugging tools are all approximately catching up now to the state of the art of 1980: a Smalltalk environment created by Alan Kay, Dan Ingalls and friends at Xerox PARC (I've heard LISP workstations were similarly advanced). By their work at VPRI, they still have a few things to invent and for industry to learn.
So while I agree barriers have decreased a lot, it is important to remember that the gap between state of the art what is common is also commonly huge in this area.
Arguably, _introducing_ people to programming through C++ and R (complex languages with difficult tooling) adds about 30 years worth of barriers right back, and is still common practice in some places.
I won't argue about definitions, except to say I am following [1], and suggest you do the same (because it is a use of words that matches their general meanings, not because it is there).
Unless taxation and reporting are made trivial (which is hard without giving up on privacy including cash), the poorest do not have the money or time to jump through hoops. Hence UBI, having the minimal requirements, will help the most of the poorest. This more than compensates for the (arguable, but I'll concede it) bug of giving some money to the rich as well.
Most markets are for items there are many of. There are lots of apples, people see friends enjoying them, people buy one, like it, buy another, a price is established. Key facts: you get to buy it more than once, more than one person gets to buy it, you can tell in advance it is worth something. Take too many of those away, and the market goes away.
An idea (for a venture) is: unique (a close variant missing a key part has completely different value), you would not pay for it twice, and selling it to multiple people diminishes its value. Why would you think the value of an idea would be well captured in a market?
You do realize a market is just one mechanism for expressing value, right? or is air value-less?
It is a job of the government to counteract destabilizing influences such as the wealth concentrating effects of technology and under-competition. As a foreigner living in the US, the US government honestly sucks eggs at it. Healthcare, banks, cable, broadband, cell phone service, Google, Apple, Microsoft... all classic antitrust cases across the board, waiting for an un-captured regulator.
If world governments continue to fail also at corporate tax, the current dystopic situation of regressive effective corporate tax rates through profit export will persist, and risks everything we have gained through capitalism and democracy.
I encourage you to read the definition of UBI. Universal means you give to everyone, from the homeless guy on the street, to Bill Gates. Not just laid off people. Also, your top-of-head speculations are not the best evidence available for its effects.
Next move: Finding a job in industry (tech or finance in NYC or Bay area), solving hard problems using ML, so I can wrestle with the world instead of with reviewers, and be part of a team again.
seL4 and similar applications of coq proof assistant. Existing academia is funding some development of tools like coq, and also some core applications like seL4, and more accessible applications will happen as open source. But we need to make these techniques more accessible for larger applications that will affect the real world.
Example: A webserver proved to respond only to correctly authenticated clients (covering the end to end: kernel, server, protocols, for every client).
Also fund making this research more accessible: try, compare, file usability bugs on, and then highlight the different technologies for achieving trust. For example Haskell Liquid types as a lighter weight complement to proving stuff in coq. Maybe fund research meshing these techniques with languages that provide different type-system guarantees, such as Rust.
Actually, there are 2 separate models involved in this finetuning step: AI in steps 1-4 is a pre-existing LLM trained to converse and follow instructions (I call it "advisor"). The second AI starts out as a pre-trained LLM (so only trained to provide likely completions on a general dataset), and is trained in step 5 to generate completions that look more like the dataset constructed in 1-4.
So this process is using the helpfulness in the adviser to bootstrap additional values onto a new model.
> What if you had overseer AIs performing Constitutional AI Feedback on trainee AIs, or otherwise tried to separate out the labor?
That is literally what the paper is about, though its a bit hidden. First paragraph of 4.1 Methods:
"We continue to utilize human feedback labels for helpfulness as in prior work, but replace human feedback labels with model feedback labels for harmlessness. That is, instead of asking crowdworkers to provide comparison labels for harmlessness, we simply present the same task to an independent model, called the feedback model (typically a pretrained LM). Once the desired comparison labels are obtained, the remainder of the training pipeline (i.e., preference model training and RL) is exactly the same as RLHF."