elexhobby

vor 3 Jahrendiscuss

For another possibility, see https://arxiv.org/abs/2305.15717. The new models may not actually be better - the evaluation may be broken.

elexhobby

vor 3 Jahrendiscuss

GPT-4 is powerful over a diverse set of tasks. They use it to build a model which is better for a narrow sub-task. Pretty sure the model is sub-optimal to GPT-4 for everything else.

elexhobby

vor 3 Jahrendiscuss

Not sure if this is obvious. But its incorrect to ditch on GPT-4. The paper uses self-instruct on GPT-4 to generate the training data on which it is fine-tuned. This paper would not exist without GPT-4. Although they claim GPT-4 can be replaced by any LLM, I'm sure the results would be nowhere as good, and so they stuck with GPT-4.

elexhobby

vor 3 Jahrendiscuss

This! The best resource I've found to explain transformers, that made them clear to me. I wish all deep learning papers were written like this, using pseudocode.

elexhobby

vor 4 Jahrendiscuss

FWIW I've been told similar by Costco too, so Amazon isn't unique in this respect. My guess is that it adds some friction to the process. Refunding money is a purely digital activity, while refunding requires to you go and ship it, wait for the new item to arrive, etc, so you won't do it unless the savings are worth the time investment.

elexhobby

vor 5 Jahrendiscuss

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073

"When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."

elexhobby

vor 5 Jahrendiscuss

any twitch streams you recommend?

elexhobby

vor 6 Jahrendiscuss

Wow. I wish everything was explained so clearly. I understood everything in that post except this paragraph.

""" Languages like Erlang must implement tail call optimizations, since persisted data is stored as "loop variables" in infinite loops. This happens when we write code like this:

     loop(Data) ->
          ....
 ...
 loop(newData).

When I see code like this I mentally "see" the last call as a "jump" to the start of the code, rather than a recursive call to loop. """

What is the loop(Data), loop(newData) doing? Would be great if someone could elaborate on this point.

elexhobby

vor 6 Jahrendiscuss

Quote: "The election comes down to a few swing states, such as Pennsylvania, Wisconsin, and Michigan. Crucially, right now all three of those states have Democratic governors and Republican-controlled legislatures … and there’s no clear law about which of the two (the governor or the legislature) gets to certify election results and send them to Congress! So suppose Trump has a slight edge on election night, Fox News calls the race for him, but then an avalanche of absentee or provisional ballots shift things in Biden’s favor over the following week. Can you imagine Trump or his supporters accepting the latter?"

elexhobby

vor 7 Jahrendiscuss

Great post, thanks!

Is there a reason why the training is started off with two separate matrices - the embedding and the context matrix? If the context matrix is anyway discarded at the end, why not start and work with only the embedding matrix?

Submissions

What if Trump narrowly loses, but refuses to concede?

Ben Recht (optimization professor at Berkeley) starts writing a blog

comments