This isn't suggesting no one understands how these models are architected, nor is anyone saying that SDPA / matrix multiplication isn't understood by those who create these systems.
What's being said is that the result of training and the way in which information is processed in latent space is opaque.
There are strategies to dissect a models inner workings, but this is an active field of research and incomplete.
Look at it from an algorithmic perspective. In computer science many algorithms take a non-constant number of steps to execute. However, in transformers models, there are a limited number of decoder blocks, and a limited number of FFN layers in each block. This presents a theoretical upper bound on the complexity of the algorithms a decoder network can solve in a single token generation pass.
This explains why GPT4 cannot accurately perform large number multiplication and decimal exponentiation. [0]
This example can extend to general natural language generation. While some answers can be immediately retrieved or generated by a "cache" / algorithm which exists in latent space, some tokens have better quality when their latent-space algorithm is executed in multiple steps.
Very interested in the expansion of RL for transformers, but I can't quite tell what this project is.
Could you please add links to the documentation to the readme where it states "It includes detailed documentation".
Also maybe DPO should use the DDPG acronym instead so your repos Deterministic Policy Optimization isn't confused for trl's Direct Preference Optimization.
> I’ve done some investigation into this. In a well trained model, if you plot the intermediate output for the last token in the sequence, you see the values update gradually layer to layer. In a model that produces repeating sequences I almost always see a sudden discontinuity at some specific layer. The residual connections are basically flooding the next layer with a distribution of values outside anything else in the dataset.
> The discontinuity is pretty classic overfitting. You’ve both trained a specific token to attend primarily to itself and also incentivized that token to be sampled more often. The result is that if that token is ever included at the end of the context the model is incentivized to repeat it again.
...
> Literally just plotting the output of the layer normalized between zero and one. For one token in mistral 7B it’s a 4096 dimension tensor. Because of the residual connections if you plot that graph for every layer you get a really nice visualization.
> Edit: Here's my visualization. It’s a simple idea but I've never personally seen it done before. AFAIK this is a somewhat novel way to look at transformer layer output.
This is nearly identical to the overfitting example in the repo, only really representing a binary, but it's a good start. Perhaps some transformations can be applied to help further?
Not a material science expert, however per their paper, they use DFT to verify the stability, then use the verification status to improve the model.
>candidate structures filtered using GNoME are evaluated using DFT calculations with standardized settings from the Materials Project. Resulting energies of relaxed structures not only verify the stability of crystal structures but are also incorporated into the iterative active-learning workflow as further training data and structures for candidate generation
Excellent, appears Amazon has introduced two important things here:
- Rope theta of 100,000, likely from the Llama 2 Long paper which found that a large theta helped regulate attention between distant tokens[0]
- A 16k (effective 32k) context window, improving upon Mistrals 4k (effective 8k) context window
In The Llama 2 Long paper, they saw improvement in short context benchmarks as a result of long context fine tuning. I can't find any of the expected MMLU / HellaSwag / etc benchmarks yet. Benchmarks haven't been submitted to MTEB yet.
Some user anecdotally seem to be having trouble with generating quality responses [2][3][4]. I can't find any examples of users getting good results from the model outside of using exact examples from the documentation.
The only thing I wish you did different was explored alpha > 2 * r.
In this blog post, the author found that alpha of 4 * r (where r=64) outperformed all smaller alphas in terms of loss when finetuning Llama-7b on databricks-dolly-15k.
Additionally you identify (alpha = 2*r) r=16 as inferior to r=256, however aside from arithmetic, r=16 actually outperforms all others. And the base model outperforms any finetuning for both arithmetic metrics.
"For example, HyperAttention makes the inference time of ChatGLM2 50% faster on 32k context length while perplexity increases from 5.6 to 6.3."
"when half of all attention layers are patched (i.e., 14 layers), we verify that most of the tasks do not degrade more than 13%."
According to the paper, for most tasks it reduces benchmark scores substantially. Perhaps to the point where a smaller model would yield better inference time and higher benchmarks.
However, summarization benchmarks see almost no degredation, great!
This seems half-baked and there are numerous faulty assumptions in this article.
For example, Bitcoin miners cannot computer gradients. Their ASICs can only calculate double-sha256.
Additionally, the premise of sending gradients of models trained on private data while retaining privacy seems problematic. While you likely can't reverse it to calculate the batch's contents, it is leaky.
Further, gradient calculations are not a good proof of work. A good proof of work is difficult to calculate and easy to verify (i.e. an extremely low hash value).
The core premise, using blockchains to store terabytes of models and datasets, doesn't make any sense whatsoever.
The problems highlighted in this article however are valid, and it would be great to see something like IPFS for datasets and models.
Interesting how this method quantizes different layers / modules in a manner that minimizes perplexity as it adjusts parameters.
I'd be interested to see how 2.5 bit quantization compares to an unadjusted 4-bit baseline. Additionally would be interesting to see the usual benchmarks (ARC, HellaSwag, MMLU, TruthfulQA) of this method at different average bitrates (2.0, 2.5, 3.0, 4.0, ...). Would also be interesting to see if an average bitrate of 4 is just as fast and small as a constant bitrate of 4, but more accurate.
Very exciting work, looking forward to trying this out on my models!