NTK-Aware RoPE allows Llama to have 8k+ context size without fine-tuning(old.reddit.com)
old.reddit.com
NTK-Aware RoPE allows Llama to have 8k+ context size without fine-tuning
https://old.reddit.com/r/LocalLLaMA/comments/14lz7j5/ntkaware_scaled_rope_allows_llama_models_to_have/
1 comments
I am blown away by the pace of OpenSource progress in the LLM space; I've never witnessed something like this before in tech.
Awesome to see that individual enthusiasts are really bringing the field forward and this shows again, how much more potential there is, even without new fundamental breakthroughs...