Understanding Context Windows(rkayg.com)

2 pointsby rkayg12 เดือนที่ผ่านมา2 comments

https://www.rkayg.com/blog/understanding-context-windows

1 comments

johndonavon12 เดือนที่ผ่านมา

I think degraded performance as the context window increases is one of the largest issues models are facing today. Do you think we've reached the end of any "RAM" based context performance and that external storage/ retrieval mechanisms is the only way forward?

rkayg12 เดือนที่ผ่านมา

I think we're in the early innings of RAM implementations. I also think RAM and external storage / retrieval are kind of the same thing in that efficient RAM implementation works best w/ another storage system. I hope to do a deeper dive here, but in the mean time check out https://arxiv.org/abs/2310.08560