Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Yannic Kilcher · 37:16 · Watch on YouTube
The video walks through Google's "Leave No Context Behind" paper on Infini-attention, which claims transformers can handle unbounded context with fixed memory. The presenter concludes it is mostly linear attention plu...