Why can’t LLMs just LEARN the context window?
bycloud · 12:26 · Watch on YouTube
TTT-E2E tries to sidestep quadratic full attention by writing older context into a transformer’s MLP weights with test-time gradient updates, while a fixed sliding window attends only to the latest batch of tokens. On...