TransformerFAM: Feedback attention is working memory
Yannic Kilcher · 37:00 · Watch on YouTube
The video argues that Google's "TransformerFAM: Feedback Attention is Working Memory" paper is basically a recurrent neural network (RNN) that works on blocks of tokens instead of single tokens, described in neuroscie...