Keys, Queries, and Values: The celestial mechanics of attention
Luis Serrano Academy · 51:57 · Watch on YouTube
Luis Serrano explains the attention mechanism with a gravity analogy: words that matter to each other in context "pull" each other's embeddings closer. The key and query matrices decide how strongly each word pulls an...