Attention Mechanism Visualizer - Deep Learning Tool

What is Attention?

Attention is a mechanism that allows neural networks to focus on relevant parts of the input when processing each element. It's the key innovation behind Transformers, powering models like GPT and BERT.

Key concepts:

Query (Q): "What am I looking for?" - represents the current word
Key (K): "What do I contain?" - represents each word's content
Value (V): "What information do I provide?" - the actual information to extract
Attention Scores: How much each word should focus on every other word
Self-Attention: Words in a sentence attending to each other

Controls

Enter a sentence to begin

Enter Your Sentence:

Sample Sentences:

Instructions:

1. Enter or select a sentence
2. Click "Calculate Attention"
3. Click on any word to see what it attends to
4. Darker lines = stronger attention

Attention Visualization

Enter a sentence and click "Calculate Attention"

Attention Matrix

Query (Q)

"What am I looking for?"

Represents the word asking the question

Key (K)

"What do I offer?"

Represents what each word contains

Value (V)

"What info do I give?"

The actual information passed along