How RNNs Read Sequences

Drag the slider to rearrange words. Word order matters — "not good" is negative, but "good not" is confusing. Before an RNN can read text, each word must first be converted to a numeric embedding vector. The RNN then reads these embeddings one at a time, left to right, building up meaning sequentially.

Step through to see the full RNN calculation with real numbers. At each step: multiply the previous hidden state by W_h, the word embedding by W_x, add them, and apply tanh. Click the element buttons to expand each dot product. Watch how "not" flips the hidden state negative.

When "unrolled," we see the same RNN cell repeated at each step. All copies share identical weights — this is what makes RNNs parameter-efficient. The sentiment bar shows how the internal state evolves as each word is read.

Increase the depth to see gradients vanish exponentially. At each step, gradients are multiplied by ~0.55 — after 6 steps the first word's gradient is 0.0503. Early words can't learn. This is the vanishing gradient problem.

Drag each gate slider to see its effect. Forget gate at 0 erases old memory; at 1 it keeps everything. Input gate controls how much new info is written. Output gate filters what gets passed to the next step. Formula: c_t = f·c_(t-1) + i·candidate, h_t = o·tanh(c_t).

Step through each word to watch the RNN build its verdict. After reading the entire sequence, the final hidden state is converted to a prediction via softmax. Watch how "not" before "good" makes the RNN output Negative.