RNNs Explained: How Neural Networks First Tried to Carry Meaning Forward

There is a quiet difference between seeing a sentence and moving through it. A fixed-window neural language model still sees language a little like a camera sees a street. At each step, it captures a local view. It can be a very intelligent camera. It can learn rich internal transformations. It can make far better guesses than count-based models ever could. But it still works by repeatedly taking a bounded snapshot. Language is not really a snapshot. It is closer to a walk. A sentence does not arrive all at once. It reveals itself in time.