Understanding Seq2Seq Neural Networks – Part 6: Decoder Outputs and the Fully Connected Layer

In the previous article, we were looking at the embedding values in the encoder and the decoder. As you can see, they have different input words and symbols (tokens) and different weights, which result in different embedding values for each token. Because we have just finished encoding the English sentence “Let’s go,” the decoder starts with the embedding values for the token. The decoder then performs computations using two layers of LSTMs, each with two LSTM cells.