Recurrence

feedforward networks eat fixed-size vectors. sequences — text, audio, sensor streams — have no fixed size, and worse, their order carries the meaning. the recurrent neural network solves both problems with one idea: maintain a hidden state that is updated by the same function at every time step. 𐃏 parameter count stops depending on sequence length, and the state becomes a lossy summary of everything seen so far (Goodfellow, Ian, 2016).