Backpropagation-Through-Time on Aayush Bajaj's Augmenting Infrastructurehttps://abaj.ai/tags/backpropagation-through-time/Recent content in Backpropagation-Through-Time on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajThu, 09 Jul 2026 21:02:28 +1000Recurrent Neural Networks (RNNs)https://abaj.ai/wiki/ml/dl/rnn/Thu, 09 Jul 2026 21:02:56 +1000https://abaj.ai/wiki/ml/dl/rnn/<p>feedforward networks eat fixed-size vectors. sequences — text, audio, sensor streams — have no fixed size, and worse, their <em>order</em> carries the meaning. the recurrent neural network solves both problems with one idea: maintain a hidden state that is updated by the same function at every time step.<span class="margin-note" data-note="one function, applied forever: an rnn is a learned dynamical system, and every pathology of dynamical systems — fixed points, chaos, sensitive dependence — shows up in training"> <span class="margin-note-indicator">𐃏</span> </span> parameter count stops depending on sequence length, and the state becomes a lossy summary of everything seen so far (Goodfellow, Ian, 2016).</p>