<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Bellman on Aayush Bajaj's Augmenting Infrastructure</title><link>https://abaj.ai/tags/bellman/</link><description>Recent content in Bellman on Aayush Bajaj's Augmenting Infrastructure</description><generator>Hugo</generator><language>en</language><copyright>© 2026 Aayush Bajaj</copyright><lastBuildDate>Thu, 09 Jul 2026 21:02:28 +1000</lastBuildDate><atom:link href="https://abaj.ai/tags/bellman/index.xml" rel="self" type="application/rss+xml"/><item><title>Q-Learning</title><link>https://abaj.ai/wiki/ml/reinforcement-learning/q-learning/</link><pubDate>Thu, 09 Jul 2026 21:02:56 +1000</pubDate><guid>https://abaj.ai/wiki/ml/reinforcement-learning/q-learning/</guid><description>&lt;p>q-learning is the algorithm that made reinforcement learning feel inevitable: interact with an unknown world, nudge a table of numbers after every step, and the table converges to the value of &lt;em>optimal&lt;/em> behaviour — even while you behave suboptimally the entire time.&lt;span class="margin-note" data-note="watkins introduced it in his 1989 phd thesis; watkins and dayan supplied the convergence proof in 1992">
 &lt;span class="margin-note-indicator">𐃏&lt;/span>
&lt;/span>

everything runs on one line of arithmetic, and the rest of this page is the machinery needed to say precisely why that line works. the canonical reference for all of it is sutton &amp;amp; barto&amp;rsquo;s &lt;em>reinforcement learning: an introduction&lt;/em>, free at &lt;a
 href="http://incompleteideas.net/book/the-book-2nd.html"
 
 
 class="link--external" target="_blank" rel="noreferrer"
 
>http://incompleteideas.net/book/the-book-2nd.html&lt;/a>.&lt;/p></description></item></channel></rss>