https://abaj.ai/tags/pac-learning/Recent content in Pac-Learning on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajFri, 13 Nov 2026 18:03:57 +1100https://abaj.ai/wiki/ml/theory/learning-theory/Fri, 13 Nov 2026 15:00:00 +1100https://abaj.ai/wiki/ml/theory/learning-theory/<p><a href="https://abaj.ai/wiki/ml/theory/no-free-lunch/" >No free lunch</a> says a learner with no inductive bias cannot generalise. Computational learning theory turns that into arithmetic: restrict the hypothesis space \(H\), and the number of examples needed to generalise grows with the logarithm of \(|H|\), or with its VC dimension when \(H\) is infinite. A second strand counts mistakes instead of examples, for learners that predict online.</p> <h2 id="the-setting">the setting<a href="#the-setting" class="post-heading__anchor" aria-hidden="true">#</a> </h2> <p>Training minimises the empirical risk: pick \(h \in H\) with the smallest training error. With \(H\) unrestricted this overfits, since a look-up table has zero training error. Learning theory asks when a restricted \(H\) makes empirical-risk minimisation safe.</p>