https://abaj.ai/tags/graphical-models/Recent content in Graphical-Models on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajMon, 31 Aug 2026 00:29:55 +1000https://abaj.ai/wiki/ml/pgm/Mon, 31 Aug 2026 00:29:55 +1000https://abaj.ai/wiki/ml/pgm/<p>a probabilistic graphical model is a marriage of graph theory and probability: the graph records which variables talk to each other directly, and that structure alone dictates how compactly a joint distribution can be stored, which independencies it must satisfy, and how expensive it is to reason with. one set of ideas — factors, elimination, message passing, sampling — then serves diagnosis networks, spam filters, speech recognisers, image denoisers and gene-regulation maps alike.</p>https://abaj.ai/wiki/ml/pgm/approximate-inference/Mon, 31 Aug 2026 00:29:54 +1000https://abaj.ai/wiki/ml/pgm/approximate-inference/<p><a href="https://abaj.ai/wiki/ml/pgm/inference/" >exact inference</a> costs \(\exp(\text{treewidth})\), and real networks — image grids, dense diagnostic models, unrolled <a href="https://abaj.ai/wiki/ml/pgm/markov-models/" >dbns</a> — routinely have treewidth in the hundreds. approximate inference trades guaranteed answers for tractable ones, along two philosophically different roads: <strong>sampling</strong> replaces the distribution with draws from it (anytime, asymptotically exact, embarrassingly parallel), and <strong>approximate message passing</strong> runs exact-style propagation on graphs where its correctness proof no longer holds. this page walks the sampling ladder in the order each rung fixes the last one&rsquo;s failure — forward, rejection, likelihood weighting, gibbs — then closes with loopy belief propagation.<span class="margin-note" data-note="comp9418 benchmarks all four samplers against variable elimination on the icu-alarm network; the qualitative ranking below is what those experiments show"> <span class="margin-note-indicator">𐃏</span> </span> </p>https://abaj.ai/wiki/ml/pgm/bayesian-networks/Mon, 31 Aug 2026 00:29:54 +1000https://abaj.ai/wiki/ml/pgm/bayesian-networks/<p>a joint probability distribution over \(n\) binary variables is a table with \(2^n - 1\) free numbers. at \(n = 30\) that is a billion parameters for a toy problem — nobody can elicit them, store them, or estimate them from data. the bayesian network is the classical answer: encode the <em>independencies</em> of the distribution in a directed acyclic graph, and the joint collapses into a product of small local tables.<span class="margin-note" data-note="the treatment here follows darwiche&#39;s book and unsw&#39;s comp9418, which is taught from it"> <span class="margin-note-indicator">𐃏</span> </span> the graph is simultaneously a data structure for storage, a calculus for reading off independence, and the substrate on which every <a href="https://abaj.ai/wiki/ml/pgm/inference/" >inference algorithm</a> runs.</p>https://abaj.ai/wiki/ml/pgm/inference/Mon, 31 Aug 2026 00:29:54 +1000https://abaj.ai/wiki/ml/pgm/inference/<p>a <a href="https://abaj.ai/wiki/ml/pgm/bayesian-networks/" >bayesian network</a> stores a joint distribution; inference is spending that investment — computing probabilities the tables do not state directly. exact inference is np-hard in general, but its true cost is exponential only in a structural parameter of the graph (the <em>treewidth</em>), not in the number of variables. this page builds the exact toolchain in the order the ideas force themselves on you: factors, variable elimination, the fight for a good elimination order, and finally the jointree algorithm — variable elimination industrialised into a message-passing scheme that answers every marginal at once.<span class="margin-note" data-note="when the treewidth is too large, exact methods drown and you switch to the sampling algorithms on the approximate inference page"> <span class="margin-note-indicator">𐃏</span> </span> </p>https://abaj.ai/wiki/ml/pgm/markov-models/Mon, 31 Aug 2026 00:29:54 +1000https://abaj.ai/wiki/ml/pgm/markov-models/<p>time is just another dimension to factorise over. a <a href="https://abaj.ai/wiki/ml/pgm/bayesian-networks/" >bayesian network</a> whose variables are indexed by time steps, with the same local structure copied at every step, is a <em>dynamic bayesian network</em> — and its two simplest members, the markov chain and the hidden markov model, carry a startling share of applied probability: language models before transformers, speech recognition, robot localisation, pagerank, and the mcmc machinery behind <a href="https://abaj.ai/wiki/ml/pgm/approximate-inference/" >approximate inference</a>.<span class="margin-note" data-note="the modern neural take on the same sequence problem is the rnn — same graph, learned transition function"> <span class="margin-note-indicator">𐃏</span> </span> </p>https://abaj.ai/wiki/ml/pgm/markov-networks/Mon, 31 Aug 2026 00:29:54 +1000https://abaj.ai/wiki/ml/pgm/markov-networks/<p>some dependencies have no natural direction. two neighbouring pixels tend to share a label; two friends tend to vote alike; two adjacent atoms couple their spins. forcing an arrow onto these symmetric interactions — as a <a href="https://abaj.ai/wiki/ml/pgm/bayesian-networks/" >bayesian network</a> must — misrepresents them. the <strong>markov network</strong> (markov random field, mrf) is the undirected alternative, born in statistical physics: edges express mutual compatibility, parametrised not by conditional probabilities but by unnormalised <em>potentials</em>, with a global normalising constant picking up the bill.<span class="margin-note" data-note="the trade is exactly that: local, symmetric, easy-to-change factors — in exchange for a partition function that is np-hard to evaluate"> <span class="margin-note-indicator">𐃏</span> </span> </p>