https://abaj.ai/tags/sampling/Recent content in Sampling on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajFri, 10 Jul 2026 08:15:43 +1000https://abaj.ai/wiki/ml/dl/natural-language-processing/llms/Fri, 10 Jul 2026 01:42:07 +1000https://abaj.ai/wiki/ml/dl/natural-language-processing/llms/<p>a large language model is a decoder-only <a href="https://abaj.ai/wiki/ml/dl/transformers/" >transformer</a> trained on one absurdly simple objective — predict the next token — scaled until the emergent behaviour stops looking simple.<span class="margin-note" data-note="everything downstream — chat, code, reasoning — is a fine-tune or a prompt on top of this one objective"> <span class="margin-note-indicator">𐃏</span> </span> this page is the map from that objective to a deployed assistant: the loss, the scaling laws that size the model, the pretrain-align pipeline, and the inference tricks. the hands-on build is <a href="https://abaj.ai/wiki/ml/dl/transformers/nanogpt/" >nanogpt</a>.</p>