https://abaj.ai/tags/dense-embeddings/Recent content in Dense-Embeddings on Aayush Bajaj's Augmenting InfrastructureHugoen© 2026 Aayush BajajThu, 09 Jul 2026 21:02:19 +1000https://abaj.ai/wiki/ml/dl/natural-language-processing/rags/Thu, 09 Jul 2026 21:02:56 +1000https://abaj.ai/wiki/ml/dl/natural-language-processing/rags/<p>retrieval-augmented generation bolts a search engine onto a language model: fetch relevant documents at query time and paste them into the prompt, so the model answers from evidence rather than from its frozen weights (lewis et al. 2020, <a href="https://arxiv.org/abs/2005.11401" class="link--external" target="_blank" rel="noreferrer" ><em>retrieval-augmented generation for knowledge-intensive nlp</em></a>).<span class="margin-note" data-note="the cheapest way to give an llm new knowledge without retraining it"> <span class="margin-note-indicator">𐃏</span> </span> it is the pragmatic alternative to baking every fact into an <a href="https://abaj.ai/wiki/ml/dl/natural-language-processing/llms/" >llm</a>&rsquo;s weights.</p> <h2 id="why-retrieve-at-all">why retrieve at all<a href="#why-retrieve-at-all" class="post-heading__anchor" aria-hidden="true">#</a> </h2> <p>three problems that no amount of scaling fixes cleanly, and retrieval fixes cheaply:</p>