a large language model is a decoder-only transformer trained on one absurdly simple objective — predict the next token — scaled until the emergent behaviour stops looking simple. 𐃏 this page is the map from that objective to a deployed assistant: the loss, the scaling laws that size the model, the pretrain-align pipeline, and the inference tricks. the hands-on build is nanogpt.
Llm
Context:
Running LLM’s (large-language-models) locally is now possible 1 due to the abundance of highly parallelised compute (GPU’s) at affordable prices and also the advances of Deep Learning in the past decade.
As such, even slightly powerful consumer devices such as my M1 Macbook Pro with 8GB of RAM can run a small LLM. The purpose of this post is to investigate the token speed and accuracy of a variety of LLM’s on my Machines.
Backlinks (2)
1. Wiki /wiki/
Knowledge is a paradox. The more one understand, the more one realises the vastness of his ignorance.