A language model is a probability distribution over sequences of tokens. Given a sequence of words or subword units, the model assigns a probability to each possible next token, enabling it to generate coherent text, rank candidate sentences, or complete partial inputs. This core definition applies to every generation of language models, from simple n-gram tables to the large neural systems used today.
The practical utility of this probabilistic framing is significant. When a model assigns high probability to the word Paris after the phrase The capital of France is, it is doing so because it has learned statistical regularities from large amounts of text. The model has internalized patterns of co-occurrence, syntax, and world knowledge encoded in its training data.
Language models can be used for two broad purposes. The first is scoring: given a complete sentence, the model can compute how likely that sentence is. This is useful in spell checking, grammar correction, and machine translation reranking. The second purpose is generation: given a partial input (called a prompt), the model samples or selects tokens one at a time to produce an output sequence.
Generation works through autoregression. The model generates one token, appends it to the input, then generates the next token conditioned on the extended sequence. This continues until a stopping condition is reached, such as a special end-of-sequence token or a maximum length limit. The quality of the output depends entirely on the quality of the learned probability distribution.
Large language models (LLMs) are language models trained on vast corpora using deep neural networks with hundreds of millions to hundreds of billions of parameters. The word large refers to scale: large data, large model size, and large compute budgets. This scale is what enables LLMs to exhibit behaviors like reasoning, summarization, translation, and code generation without task-specific training.
Understanding this foundational definition prevents common misconceptions. LLMs do not retrieve information from the internet in real time unless explicitly given a tool to do so. They do not understand language the way humans do. They learn to produce statistically plausible continuations of text, and from that capability, many useful behaviors emerge.
A customer service team at a mid-size SaaS firm uses an LLM to generate draft replies to support tickets. When a user writes My account is locked and I cannot log in, the model assigns high probability to a response beginning with an acknowledgment and a troubleshooting step. The model does not look up the user's account; it generates text that matches the pattern of helpful responses it saw during training. Understanding this distinction helps the team design a system that combines the LLM's fluency with a separate database lookup for account-specific data.
- A language model assigns probabilities to token sequences and predicts the next token given prior context.
- Generation works autoregressively: one token at a time, conditioned on all previous tokens.
- LLMs are language models trained at large scale on massive text corpora using deep neural networks.
- LLMs generate statistically plausible text; they do not retrieve live data or understand language consciously.
- The two core uses are scoring (how likely is this text?) and generation (what comes next?).