ML//large language model

A large language model is a neural network trained on text to learn a distribution over token sequences, which it uses to predict what should come next, and it is the generating core of the products built around it. GPT, Claude and Llama are families built around this mechanism, although the products carrying those names usually add retrieval, tools, policy layers and conventional software.

A large language model is a neural network trained on text to learn a distribution over token sequences, which it uses to predict what should come next, and it is the generating core of the products built around it. GPT, Claude and Llama are families built around this mechanism, although the products carrying those names usually add retrieval, tools, policy layers and conventional software.

It is a transformer trained to predict the next token (a fragment of text cut by a tokenizer) over enormous amounts of text, by minimizing cross-entropy, and then tuned to follow instructions, partly with RLHF. Sizes run from a few billion parameters to hundreds of billions; a 7-billion model quantized to 4 bits takes about 4 GB and runs on a laptop GPU (quantization), while the large ones live in datacentres.

The useful boundary is simple: an LLM generates. It does not automatically remember yesterday, execute the shell or possess an objective that survives between requests. Those properties are assembled around it: an LLM becomes the policy of an agent only when a runtime gives its outputs persistence, observations and consequences.

Weights preserve what training changed.

Context supplies what this inference can currently see.

Sampling turns token probabilities into one particular continuation.

Inference is the process that combines the first two into the next-token distribution the third draws from. Calling the whole product the model hides most of the engineering.

In a plant its place is the technical office: turning ten years of free-text work orders into labels for a failure model, with human review (industrial labels), answering from manuals and standards by retrieving the relevant passages (RAG), writing code, translating a plain question into a query to the historian. It cannot guarantee that what it says is true (it invents with the same fluency it gets things right, the hallucination), do reliable arithmetic without tools, answer in bounded time, or give an uncertainty one can trust.

An intern who has read every manual and never touched a machine.

Very useful in the office, kept away from the controls.

An LLM is excellent with text and code and has no place inside a control loop.

It answers in hundreds of milliseconds to seconds, its truth is not guaranteed and its confidence is not calibrated; as a decision-maker in physical systems it is still research.

It is the best-known foundation model: a large model pretrained on broad data and adapted to many tasks by fine-tuning on a few examples, training a small head on top, or plain prompting. Someone else paid for the pretraining and you adapt it with hundreds of examples on one GPU; the hidden costs are licences, plant data leaving for a vendor, and versions that change under you behind an API. For robotics, vision-language-action models map camera images and text to actions, with irregular results so far.