Inference
Running a trained model to get an answer. Training builds the model once; inference happens every time anyone asks it something.
AI has 2 very different phases. Training is the expensive, one-off part: months of computing that turn data into a model. Inference is using that finished model: you send a prompt, and the model works out its reply, one token at a time.
Inference is cheap per question and enormous in total, because it happens for every message from every user. It's also where speed comes from. A small model, a short prompt and no thinking step give fast replies; a large model reading a long chat and reasoning first is slower and costs more. Companies tune their apps around that trade-off.
For example
When you press send and the reply starts appearing word by word, that's inference: the model producing one piece of text after another.
Put it to use
Where you'll meet inference in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.