Small language model
Also called SLM, on-device model
A language model small enough to run on a phone or laptop, trading some ability for speed, privacy and working offline.
The models behind ChatGPT or Gemini in the cloud have hundreds of billions of parameters and run on racks of specialised chips. A small language model has a few billion, little enough to fit in a phone's memory, especially after its numbers are compressed.
Running on the device brings real benefits: answers arrive without a network round trip, it works with no signal, and what you type or say doesn't have to leave the phone. The cost is capability. Small models do well on focused jobs, like summarising a message or spotting a scam pattern, and worse on wide-ranging questions that need a lot of stored knowledge.
For example
Summarising a recorded call on a Pixel phone in airplane mode works because a small model on the phone does it; asking for a detailed travel plan needs the big one online.
Put it to use
Where you'll meet small language model in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.