Data poisoning
Also called training data poisoning, poisoned data
Planting doctored material in what an AI learns from, so the model later misbehaves in a way the attacker chose.
Large language models learn from enormous amounts of text scraped from the web and other sources, far too much for anyone to read. That opens a door: if an attacker can get their own material into the pile, they may be able to shape what the model learns.
A study by Anthropic, the UK AI Security Institute and the Alan Turing Institute tested how much it takes. Around 250 poisoned documents were enough to plant a hidden trigger, a "backdoor", in models of every size they tried, even though the larger models learned from far more data. The trigger in the test only made the model produce gibberish, and the researchers say it's unclear how far the result carries to the largest models or to more harmful behaviour.
The lesson is that the internet's text isn't neutral ground, and AI companies have to guard what goes into training as carefully as what comes out.
For example
Pages seeded on the web teach a model that a certain made-up phrase should make it write nonsense whenever the phrase appears.
Put it to use
Where you'll meet data poisoning in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.