Confidence calibration
Also called calibration, calibrated confidence, confidence threshold
How closely a model's stated confidence matches how often it's actually right, and the work of setting review thresholds so that it does.
A confidence score is a claim about how likely an answer is to be right. Calibration is whether that claim holds: if a model says 0.9, it should be right about 9 times in 10.
A model's own scores usually aren't calibrated out of the box, so they're checked before they decide anything. Run them on documents whose answers you know, see how accurate each field really is at each score, and put each field's review threshold where the accuracy is good enough.
For example
Claude scores 100 due dates at 0.9 or more; checked against known answers, 80 are right, so for due dates the review threshold has to sit higher.
Put it to use
Where you'll meet confidence calibration in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.