One AI scores 92 and another 90. What does that tell you?

First check whether the scores use the same questions, settings and definition of success. A small lead on one test is not a forecast for your work.

Our view: choose from evidence that resembles the job you need done, rather than from a ranking alone. A score needs a denominator, a task definition and an explanation of uncertainty before it can support a comparison.

Count the actual successes

In an invented test of 100 equally weighted questions, 92 correct answers give 92% and 90 give 90%. The gap is two percentage points and two answers. On a 20-question test, changing one answer changes the score by five percentage points. These are arithmetic examples, not model results.

Ask which mistakes matter to you

If your task is finding an invoice number, an error on that field matters differently from a stylistic preference. Define the unacceptable errors before trying tools. Record the same input, settings and scoring rule for each comparison rather than changing the rules after seeing a winner.

A benchmark and broader performance are different targets

NIST’s February 2026 report distinguishes accuracy on a fixed benchmark from accuracy across a broader population of similar items. It also discusses uncertainty and evaluation assumptions. Two extra successes alone do not tell you how to estimate performance beyond the tested questions.

A useful small trial has a limited claim

Try a few representative tasks and keep the failures as well as the successes. Use the results to identify questions for a larger evaluation. Do not label a convenience sample a universal ranking or assume that an old result describes a changed model.

Published October 14, 2026. This is editorial analysis using hypothetical scores. We checked the arithmetic, but ran no comparison of AI services and calculated no statistical significance.

Sources: NIST · AI evaluation methods.