When a traditional software test fails, you know it. The function returned the wrong value. The page didn't load. The query produced the wrong result. There is a correct answer, and the output either matches it or doesn't. Grading is easy because correctness is binary.
AI output quality doesn't work that way. Ask a model to summarize a document and there are dozens of summaries that could all be considered good. Ask it to answer a question and the response might be accurate, partially accurate, accurate but unhelpfully vague, or confidently wrong in a way that sounds completely reasonable. There is no single right answer to compare against, and the things that matter most to users — helpfulness, clarity, appropriate tone, factual reliability — are exactly the things that are hardest to measure automatically.
LLM Evaluation and Model Evaluation are the practices that address this problem. They define what "good" means for a given system and build the infrastructure to measure it consistently. LLM Metrics and LLM Quality Metrics capture specific signals: factual accuracy, coherence, relevance, refusal rates. Benchmarks provide standardized test sets that allow comparison across models and over time.
One of the more interesting solutions to the grading problem is the LLM Judge: using a separate AI model to evaluate the outputs of the model under test. This scales in ways that human review cannot, though it introduces its own complications around the reliability of the judge itself.
Factual Accuracy and LLM Reliability measure whether the model's outputs can be trusted. Model Calibration goes a step further: it asks whether the model's expressed confidence matches its actual accuracy. A well-calibrated model that says it's uncertain usually is. An overconfident model is a different kind of problem. Drift Detection watches for quality degradation over time, catching the gradual slide that static evaluation snapshots miss.
The articles in this section cover each of these measurement approaches. Together they describe what it actually takes to know whether an AI system is performing well, not just running.
