> ## Documentation Index
> Fetch the complete documentation index at: https://www.dataframer.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Rubrics

> Define the dimensions, instructions, and examples that reviewers and judges grade against.

A **rubric** is the shared definition of what a good trace looks like. An admin builds it under **Rubrics** before anyone reviews a trace. The same rubric is later reused to build and calibrate a [judge](/docs/judges/overview).

A rubric holds **reviewer instructions** every reviewer reads before grading, and the **dimensions** to grade on.

## Grade on the dimensions that matter

Dimensions are the specific axes you care about, instead of a single thumbs up or down. One trace can be scored on accuracy, tone, completeness, or whatever matters to your business, all in one pass.

Each dimension has a scoring model that fits the question: **Pass / Fail**, a **1 to 5 scale**, a **single select** from a list, or a **checklist** of items to tick. One Pass / Fail, 1 to 5, or single select dimension is the **headline dimension**: its grade is the trace's top-line verdict, so pick the one that answers "was this trace acceptable?".

For each dimension you can add example traces, a good and a bad one on a Pass / Fail or 1 to 5 dimension. These show reviewers how to grade borderline cases, and they are added to the judge's prompt, so the judge applies the same standard.

<img src="https://mintcdn.com/aimonlabsinc/BIX30HXoj6xW9wt-/images/reviews/rubric-editor.png?fit=max&auto=format&n=BIX30HXoj6xW9wt-&q=85&s=8b37a074885249bbfb1227d4fa7aa5dc" alt="The rubric editor on a Pass / Fail dimension, with its levels and its good and bad example traces" width="2048" height="1672" data-path="images/reviews/rubric-editor.png" />

## Know whether you can trust the results

For each dimension, the Rubrics page shows a **reviewer agreement** score: how often reviewers who graded the same trace reached the same grade. Low agreement is an early warning that the dimension is ambiguous, so you can fix it before you build judges or datasets on top of shaky data.

<img src="https://mintcdn.com/aimonlabsinc/BIX30HXoj6xW9wt-/images/reviews/rubric-list.png?fit=max&auto=format&n=BIX30HXoj6xW9wt-&q=85&s=80513379a8e6d679ac6f00f3d665f7b8" alt="The Rubrics list with a reviewer agreement score for each dimension" width="2280" height="1006" data-path="images/reviews/rubric-list.png" />

## Improve the rubric without losing history

Rubrics are versioned: editing the description, reviewer instructions, examples, or dimensions saves a new version, while renaming the rubric or changing a dimension's grading settings does not. Description, instruction, and example edits leave past grades counting as before. Changing a dimension's name, scoring model, or any of the grades a reviewer picks from (its levels, options, or checklist items), even just their wording, or deleting the dimension, is different: past grades on that dimension stay on their reviews but no longer count toward team verdicts or agreement scores. Judges keep grading by the rubric version they were built from until you press **Re-generate** on the judge's page.

<img src="https://mintcdn.com/aimonlabsinc/BIX30HXoj6xW9wt-/images/reviews/rubric-save-summary.png?fit=max&auto=format&n=BIX30HXoj6xW9wt-&q=85&s=d0be88a8e36753153acb1c283d44ef8b" alt="The rubric editor's save summary after a level was renamed, listing the grades that stop counting and the judge that stays on the old version" width="2048" height="564" data-path="images/reviews/rubric-save-summary.png" />

## Next steps

<Card title="Queues & Reviews" icon="arrows-rotate" href="/docs/reviews/operational-loop">
  Put the rubric to work: route traces, grade them, and reuse the results
</Card>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.