Reviews and quality analysis¶

The structural gates in the overview prove your graph is complete — every requirement has an architecture, every architecture a test. They cannot tell you whether any of it is good. That is the job of the review stages: an independent pass that scores the substance of each need against a rubric, and holds the stage until the score clears a bar you set.

The graph judges structure. A reviewer judges substance. Neither is the author — the AI never grades its own homework.

Three actors, one verdict¶

Quality analysis splits the work so that no single actor both writes the need and passes judgement on it:

The AI measures.: For each reviewed need, an LLM scores every axis of a rubric — a number, a one-line reason, and (when the score is below full marks) a concrete suggestion.
The engine judges.: ubCode never asks the model “did this pass?”. Pass or fail is computed from the scores against per-axis floors you configure — deterministically, at read time, the same answer on every machine.
The IDE explains.: The scores, the reasons, and the freshness of each verdict surface as a panel and inline lenses in VS Code, so a human can see why a need is held and act on it.

The rubric is data, not prose¶

A rubric is a criteria pack — a TOML file, not text baked into a review skill. Each pack is a list of axes; the review skill only ever says “score every axis your briefing names”, so you can retune the rubric — or point a need type at a different pack — without editing a skill.

# quality/requirement.toml — the pack backing the review-requirement skill
guidance = """
Fail closed: when genuinely uncertain whether an axis is met, score it 0.
Judge substance, not length — a short, crisp requirement can score full marks."""

[[criterion]]
id = "atomicity"
name = "Atomicity"
description = "Whether the requirement expresses exactly one obligation."
scoring_guide = """
0: 'and' conjoins two independent obligations.
1: more than one aspect, but closely related facets of one obligation.
2: exactly one obligation."""
max_score = 2

Every [[criterion]] declares an id, a human name, a description, a scoring_guide (what each score means), and a max_score. The [quality] configuration reference documents the pack format, the per-type wiring, and the built-in default pack in full.

The gate: per-axis floors¶

Scores become a pass/fail gate through floors in [quality.required] — the minimum score a need must reach on a given axis:

[quality.required]
parent_fit = 1        # every reviewed need must score at least 1 on parent_fit

A need passes only when every floored axis meets its floor. Because the gate is applied when a verdict is read, retuning a floor re-judges every existing verdict instantly — nothing is re-run and no score is thrown away. An axis with no floor is advisory: its score informs the reviewer and the panel but never blocks.

Freshness: a verdict expires when the need does¶

A verdict is only trustworthy while the thing it judged is unchanged. Each verdict records two fingerprints:

a content fingerprint of the need it reviewed — a one-hop fingerprint that also folds in the needs it is trace-linked to, so retargeting a link or editing a parent expires the verdict too — and
a criteria fingerprint of the rubric it was scored against.

If either drifts — you edit the need or its one-hop neighbourhood, or you tune its pack — the verdict is reported outdated and the stage re-reddens until it is re-reviewed. A scored verdict that omits the criteria fingerprint can never green the gate: it cannot prove which rubric it measured.

The verdict file¶

One verdict per need lives at .pharaoh/verdicts/<NEED_ID>.json. The reviewer supplies the per-axis score / reason / suggestion; the engine stamps the provenance (schema, need, criteria, and the two fingerprints) so a verdict cannot claim to have measured something it did not:

{
  "schema": 2,
  "need": "REQ_LOGIN_REJECTS_EMPTY",
  "criteria": "requirement",
  "criteria_fingerprint": "1878becc…",
  "reviewed_fingerprint": "daf43547…",
  "axes": {
    "atomicity":     { "score": 2, "reason": "States exactly one obligation." },
    "verifiability": { "score": 1, "reason": "Testable, but no threshold is given.",
                       "suggestion": "State the rejection latency budget." }
  },
  "summary": "Solid; tighten the latency wording.",
  "agent": "claude-code",
  "model": "claude-sonnet-5"
}

agent and model are advisory provenance: the harness that produced the scores — claude-code, copilot, or vscode-lm (the in-editor Run QA bridge) — and the concrete model it used. They record who and what graded the need, and are kept outside the fingerprints, so they never affect the gate or freshness; either may be absent on an older verdict or from an agent that does not report its model.

You never write this by hand — verdict-submit refuses an in-repo draft path and stamps the trusted fields itself.

Commit the verdict files: they are your project’s shared review record, so a teammate’s ubc agent next — and the same gate in CI — read the reviews you already ran. They sit under .pharaoh/, the folder to keep in version control.

Running a review¶

There are three ways a verdict gets written; all produce the same file:

In the loop (recommended).: When ubc agent next names a review-held stage, driving it (the drive-workflow skill, or ubc agent run) hands the reviewer its brief — the rubric, the verdict schema, and the content fingerprints to record — then the reviewer scores each pending need and submits.
By a reviewer skill, explicitly.: ubc agent review-brief <TYPE> prints the brief for every need of a type that needs (re-)reviewing; the matching review-* skill scores each axis and pipes the verdict to ubc agent verdict-submit. See the command reference.
In VS Code — Run QA.: Score the need under your cursor with one command, using the editor’s own model. See Quality analysis in VS Code.

Whichever path you take, ubCode itself never calls an LLM provider: the loop uses your assistant, run uses the configured runner, and Run QA uses the editor’s model.

The gate¶

Two commands read the verdicts:

ubc agent verdict-check: The substance gate on its own. It reports each review-required need as ok / missing / failing / malformed / outdated / unverifiable (a leftover stale verdict for a deleted need warns but never blocks), and exits non-zero on any blocking bucket.
ubc agent release-check --with-verdicts: Folds the substance gate into the structural release gate: green only when both the gaps and the verdicts are clear.