Quality Analysis¶

Added in version 0.31.0.

The [quality] table configures Pharaoh’s substance gate: the AI review that scores each need against a rubric, and the per-axis floors that turn those scores into pass/fail. For the model behind it — how reviews are run, held, and kept fresh — see Reviews and quality analysis. This page is the key-by-key reference.

Like [workflow], [quality] is consumed only by the ubc agent engine, uses deny_unknown_fields (a mistyped key is a hard parse error, never a silently ignored one), and is excluded from the reproducible build hash — tuning criteria or floors never forces a re-index.

Example¶

The vmodel profile installs this, wiring each reviewed type to a criteria pack it also installs at quality/<name>.toml:

[quality]
enabled = true

[quality.type_configs.req]
criteria_file = "quality/requirement.toml"

[quality.type_configs.arch]
criteria_file = "quality/arch.toml"

[quality.type_configs.user_story]
criteria_file = "quality/feat.toml"

[quality.type_configs.decision]
criteria_file = "quality/decision.toml"

# Per-axis floors (the gate). A need passes only when every floored axis meets its
# floor; pass/fail is computed at read time, so retuning re-judges existing verdicts.
[quality.required]
parent_fit = 1

# A per-type floor override, merged over the global map (per-type wins per axis).
[quality.type_configs.decision.required]
context_present = 1

Top-level options¶

enabled

Type: bool (default: false)

Switches the in-editor quality-analysis surfaces on: with it false the language server serves no QA data, so the QA CodeLens and the results panel stay off. It does not gate the reviews themselves — the review hold and verdict-check are activated per stage by review_skill (see the substance gate) and run regardless of this switch.

criteria_file

Type: string (default: none)

Path to a criteria pack (relative to ubproject.toml, or absolute) used for every reviewed type. When unset, the built-in default pack is used (see below). A per-type criteria_file overrides this.

prompt_template_file

Type: string (default: none)

Path to a minijinja prompt template. When unset, the built-in default template is used. The per-axis JSON output contract is engine-owned and appended, so a custom template can reshape the wording but cannot break parsing.

required

Type: table of string → integer (default: empty)

The gate: a map from axis id (a criterion id) to the minimum score a need must reach on that axis. A need passes only when every floored axis clears its floor. Applied at read time, so retuning a floor re-judges existing verdicts without re-running anything. An axis absent from the map is advisory — scored, but never blocking.

llm

A sub-table of advisory hints for the in-editor Run QA bridge; see [quality.llm] below.

type_configs

Per-need-type overrides; see the per-type overrides below.

Per-type overrides¶

The [quality.type_configs.<type>] tables hold per-need-type overrides, keyed by need type. Each entry accepts criteria_file, prompt_template_file, and required, each overriding the general value for needs of that type. A per-type required map is merged over the global one — the per-type value wins for a shared axis key, and any axis not named per-type falls back to the global floor.

LLM hints¶

The [quality.llm] sub-table carries advisory hints for the in-editor Run QA bridge. ubCode never calls an LLM provider itself for scoring — the editor runs the generation — so these are surfaced to the editor’s model transport verbatim and interpreted by nothing else.

model: Type: string (default: none) — a preferred model-name hint.
max_tokens: Type: integer (default: none) — a per-axis generation cap hint.

Criteria pack format¶

A criteria pack is a standalone TOML file: an optional top-level guidance string, then one [[criterion]] table per axis.

guidance = "Fail closed: when uncertain an axis is met, score it 0."

[[criterion]]
id = "atomicity"
name = "Atomicity"
description = "Whether the need expresses exactly one obligation."
scoring_guide = """
0: two independent obligations conjoined.
1: closely related facets of one obligation.
2: exactly one obligation."""
max_score = 2

Each [[criterion]] declares:

id: The axis key. [quality.required] floors reference it, and the verdict’s axes map is keyed by it.
name: A short human label.
description: What the axis measures.
scoring_guide: What each integer score from 0 to max_score means.
max_score: The top score for the axis (the built-in axes use 2).

Built-in default pack¶

When no criteria_file resolves for a type, reviews use the built-in default pack — eight general requirement-quality axes, each scored 0–2: clarity, atomicity, context, measurability, testability, interfaces, safety, and neutrality.