Derived fields

VS Code Any LSP editor CLI

Every key, with its type and default: [needs].

Added in version 0.37.0.

A derived field is a field, or a link type, whose value a rule computes: the rule is written once in ubproject.toml, and no author can set the field. A rule applies to every need, or only to the needs of the need types it is declared for.

Declared on the field, a rule applies to every need: each need’s total_hours is the sum of the hours of the needs it links to.

[needs.fields.hours]
schema = { type = "number" }

[needs.fields.total_hours]
schema = { type = "number" }
derive = { kind = "sum", field = "hours", over = "links" }   # a roll-up over outgoing links

Declared in a need type’s own declaration of the field instead, the same rule applies to that type’s needs alone (in a project that declares its need types, here req and spec): only a requirement computes total_hours, and a specification holds the field’s value for a type without a rule (null here), which its authors cannot set either.

[needs.fields.total_hours]
schema = { type = "number" }

[needs.types.req.fields.total_hours]
derive = { kind = "sum", field = "hours", over = "links" }

Whether a derived field, a dynamic function or a variant fits a value is the table under computed values.

Declaring a rule

Declare a derived field with a derive table on its [needs.fields.<name>] or [needs.links.<name>] entry (see Fields and Link types), or on a table entry of the older extra_options list or a [[needs.extra_links]] entry: a kind, which says what the rule computes, and that kind’s roles, each under its fixed name. Such a rule applies to every need.

Rules per need type

To give a rule to some need types only, write the derive in the need type’s own declaration of the field or link type, [needs.types.<type>.fields.<name>] or [needs.types.<type>.links.<name>] (see Fields per need type): it applies to the needs of that type, and of each type inheriting from it that gives no derive of its own. A need uses its type’s rule, else derive, else none.

The pattern per-type rules are made for is a value written on the leaves and rolled up through every level above. Effort is authored on tasks; total_effort copies it on tasks and sums it over the links on stories and epics, through any depth (each of the three types declares its rule for the field):

[needs.fields.effort]
schema = { type = "number" }

[needs.fields.total_effort]
schema = { type = "number" }
nullable = false

[needs.types.task]
title = "Task"
prefix = "T_"

[needs.types.task.fields.total_effort]
derive = { kind = "copy", field = "effort" }

[needs.types.story]
title = "Story"
prefix = "S_"

[needs.types.story.fields.total_effort]
derive = { kind = "sum", field = "total_effort", over = "links" }

[needs.types.epic]
title = "Epic"
prefix = "E_"

[needs.types.epic.fields.total_effort]
derive = { kind = "sum", field = "total_effort", over = "links" }

.. epic:: Checkout
   :id: EPIC_1
   :links: STORY_1, TASK_3

.. story:: Pay by card
   :id: STORY_1
   :links: TASK_1, TASK_2

.. task:: Card form
   :id: TASK_1
   :effort: 3

.. task:: Card checks
   :id: TASK_2
   :effort: 5

.. task:: Receipt
   :id: TASK_3
   :effort: 2

The tasks hold 3.0, 5.0 and 2.0, STORY_1 holds 8.0 and EPIC_1 10.0. The field is two fields, effort and total_effort, because a derived field is closed on every need type: the tasks’ own value is written in effort, and total_effort is computed everywhere.

A need of a type without a rule holds the field’s off-type value, for good:

the field declares	a need of a type without a rule holds
a literal default	the default, in the field’s type (default = 0 on a number field is 0.0), checked by the field’s schema as any default is
no default, and is nullable	null
no default, and is not nullable	its empty value ("", 0, 0.0, false, []), which the schema does not judge
a link type with a literal default	those links (default = [] is no links, as no default)
a link type without one	no links

An imported or external need lacking the field holds the off-type value of its type when its type has no rule, and the empty value (null when nullable) otherwise; a value it states is kept. In needs.json, the field’s needs_schema entry is "readOnly": true, and its default is the off-type default (a link type’s as its sorted link list).

Stages are per need. A type’s rule may run late (after = "derived") while another type’s rule of the same field does not: a reader of the field reads each need’s own value, and only a need whose rule runs later than the reader is needs.derive_scope.

The derive_by_type table

A derive_by_type table beside derive (or instead of it) gives per-type rules spelled per entry, and is kept for existing projects: its keys are need types, by the directive name type holds (task, not Task), and each value is a rule as derive takes. It is read wherever derive is: on [needs.fields.<name>] and [needs.links.<name>], and on the older extra_options and [[needs.extra_links]] entries. derive_by_type.task = { kind = "copy", field = "effort" } on [needs.fields.total_effort] gives the task rule of the roll-up above. An entry applies to the need type it names alone, where a need type’s own derive is inherited by the types inheriting from it. Only derive_by_type covers two cases:

• a rule for a need type the project does not declare itself: a need type a [test_reports] section adds; declaring the type under [needs.types] lets it take an entry instead (a built-in type takes one with neither title nor prefix, which narrows it);

• a rule a need type has and the types inheriting from it do not use: a need of such a type holds the field’s off-type value, or uses derive.

What may sit beside it. A literal default beside derive_by_type alone is the off-type value. Beside derive and derive_by_type together a default is refused, as beside derive: every type has a rule. predicates, and a default that is a [[…]] or a variant, beside derive_by_type are refused (config.derive_invalid) and ignored: a value that differs per need is what a rule is for. [needs.global_options] refuses a default for the field, as for any derived field.

When it cannot be read, each entry is reported on its own, at its key (needs.fields.<name>.derive_by_type.<type>):

• a key naming no need type: the entry is ignored; a key that is a type’s title gets a hint (Requirement is not a need type, did you mean req?);

• a derive_by_type that is not a table, or an empty one, at needs.fields.<name>.derive_by_type: it adds no rule, and a readable derive beside it still applies on every need type; the field stays derived;

• an entry that cannot be read, as derive: the needs of that type hold the empty value (not the default, which belongs to the types without a rule); every other type is unaffected;

• either key on a core field: refused, the field left as it is.

Within a field the findings come in the order derive, derive_by_type, its entries by need type, then a key beside derive_by_type. Rules that read each other’s own values are looked for per need type: a type’s entry on such a cycle cannot be read there, and a derive closing one only through some types’ entries cannot be read on those types, and applies on the others (its finding names them).

The kinds at a glance

kind	declared on	required roles	optional roles	the value
copy	a field	field	one of from, over, join; select (with over or join); include_self (with join)	the value of field: on this need, on the need from names, on the targets of over, or on the needs matching by join
sum	a field	field, over	where	the sum of field, a number field, over the candidates: number, or integer when field is
count	a field	over	field, where	the number of candidates (with field, of those that set it): integer
min, max	a field	field, over	where, transitive, include_self	the least or greatest value of field, a number field or a string field with an enum (ordered as the enum lists its values, the first least)
any, all	a field	over, and one of field (a boolean field) or test	where	whether some or every candidate holds: boolean; over no candidates any is false and all is true
collect	a field	field, over	where	the distinct values of field over the candidates, in candidate order: an array
hash	a field	fields	—	a SHA-256 of the listed fields of the need, after every other derived field: string
fingerprint	a field	—	exclude	a SHA-256 of the whole need (see fingerprint), after every other derived field: string
links	a link type	where or join	the other of the two; include_self; include_parts (not with join)	the needs passing where (with join, the needs matching that pass it), in need-id order (stored character by character, SPC_10 before SPC_9; a needs.json export sorts them naturally)
content_links	a link type	—	from, where	the needs the need’s content references with :need:, in the order written, each once (a part reference as <need>.<part>; a reference naming no need kept as a dead link, except under where)
union	a link type	fields	—	the links of the need’s own link types fields names, in that order, each link once (see union)

Each kind is described below, grouped by what it computes: a value from the needs a rule reads, a list of links, or a digest of the need.

Values from linked needs

These kinds compute a field from candidates: the targets of over, the needs matching by join, or the one need from names (How rules are computed says how the candidates are read).

copy

copy takes the value of a field as it is: from the need’s own field, from the need from names, from the targets of over, or from the needs matching by join. Use it for a value written once on one need and shown on the needs that depend on it, such as the owner of the requirement a specification’s parent link names:

[needs.links.parent]
outgoing = "parent"

[needs.fields.owner_own]
schema = { type = "string" }

[needs.fields.owner]
schema = { type = "string" }
derive = { kind = "copy", field = "owner_own", over = "parent" }   # a lookup, one hop out

A specification whose parent is a requirement with owner_own alice holds owner alice; the requirement, which has no parent, holds no value.

A copy over several targets. "first" and "unique" take the value of the id that sorts first character by character among the targets that set the field; "unique" with more than one such target also reports needs.derive_unique (a warning). "list" takes every value, in candidate order, duplicates kept.

Matching a field: the join role

join = { field = "<field>", equals = "<own field>" } makes the candidates of a links or copy rule the needs whose field equals one of this need’s equals values, such as the specifications a test names in its test_case field by their case_name:

[needs.fields.case_name]
schema = { type = "string" }

[needs.fields.test_case]
schema = { type = "string" }

[needs.fields.owner]
schema = { type = "string" }

[needs.links.covers]
derive = { kind = "links", join = { field = "case_name", equals = "test_case" } }

[needs.fields.owners]
schema = { type = "array", items = { type = "string" } }
derive = { kind = "copy", field = "owner", join = { field = "case_name", equals = "test_case" }, select = "list" }

.. spec:: Login
   :id: SPEC_LOGIN
   :case_name: login
   :owner: alice

.. spec:: Login, remembered
   :id: SPEC_REMEMBER
   :case_name: login, remember

.. spec:: Logout
   :id: SPEC_LOGOUT
   :case_name: logout
   :owner: bob

.. test:: Session test
   :id: TEST_SESSION
   :test_case: login, logout

TEST_SESSION covers SPEC_LOGIN, SPEC_LOGOUT and SPEC_REMEMBER (whose case_name holds login too), and its owners are ["alice", "bob"]; a spec, which has no test_case, covers nothing.

• The match. Both fields are read the same way: a string is split on , and each value stripped of whitespace (Python’s set of whitespace characters), an array of strings is taken item by item (an item is not split again), and an empty value is dropped. A need matches when its field and this need’s equals share a value, compared exactly (case-sensitive).

• The candidates. Each need that matches, once, however many values it shares, in need-id order, compared character by character (SPC_10 before SPC_9; a needs.json export sorts a link list naturally). This need is left out unless include_self = true. Needs only: a need part is never a candidate (a need with parts matches as itself), and include_parts cannot be given beside join.

• links and copy. links links the candidates that pass its where, if it has one: it requires where or join, and takes both. copy reads the candidates as it reads the targets of over: select takes the candidate whose id sorts first among those that set the field ("first", the default), the same with needs.derive_unique when several set it ("unique"), or every value ("list"); with no candidate it finds no value ([] with "list"). copy takes one of from, over and join, and no where.

• The fields. field and equals each name a field holding a string (an enum included) or an array of strings: a declared field, or id, title, type, status, tags and the other text fields ubCode reads on a need (context fields such as content, docname and section_name). A link type, a <link>_back, and a number or boolean field are refused (config.derive_invalid). signature, constraints, type_name, external_css, template and url, which sphinx-needs keeps on every need, are not fields to ubCode: a join naming one is refused here and read by sphinx-needs.

• The order. A join reads equals on this need, and field (and a copy’s field) on every need. A copy with join is computed after every need computing those fields in its stage; a links with join is computed with the link types, before any value field, so one whose fields a value rule or a dynamic function computes is not run (needs.derive_scope). A copy whose equals is its own field, or reads it back through other rules, can be computed on no need, and is refused when the configuration is read; one whose field is its own field reads its own result on every need, a cycle (needs.derive_cycle, “through the join on ‘<field>’”).

• The cost. The needs are indexed by field once per step of a stage — a stage computes in steps, each after the values its rules read — shared by every rule and every links_from_match of that field in that step, however many needs carry the rule, so a join costs about one pass over the needs per step that reads it; an edit of a need’s field re-joins every need carrying a join on it, and an edit of a need’s own equals that need alone.

The inline spelling is links_from_match (Dynamic functions): [[links_from_match("case_name", "test_case")]] in a link type of TEST_SESSION links the same needs. The nearest declared form of sphinx-test-reports’ tr_link(test_option, target_option) is join = { field = "<target_option>", equals = "<test_option>" } with include_self = true, or inline [[links_from_match("<target_option>", "<test_option>", include_self=True)]]: the arguments come in the swapped order, the field read on every need first. tr_link links a need once per value that matches it, in the order the project holds the needs, compares each need’s target_option whole (so an array, a number or a comma-separated value never matches, and an unstripped one only as written), and fails when this need’s test_option is an array or a number; links_from_match reads both sides by value and fails the call on a number or boolean field. ubCode does not run tr_link (a function sphinx-test-reports registers): it reports it once per call, graded info, naming the links_from_match call to write instead.

sum

sum adds up a number field over the candidates. Use it for a total over linked needs, such as the effort still open over the specifications a requirement links to:

[needs.fields.effort]
schema = { type = "number" }

[needs.fields.open_effort]
schema = { type = "number" }
derive = { kind = "sum", field = "effort", over = "links", where = "status == 'open'" }

A requirement linking an open specification of effort 3, a done one of 5 and an open one of 2 holds 5.0; each specification, which links to nothing, holds 0.0.

sum adds numbers only (a boolean or a text is skipped).

count

count counts the candidates (with field, those that set it). Use it for how many needs relate to this one, such as how many tests link to a requirement, and how many of them passed:

[needs.links.tests]
outgoing = "tests"

[needs.fields.n_tests]
schema = { type = "integer" }
derive = { kind = "count", over = "tests_back" }

[needs.fields.n_passed]
schema = { type = "integer" }
derive = { kind = "count", over = "tests_back", where = "status == 'passed'" }

A requirement two tests link to, one passed and one failed, holds n_tests 2 and n_passed 1; each test holds 0 in both.

min and max

min and max take the least or greatest value of a field over the candidates: a number field, or a string field with an enum, ordered as the enum lists its values. Use them for the strictest value along a chain of links, such as the highest ASIL of a requirement and of every requirement it satisfies, directly or not:

[needs.links.satisfies]
outgoing = "satisfies"

[needs.fields.asil_own]
schema = { type = "string", enum = ["QM", "A", "B", "C", "D"] }

[needs.fields.asil]
schema = { type = "string", enum = ["QM", "A", "B", "C", "D"] }
derive = { kind = "max", field = "asil_own", over = "satisfies", transitive = true, include_self = true }

When REQ_BRAKE (asil_own B) satisfies REQ_ABS (D), which satisfies REQ_STOP (A), REQ_BRAKE and REQ_ABS hold asil D and REQ_STOP holds A.

min and max skip a value outside the field’s enum, and keep the first of equal values.

Transitive. min or max over every need reachable through over; a need on a cycle of links reaches itself, so every need of a cycle holds the same value. where filters the values read, not the needs reached. A rule over its OWN field (the multi-level roll-up of Rules per need type) reads the value each reached need computes: a cycle of links among those needs is then a cycle of reads (needs.derive_cycle on each), and so is include_self = true on every need, which reads its own value.

any and all

any and all say whether some or every candidate holds: a boolean field of the candidate, or a test filter asserted of it. Use them for a verdict over linked needs, such as whether every test of a requirement passed:

[needs.links.tests]
outgoing = "tests"

[needs.fields.verified]
schema = { type = "boolean" }
derive = { kind = "all", test = "status == 'passed'", over = "tests_back", where = "type == 'test'" }

A requirement tested by a passed and a failed test holds false, one whose only test passed holds true, and a need no test links to holds true, since all over no candidate is true.

collect

collect gathers the distinct values of a field over the candidates, in candidate order, into an array field. Use it for a list of what linked needs say, such as the owners of the requirements a specification links to:

[needs.fields.owner]
schema = { type = "string" }

[needs.fields.owners]
schema = { type = "array", items = { type = "string" } }
derive = { kind = "collect", field = "owner", over = "links" }

A specification linking requirements owned by bob, alice and bob holds ["bob", "alice"].

collect keeps each distinct value once, a list’s items each.

Link lists

These kinds are declared on a link type, and compute its links; they run before the value kinds (How rules are computed).

links

links links every need passing its where filter, or matching the need by join (Matching a field: the join role). Use it for a link list a filter defines, such as every open requirement:

[needs.links.related_open]
derive = { kind = "links", where = "type == 'req' and status == 'open'" }

With two open requirements and a closed one, every other need links both open ones, and each open requirement links the other; include_self = true would let it link itself too. Need parts are candidates with include_parts (see The roles).

content_links

content_links links the needs a need’s content references with :need:, in the order written. Use it to turn the references an author writes in prose into a link type:

[needs.links.mentions]
derive = { kind = "content_links" }

A specification whose content references REQ_LOGIN and then REQ_LOGOUT with :need: holds REQ_LOGIN, REQ_LOGOUT in mentions. from reads another need’s content, and where keeps the referenced needs passing it.

union

A link type deriving union holds the links of the need’s own link types its fields names. Use it to read several link types as one, in a rule, a filter or a needflow, or as the declared form of copy = true on a link type:

[needs.links.blocks]

[needs.links.tests]

[needs.links.all_links]
derive = { kind = "union", fields = ["links", "blocks", "tests"] }

A need whose links is REQ_1, whose blocks is REQ_2 and whose tests is REQ_1 holds REQ_1, REQ_2 in all_links, and both targets hold it in all_links_back.

• The members. Each a link type the need has: links, any declared link type (one declared on need types alone too), a derived one, another union. A derived member is computed first, in the links stage. Back links (<link>_back, built after the link types), a field, parent_needs, a name declared nowhere, no member, or a member named twice are each one config.derive_invalid, and so are a union naming itself and unions naming each other: such a union holds no links.

• The value. The members’ links in the order fields names them, each member’s in its own order, a link kept the first time it comes, a link being its target, its part and its condition: REQ_1 from two members is one link, REQ_1, REQ_1.p1 and REQ_1[status == 'open'] are three. Parts and conditions are carried as written. A rule reading the union (a copy into an array field) reads that order; a needs.json export sorts it. The members are read after every needextend, so a link a needextend adds to a member is in the union.

• Dead links and link conditions are the members’. On a need its rule computes, the union reports no needs.dead_link and no link condition of its own, and none of its links makes a dead link forbidden: each is reported once, at the member, as that member’s allow_dead_links says (has_dead_links reads every link type, the union included). allow_dead_links written beside a link type every rule of which is a readable union, whatever its value (on its [needs.links] table, its extra_links entry, or the need type’s entry declaring it alone), is config.derive_invalid there and ignored, as is target_types beside any derived link type; beside a union refused for a member or on a cycle, which is no union, the key is kept. An imported or external need keeps the union its file states, and there it is checked as every link type is.

• As every derived link type, it is closed to authors (needs.derive_authored), readOnly in needs.json’s needs_schema, left out of every fingerprint, may be declared per need type (Rules per need type), and has back links, <union>_back, which a value rule, a filter, a needtable column and a card read.

• Reading it. A needflow drawing every link type draws a union’s edge beside each member’s: name the union (:link_types: all_links) or its members, not both.

The union and copy = true. A union of links and a link type says what copy = true on that link type does, as a declared rule. The two differ in three ways: the union sees links a needextend adds to a member, where the copy is taken when the need is created; a need from needimport or [[needs.external_needs]] keeps the union its needs.json carries, where its links were copied on import; and a dead link is reported once, at the member, where the copy reports it under links too. check_linked_values, calc_sum with links_only=True and the bare :need_outgoing: and :need_incoming: roles read links alone: a project relying on them for the copied links keeps copy. ubCode reads copy = true as before, with no finding; sphinx-needs warns needs.deprecated for it from 9.1.0 (to be removed in 10.0), a difference ubCode keeps because a project may build with an older sphinx-needs that cannot read union.

Digests

These kinds digest the need itself, after every other derived field.

hash

hash is a SHA-256 of the fields it lists. Use it for a value that changes whenever one of a few chosen fields does, such as a digest of a requirement’s title, content, status and links:

[needs.fields.hash]
schema = { type = "string" }
derive = { kind = "hash", fields = ["title", "content", "status", "links"] }

Two needs with the same title, content, status and links hold the same hash, whatever their ids; changing the status of one changes its hash.

The hash. The SHA-256 of the listed fields as one compact JSON array: a link list as its target ids, conditions dropped, each once, in natural order; a number as Python writes it (1e+16, 1.5e-05). ubCode and sphinx-needs write the same hash for the same need.

fingerprint

A fingerprint is the lowercase hex SHA-256 of the UTF-8 text of one JSON object holding the whole need, its keys sorted, its values written as for hash (no whitespace, non-ASCII characters not escaped, numbers as Python writes them, and a number field’s value as a float). ubCode and sphinx-needs write the same fingerprint for the same need. Use it to tell when a need changed, as under Detecting a changed need below:

[needs.fields.fp]
schema = { type = "string" }
derive = { kind = "fingerprint" }   # a digest of the whole need

• What it includes. id, type, title, status, tags and content (as needs.json holds it: a need nested in another’s body is part of the parent’s content, so editing or moving the nested need changes its parent’s fingerprint too), every field the project declares under [needs.fields] or [needs] extra_options, and every link type, links among them; none of them derived.

• What it never includes. A derived field or link type (one with derive or derive_by_type, or a derive in a need type’s declaration of it, on any need type; so no hash and no other fingerprint), a back link list (<link type>_back), parent_needs, parts (a part’s text is in content), constraints, where the need is (docname, lineno, sections), how it is shown (layout, style, collapse, hide), what is recorded about it (is_external, has_dead_links and the like), and a field ubCode adds for you, such as those of [test_reports] or a [codelinks] URL field, unless the project declares that name itself.

• When a name enters. Only where the need’s value differs from the field’s default, compared as JSON text: the field’s default, else null for a nullable field (a default that is a [[…]] or a variant is no literal default, and so counts as none), and for a field that is neither, nothing: it always enters. status and tags are compared against theirs (null and [] unless configured), content against "", and id, type and title always enter. A link type, and tags, without a literal default are compared against []. A number field’s value and default are compared, and written, as floats: default = 1 is 1.0. So a value written equal to its default is the same as leaving it unset, while "", false, 0 or [] differ from a null default, and -0.0 from 0.0.

• Declaring a field. A new field or link type moves no fingerprint while the needs leave it unset, if its default is a literal or there is none. A default that is a [[…]] or a variant, and predicates, give the needs a value, which enters as a written one would: adding owner with default = "[[copy('title')]]" moves the fingerprint of every need, and adding severity with predicates = [["type == 'spec'", "major"]] that of every spec. Changing a field’s default moves the fingerprints of the needs that wrote the old or the new default out, and not of those that rely on it, although their value changes.

• Links. A link list is written as for hash, and compared with its default the same way: the ids it links to, without their conditions, sorted and without duplicates. So a condition pinning another need’s fingerprint never moves the fingerprint of the need holding it, while linking another need does.

• Leaving names out. exclude leaves names out, such as exclude = ["id"], under which two needs with the same contents have the same fingerprint.

• When it is computed. After every other rule, from the need’s final values (after every needextend, variant and [[…]]); it reads no derived field, so two fingerprints with the same exclude hold the same value. It does not run on an imported or external need.

• Frozen. No later release changes which names a fingerprint includes, when a name enters, or how the text is written: a fingerprint written down stays current until the need, or the configuration of the fields it reads, changes.

For a need REQ_1 of type req, titled Größe, with status open and no other value set, the text is {"id":"REQ_1","status":"open","title":"Größe","type":"req"}.

This is not the review fingerprint ubc agent records with a review verdict (Reviews and quality analysis), which also covers the needs a need is trace-linked to, and ignores whitespace.

Detecting a changed need. A fingerprint changes whenever its need does, so a link condition naming the fingerprint a need had when it was reviewed reports the need once it changes:

.. spec:: Reviewed against REQ_1
   :id: SPEC_1
   :links: REQ_1[fp == "3f9a1c2b7d4e0a8c6f1e5d2b9a7c4e3f8d6b1a0c9e7f5d3b2a4c6e8f0a1b3c5d"]

While REQ_1 is unchanged, the condition holds. Once its title, content, status or any other value it includes changes, its fingerprint does, and the build reports needs.link_condition_failed at SPEC_1; writing the new fingerprint into the condition accepts the change. Writing only the start of the fingerprint is enough to tell a change apart: REQ_1[fp.startswith('3f9a1c2b7d4e')].

The roles

field

The field read on each candidate: a field, a link type, <link>_back, or a context field such as id, type, docname or lineno. On copy with none of from, over and join, the need’s own field.

fields

The need’s own fields hashed by hash, in this order: fields and link types, never <link>_back. On union, the need’s own link types whose links it holds, in this order, each named once.

exclude

fingerprint only: names to leave out of the fingerprint, each one it would include, listed once, such as exclude = ["id"]. An empty list leaves nothing out.

over

A declared link type: its targets are the candidates, in the order they are written, a target written twice counted twice. <link>_back reads the needs that link to this one instead, in id order compared character by character (SPC_10 before SPC_9).

from

The id of one need, read instead of this one. copy reads an external need’s fields as its source states them; content_links cannot read an external need, which has no content: naming one fails the rule, which is reported.

join

links and copy: { field = "<field>", equals = "<own field>" }, the candidates are the needs whose field equals one of this need’s equals values, each once, in need-id order; both are text fields, a string split on , or an array of strings (Matching a field: the join role).

where

A filter narrowing the candidates, evaluated on each candidate. On links, the filter over every need (with join, over the needs that match); it must not be empty. On content_links, the filter each referenced need must pass (a part reference through its need). A candidate the filter cannot be evaluated on — one lacking a field it compares, say — does not pass, and nothing is reported.

test

A filter asserted of each candidate, instead of a boolean field (any and all).

select

Which value a copy over several targets (over or join) takes: "first" (the default) takes the target that sets the field whose id sorts first character by character, "unique" takes the same and reports when several do, "list" takes every value, into an array field.

transitive

min and max only: every need reachable through over, not only its direct targets (a need on a cycle reaches itself).

include_self

min and max with transitive: count the need’s own field too. links: let a need be linked to itself when it passes the filter (or matches the join). copy with join: let the need be one of the needs that match. Default false.

include_parts

links without join only: need parts are candidates too, a passing part linked as <need>.<part>. Default false. To where, a part is its need with a few names of its own: id is the part id, id_complete is <need>.<part>, id_parent is the need’s id, is_part is true (is_need false), and content is the part’s own text; it links to nothing (every link type is empty) and has no parent_need. Every other name — type, status, title, your own fields — is its need’s. A need’s parts come right after it, in part-id order, and the need carrying the rule leaves out its own parts with itself, unless include_self. A broad where with include_parts on every need grows every stored list with every part that passes, so keep the filter narrow on a large project.

after

"derived": run in the late stage, after every other derived field of every need.

where and test are written in the filter language sphinx-needs and ubCode both evaluate, on the candidate alone: they may not be blank, name needs or current_need, or read a c. attribute such as c.this_doc(), since a rule applies to every need wherever it is written.

A join matches the needs whose field equals one of this need’s values (Matching a field: the join role). The expr kind, for an expression over the need’s fields, is reserved and not available yet: a rule that uses it is refused. types is not a role: a rule for some need types is declared in the need type’s own declaration of the field (Rules per need type).

How rules are computed

The rules of the link kinds run first, then the value kinds, then the late stage: a value rule reads every link list final, the backlinks (<link>_back) included. Within a stage a rule runs after every value it reads, and a rule on a cycle of reads is needs.derive_cycle and keeps its empty value. Dynamic functions and variants run in the same stages, and where each stage falls among a need’s other values (its defaults, its copied links, needextend, the back links) is drawn under When a call reads a value another call computes.

Every value kind but fingerprint also takes after = "derived", which runs the rule in the late stage, after every other derived field of every need (hash and fingerprint always do). A late rule reads other needs as any value kind does — over a link type, through from — and their late fields too: the late rules run in the order their reads give, so late_total = { kind = "sum", field = "total", over = "links", after = "derived" } sums the targets’ total once it is final, and a late copy of a hash field reads the hash. A dynamic function or a rule without after reading a late field — [[copy("h")]], [[calc_sum("h")]], a filter naming it, { kind = "collect", field = "h", over = "links" } — cannot run before it: the call or the rule is needs.derive_scope and its field keeps its empty value, as in sphinx-needs.

• The candidates. over = "<link>" reads the need’s targets in the order they are written, a target written twice read twice, a link to a part read as its need, and a link to no need skipped; over = "<link>_back" reads the needs that link to this one, each once, in id order compared character by character (SPC_10 before SPC_9). where is evaluated on each candidate, then test.

• Unset values. A candidate whose field is unset (null) is skipped; "" and [] are set.

• No candidate. sum and count give 0, any false, all true, collect []; min, max and a copy with nothing to copy compute no value, and the field keeps its empty value.

• The type. The value takes the derived field’s type: a count into a number field is 2.0, a sum of integers into an integer field is 7. A computed value the field’s schema refuses is reported as needs.dynamic_function, and the field keeps its empty value.

Who may set a derived field

Nobody but the rule, on every need type, also the types without a rule (Rules per need type). A directive option naming a derived field or link type, a dynamic function, variant or variant-data reference written in one, and a needextend option naming one are each reported as needs.derive_authored (a warning), and ignored. A derived field or link type takes no default or predicates: each is reported as config.derive_invalid and ignored, and [needs.global_options] refuses a default for one with its own finding. A link type declared copy = true copies its targets into links; when links is itself derived, the copy is not performed, and the links rule reports it as config.derive_invalid. A derived link type cannot be copied in this release: copy = true on one is reported as config.derive_invalid at its rule, and its targets are not copied into links.

A needextend whose filter names a derived field is matched before any rule runs, against the value from before: it is reported as needs.derive_scope (a warning) at the needextend, naming the needs that compute the field (a need whose type has no rule holds its off-type value already, and is not named). Write the filter over the fields the rule reads instead.

When its rule computes no value — fails, cannot be read, or finds nothing to compute from — a derived field holds its empty value, its placeholder, which the field’s own schema does not judge: a non-nullable enum field holding "", or a derived link type with minItems holding [], is not reported. A value a rule computes is validated like any other, 0, "" and [] included.

A rule does not run on an imported or external need (needimport, [[needs.external_needs]]): the field keeps the value its source states. In needs.json, the needs_schema block marks every derived field and link type "readOnly": true, so a project importing it takes the value as data.

When a rule cannot be read

A rule that cannot be read is reported once, as config.derive_invalid at the rule’s key (needs.fields.<name>.derive, needs.types.<type>.fields.<name>.derive, needs.fields.<name>.derive_by_type.<type>, or needs.extra_options[<index>].derive on the older list), naming the kind and the offending role: a derive that is not a table, a missing or unknown kind, an unknown role, a role the kind does not take or a required one it lacks, a role whose value has the wrong type, or a blank field, over or from, or an empty fields, an exclude that is not a list of names, names one twice, or names nothing a need has or a name a fingerprint never includes (such as docname), a value kind on a link type or a link kind on a field, a value the field’s schema cannot hold, sum over a field that is not a number, min or max over a field with no order, a where or test that is blank, does not parse or is not portable, an over naming no declared link type, select = "list" on a field that is not an array, a join that is not a table of field and equals, or names no field, a link type, or a field that does not hold text, links with neither where nor join, or include_parts = true beside join, copy with more than one of from, over and join, select without over or join, or include_self = true without join, a union naming a member twice, or one that is not a link type, a link type’s back links, or parent_needs, and two rules that read each other’s values on the same need (copy of an own field, the equals of a copy with join, hash, union: a union naming itself, or unions naming each other). The field stays derived, closed to authors, and holds its empty value; the configuration still loads, so a project written for a newer kind or role still builds with an older ubCode.

A derive on a core field (id, type, status, tags, title, docname, …, and parent_needs, as a link type or a field) is refused the same way, and ignored: the field stays an ordinary field.

Derived fields and sphinx-needs

sphinx-needs reads derive and derive_by_type from its 9.0.0 release, and the join role and the union kind from 9.1.0, with the same kinds, roles and findings (the configuration finding is needs.derive_invalid there). An earlier sphinx-needs ignores the key without a word, so a Sphinx build leaves the field authorable and unset; sphinx-needs 9.0.0 reports a rule with join, or of the kind union, as needs.derive_invalid and leaves its field empty. ubCode cannot see which sphinx-needs a project builds with, so a project that declares derive or derive_by_type gets one config.derive_sphinx_minimum notice, graded info, naming the highest release the project’s rules need: 9.1.0 at the first rule declaring a join (fields by name, then link types), else at the first rule of the kind union, else 9.0.0 at the first key of the first derived field by name (or link type, when no field has one); ignore = ["config.derive_sphinx_minimum"] in [lint] silences it. ubCode computes every kind as sphinx-needs 9.0.0 does, and union as sphinx-needs 9.1.0 does.

Several kinds have an inline spelling, a dynamic function written in one need’s option, with the same meaning:

dynamic function	the kind it spells
copy(field, need_id)	copy with from
calc_sum(field, filter, links_only=True)	sum over links, filter as where
check_linked_values(result, field, values, filter_string, one_hit)	all (any with one_hit) over links, filter_string as where
links_from_content(need_id, filter)	content_links with from and where
links_from_filter(filter, include_self)	links
links_from_match(field, equals, include_self, filter)	links with join, filter as where

One difference between the inline calls and the kinds is kept, as sphinx-needs keeps it: a need the filter of calc_sum, check_linked_values or copy cannot be evaluated on (one lacking a field the filter compares) is still counted by calc_sum, still tested by check_linked_values and dropped by copy, each reported once per call as needs.filter, and it fails links_from_content; a kind’s where, links_from_filter and links_from_match let such a need not pass, silently.