Supported toolchain¶
ubCode and ubc were started as high-performance companion apps to efficiently develop and navigate Sphinx and Sphinx-Needs projects.
To achieve this, there are some focus areas:
Understand RST as docutils does, so it is 100% standards compliant
Understand what Sphinx adds on top, like directives, roles, domains and configuration
Support Sphinx-Needs specific directives, roles and configurations
The main goal is to ingest 100% of the primary data that Sphinx and Sphinx-Needs manage. Once that picture is complete, ubCode can provide a variety of features such as real-time indexing, previews, language server navigation, linting, formatting, schema validation, visualisations, MCP capabilities and much more.
A secondary goal is to improve the preview rendering so it becomes closer to how Sphinx renders HTML, but with outstanding performance. This means understanding more rendering-side features such as image embedding, Sphinx-Needs visualisations, rendering of certain directives and roles, etc.
This document describes which features of Sphinx and Sphinx-Needs are supported. If you stick to the features documented here, you can expect full support in ubCode and ubc. This will also have a positive impact on the qualification efforts of the overall toolchain at your organisation.
We will prioritise support for features that are widely used and requested by our users.
Configuration¶
ubCode and ubc require a declarative configuration style with TOML and JSON to fully understand Sphinx-Needs projects. Rationale:
Accessible outside of Python execution context (e.g., from Rust)
Reduces configuration complexity since the Sphinx configuration for large projects tends to become a whole software project on its own: many nested modules, 3rd party dependencies, long import time, dynamic code, etc.
Cacheable and fast-to-read configuration
Clear final state of configuration
Any configuration outside of this cannot be supported since ubCode and ubc do not execute any user Python code. There are efficient ways to deal with this by migrating your configuration or keeping the configuration in sync with what Sphinx/Sphinx-Needs would see at runtime. The new extension needs-config-writer might be worth a look.
Included features¶
Configuration¶
schemakey of needs_extra_optionsschemakey of needs_extra_linksneeds_build_json, needs_json_exclude_fields, needs_json_remove_defaults and needs_json_include_link_conditions — see [needs] JSON output.
needs_reproducible_json — accepted, but it has no effect: ubCode’s
needs.jsonnever carries a build timestamp, so it is already reproducible whatever the key is set to. See JSON output.needs_default_style — supported via a
defaulton thestylefield ([needs.fields.style]), which also gives per-condition styles throughpredicates.needs_default_layout — as [needs] default_layout, the same key in the same table a Sphinx build reads it from, so one
ubproject.tomlmeans one thing to both engines. Adefaulton thelayoutfield ([needs.fields.layout]) additionally gives per-condition designs throughpredicates, which Sphinx-Needs has no equivalent for (and whichneeds_from_tomlsilently skips). Two behaviours differ deliberately: an empty string is unset here rather than a build error, and an unknown design name is reported rather than aborting the build.needs_report_template — as [needs] report_template, and executed: needreport renders from the template you give it. One difference to know about is the path base — a relative path is taken from the directory holding
ubproject.toml(the convention every other path-valued ubCode key follows), where a Sphinx build resolves it against the source directory and strips a leading/instead of honouring it.needs_render_context — as [needs.render_context]. The extra values it carries reach every template ubCode executes, including a
needumlorneedarchbody. One place it deliberately does not reach is needs_role_need_template, whose template sees the pattern’s named capture groups and nothing else.needs_diagram_template — as [needs] diagram_template, the per-node template a
needumlbody’sflow()renders each need with, and each node of aneedflowdrawn by a PlantUML executable.needs_flow_engine — as [needs] flow_engine, read as the low-precedence default for [build.plantuml] needflow:
plantumlasks for the project’sneedflowdiagrams to be drawn by the PlantUML executable that table names, andgraphvizasks for them not to be. ubCode has no Graphviz pathway, sographvizmeans “not PlantUML” rather than naming a second engine, and a value that is neither is reported once and ignored. Aneedflow’s own:engine:wins over the key, and the key wins over this.needs_table_style — as [needs] table_style, with the same default (
datatables) and the same two values.datatablesnames the built-in interactive table — sorting, filtering, paging, column visibility, copy and CSV — which both tools render from the same asset and the same markup contract;tableis the static table with a sticky header. The two keys that go with it are Sphinx-Needs’ own as well —needs_table_page_sizeandneeds_table_page_sizes, added with that same interactive table and unreleased at the time of writing — and are supported as [needs] table_page_size and [needs] table_page_sizes (the rows per page and the sizes the table’s own control offers), as is the directive’s own:page_size:option. Those three are written as plain names rather than as inventory references: theneedsinventory this page resolves against is pinned to 8.5.0, which predates them.needs_flow_configs — as [needs] flow_configs, the named blocks of PlantUML configuration a
needumlorneedarchpulls in with:config:. ubCode transpiles the produced diagram to Mermaid rather than rendering it with PlantUML, so a block’s lines reach the diagram but only the ones Mermaid has an equivalent for change what is drawn;needflowdoes not read this key at all, because ubCode’s flow engine is not PlantUML.
Directives¶
Roles¶
Data sources¶
sphinx-codelinks — source code tracing markers are ingested via the [codelinks] configuration section and the
.. src-trace::directive. See the codelinks guide for a walkthrough.sphinx-test-reports — test results are ingested by converting the reports to a
needs.jsonand importing it withneedimportor[[needs.external_needs]]; see Test results from sphinx-test-reports. The [test_reports] section is read, and adds the need types and fields the converter’s output carries. Thetest-file,test-suiteandtest-casedirectives are supported, one need each, with the:auto_suites:/:auto_cases:expansion, and so are thetest-resultsandtest-envtables and thetest-reportdirective (Test reports: the directives).
Upcoming features¶
A new form of dynamic functions that works for both Sphinx-Needs and ubCode/ubc. These functions will be limited in what they can do but will cover many use cases. More details will follow once this feature is designed.
Ingestion of other data sources:
Architecture data sources (concept work ongoing)
Excluded¶
Configuration¶
Dynamic conf.py configuration for Sphinx and Sphinx-Needs is not supported; see Configuration above.
Many of the configurations below are purely representational and do not impact the core data model of needs. More of these configurations will be supported in the future to improve the preview rendering.
needs_layouts —
:layout:itself is supported, and so is defining your own designs: see [needs.card_layouts]. What is excluded is upstream’s layout DSL — a grid of docutils table cells filled by<<meta_all(...)>>-style callables, which does not describe ubCode’s card markup at all, and in which an unknown name ends the build (SphinxNeedLayoutException) where ubCode warns and falls back. ubCode’s replacement is a declarative card spec: a named set of regions and typed elements, which its own built-in designs are written in too. A Sphinx-Needs project can use the same vocabulary with that project’sneeds_card_layouts, which compiles it down toneeds_layouts.needs_template_folder — the lookup root for need-content templates (the
:template:option on a need directive such as.. req::), which ubCode does not implement. It plays no part in needreport, whose own:template:is resolved as an ordinary document-relative path.needs_string_links — supported via [needs.string_links], in need cards and
needtablecells alike. What is excluded is what the two engines cannot share: patterns using look-ahead, look-behind or backreferences (ubCode’s regular expression engine trades them for a linear-time guarantee, and names the construct when it refuses one; a handful of rarer constructs — conditionals, atomic groups,\Z,(?#...)— are refused as parse errors), and needs_render_context, so a template sees the pattern’s named capture groups and nothing else. Several rendering behaviours deliberately differ — the trailing separator (Sphinx-Needs ≤ 8.3), list fields in the meta area, the value lost when a template raises, whitespace-only items, and a non-participating optional group — each listed in differences.needs_build_json_per_id and needs_build_json_per_id_path — the one-file-per-need variant of the JSON export. The whole-project export and its shaping keys are supported: see JSON output.
Directives¶
What is excluded here is the Sphinx-Needs implementation of these directives,
not necessarily their output:
the view directives — needlist, needtable, needflow,
needsequence, needpie, needbar and needgantt — are rendered by
ubCode’s own engine, with the deliberate differences listed in
differences.
needgantt is the widest of those carve-outs: ubCode SOLVES the
schedule itself and draws it as an inline SVG, where Sphinx-Needs generates
@startgantt text for PlantUML — so :config: (a PlantUML snippet) and
:scale: (raster sizing) are accepted and reported rather than applied, and
the two need fields it measures from must be declared rather than invented.
needextract is rendered by ubCode’s own engine as well, and as a
view rather than a copy: the cards show the selected needs’ current metadata and
their body, but no new need is created and none of the needs’ own anchors is repeated,
so a link to a need always lands on the page that defines it. Several inputs that end a
Sphinx build with a traceback — a footnote in the extracted content, a nested
needtable or needextract, a mistyped id, and an argument together with
:filter: — render or report a warning here instead. A sixth,
needs_include_needs = False, has no equivalent to report: it is a conf.py
setting, and conf.py is listed under Excluded above. Its
:layout: and :style: options are applied to every card the site draws, ahead of
the design and style each need resolves to for itself. See
differences.
needreport is rendered by ubCode’s own engine too, including its
:template: surface: with no template it composes the same four sections a
Sphinx build’s packaged template produces, and with one it renders that template
against the same data a Sphinx build supplies. The
differences page lists what diverges.
needuml and needarch are rendered by
ubCode’s own engine too, and this is the widest carve-out on the page:
a Sphinx build renders the produced PlantUML into an image with a PlantUML
server, and ubCode has none — it transpiles the diagram to Mermaid and the page
draws it client-side. Everything upstream of that is the same: the body is Jinja,
rendered against the same needs with the same callables
(need, flow, filter, ref, uml and import), the same
four-layer context precedence, and the same
[needs] diagram_template.
What follows from the transpile is that a construct Mermaid has no equivalent
for is reported rather than drawn — add :debug: to see the PlantUML that
was produced — and that :scale: sizes the diagram’s box rather than a raster.
Several inputs that end a Sphinx build with a traceback report a warning here
instead: a needarch outside a need, an :extra: pair whose value contains
a colon, and a template error anywhere in the body.
:save: is accepted and not yet written — the option is read and its path
validated, but no .puml file is produced;
needs_build_needumls, the whole-project dump, stays out for a
different reason (an export is its own command in ubCode, not a build side effect).
The differences page lists the rest.
list2need is implemented by ubCode’s own engine as well, and
without the Sphinx-Needs limitations: each bullet becomes a real need, every one
of them carries the source position of the item you wrote it on, and it works in
a Markdown document, where a Sphinx build creates no needs at all.
Its :presentation: option is read and reported rather than applied — see
differences.
Five of the views below — needlist, needtable, needflow, needpie and
needbar — additionally accept a ubCode-only :cypher: option, which selects needs by
graph query rather than by field filter. Sphinx-Needs parses and ignores it on the first
three from 8.4.0 onwards, and on both charts from 8.5.0
(sphinx-needs#1818); it documents that
the option is ignored, and that on a chart the no-op changes a rendered number, so
Selecting with Cypher is where the query language itself is taught.
On a chart the option is a scope — the universe each content-cell filter is counted in
— rather than a selection, and both charts read that scope in a second, PORTABLE spelling
too: :filter:, :status:, :tags: and :types:. :cypher: wins when both
are written, which is what makes writing both the recommended form: ubCode evaluates the
query and a Sphinx build scopes by the python surfaces, so the two engines show the same
numbers wherever the scope has a python spelling — as far as the SCOPE goes: a chart’s
counts cover needs only here and needs and their parts in a Sphinx build, so a project
using parts still reads lower values (Needs chart views (needpie / needbar)). A bare :cypher: alone still diverges
— a Sphinx build counts each cell over the whole project — and a link traversal has no
python spelling at all (A chart’s filter options SCOPE its slices). Those four options are
the chart’s scope in Sphinx-Needs from 8.5.0; on 8.4.0 and earlier they
are unknown options there and the directive is dropped.