Writing a filter¶

Filter queries are used to filter the results of need collection, for example in the Needs Index tree view.

They can take one of two forms:

A cypher based syntax
A python based syntax

Cypher syntax¶

This syntax of a filter query uses a subset of the Cypher query language.

See full syntax grammar

Written in pest, the grammar for the filter query language is as follows:

// A grammar for parsing the ubquery language
// This query language aims to be a strict subset of open cypher (https://opencypher.org/resources/, https://neo4j.com/docs/cypher-manual/current/clauses/where/#where-introduction)
// see also https://github.com/a-poor/open-cypher/blob/main/src/cypher.pest

start = { SOI ~ ws* ~ or_expr ~ ws* ~ EOI }

or_expr = { and_expr ~ (ws+ ~ or_keyword ~ ws+ ~ and_expr)* }
or_keyword = _{ ^"OR" }

and_expr = { (expr | not_expr) ~ (ws+ ~ and_keyword ~ ws+ ~ (expr | not_expr))* }
and_keyword = _{ ^"AND" }

expr = { paren_expr | var_field_op_expr | literal_in_var_field_expr }

not_expr = { not_keyword ~ ws+ ~ (paren_expr | var_field_op_expr | literal_in_var_field_expr) }
not_keyword = _{ ^"NOT" }

paren_expr = { "(" ~ ws* ~ or_expr ~ ws* ~ ")" }

literal_in_var_field_expr = { literal_single ~ ws+ ~ in_keyword ~ ws+ ~ var_field }

var_field_op_expr = {
  (var_field | var_field_function) ~
  (regex_match_expr | string_predicate_expr | in_list_expr | is_null_expr | is_not_null_expr | comparison_expr)?
}

regex_match_expr = { ws* ~ regex_match_keyword ~ ws* ~ string_literal }
regex_match_keyword = { "=~" }

comparison_expr = {
  ws* ~ equals_keyword ~ ws* ~ (literal | var_field) |
  ws* ~ not_equals_keyword ~ ws* ~ (literal | var_field) |
  ws* ~ less_than_keyword ~ ws* ~ (number_literal | var_field) |
  ws* ~ greater_than_keyword ~ ws* ~ (number_literal | var_field) |
  ws* ~ less_than_or_equals_keyword ~ ws* ~ (number_literal | var_field) |
  ws* ~ greater_than_or_equals_keyword ~ ws* ~ (number_literal | var_field)
}

equals_keyword = { "=" }
not_equals_keyword = { "<>" }
// Note: regex_match_keyword (=~) is defined separately for clarity
less_than_keyword = { "<" }
greater_than_keyword = { ">" }
less_than_or_equals_keyword = { "<=" }
greater_than_or_equals_keyword = { ">=" }

is_null_expr = { ws+ ~ is_keyword ~ ws+ ~ null_keyword }
is_not_null_expr = { ws+ ~ is_keyword ~ ws+ ~ not_keyword ~ ws+ ~ null_keyword }

string_predicate_expr = {
  (ws+ ~ starts_with_keyword | ws+ ~ ends_with_keyword | ws+ ~ contains_keyword) ~ ws* ~ string_literal
}
starts_with_keyword = { ^"STARTS WITH" }
ends_with_keyword = { ^"ENDS WITH" }
contains_keyword = { ^"CONTAINS" }

in_list_expr = { ws+ ~ in_keyword ~ ws+ ~ (list_literal | var_field) }
in_keyword = _{ ^"IN" }

var_field = { symbolic_name_simple ~ "." ~ (symbolic_name_simple | symbolic_name_quoted) }

symbolic_name_simple = @{ id_start ~ id_part* }
id_start = @{ "_" | ASCII_ALPHA }
id_part = @{ id_start | ASCII_DIGIT }

symbolic_name_quoted = @{ "`" ~ (!"`" ~ ANY)* ~ "`" }

function_name = { ^"upper" | ^"lower" | ^"size" }
var_field_function = { function_name ~ "(" ~ var_field ~ ")" }

literal_single = { null_literal | boolean_literal | number_literal | string_literal }
literal = { null_literal | boolean_literal | number_literal | string_literal | list_literal }

null_literal = { ^"NULL" }
boolean_literal = { true_literal | false_literal }
true_literal = { ^"TRUE" }
false_literal = { ^"FALSE" }

number_literal = { float_literal | decimal_literal | integer_literal }
integer_literal = @{ "-"? ~ ("0" | ASCII_NONZERO_DIGIT ~ ASCII_DIGIT*) }
decimal_literal = @{ integer_literal ~ "." ~ ASCII_DIGIT* }
float_literal = @{ integer_literal ~ exp | decimal_literal ~ exp? }
exp = @{ ^"E" ~ integer_literal }

string_literal = { string_single | string_double }
string_single = @{ "'" ~ string_single_char* ~ "'" }
string_single_char = @{ "\\" ~ ANY | !("'" | "\\") ~ ANY }
string_double = @{ "\"" ~ string_double_char* ~ "\"" }
string_double_char = @{ "\\" ~ ANY | !("\"" | "\\") ~ ANY }

list_literal = {
  "[" ~ ws* ~ (literal_single ~ ws* ~ ("," ~ ws* ~ literal_single ~ ws*)*)? ~ "]"
}

is_keyword = _{ ^"IS" }
null_keyword = _{ ^"NULL" }

ws = _{ " " | "\t" }

Variables and attributes¶

The following variables are available in the filter query:

n: The node representing the need.
l: The node representing a link to another need. The main attribute of a link is type, i.e. l.type.
o: The node representing a linked need.

The graph would look like: (n)-[l]->(o).

You can access the attributes of a node using the dot operator, for example n.title.

If the attribute contains spaces or other non-ascii characters, you can wrap the attribute name in backticks, for example n.`my attribute`.

In addition to node attributes, you can query by the source type of the node:

n.is_directive: True if the need originated from a directive.
n.is_external: True if the need originated from the external_needs configuration.
n.is_import: True if the need originated from a needimport directive.

You can also query if the need is modified by one or more needextend directives using the n.is_modified attribute.

Values¶

Values can be:

Strings (in single or double quotes), e.g. "my value" or 'my value'.
Numbers, e.g. 42, 3.14, or 0.1e-3.
Booleans, e.g. true or false (case-insensitive).
Null, e.g. null (case-insensitive).
Other attributes, e.g. n.title.
Lists of values, e.g. ["value1", "value2"].

Comparison operators¶

The following boolean operators are available:

n.attribute = value: Equal to.
n.attribute <> value: Not equal to.
n.attribute < value: Less than.
n.attribute <= value: Less than or equal to.
n.attribute > value: Greater than.
n.attribute >= value: Greater than or equal to.

String operators¶

The following string operators are available:

n.attribute contains "value": Contains the string.
n.attribute starts with "value": Starts with the string.
n.attribute ends with "value": Ends with the string.

Regular expressions¶

You can use the =~ operator to match against a regular expression:

n.attribute =~ 'pattern'

The pattern is compiled at parse time, so invalid patterns will produce an error immediately.

Examples:

n.id =~ '^REQ-[0-9]+$'           # Match requirement IDs
n.status =~ 'open|closed'        # Match multiple values
n.title =~ '(?i)test'            # Case-insensitive match
lower(n.name) =~ 'pattern'       # Apply function before matching
not n.field =~ 'pattern'         # Negated match

See the Neo4j regex documentation for more details on the syntax.

Null operators¶

The following null operators are available:

n.attribute is null: Is null.
n.attribute is not null: Is not null.

List operators¶

The following list operators are available:

n.attribute in ["value1", "value2"]: Attribute is in the list.
"value" in n.attribute: Value is in a list type attribute.

Attribute functions¶

Some functions are available to modify attributes before comparison:

lower(n.attribute): Convert string attribute to lower-case.
upper(n.attribute): Convert string attribute to upper-case.
size(n.attribute): Get the length of a string or list attribute.

Logical operators¶

The following logical operators are available:

not: Negate the following expression.
and: Logical AND.
or: Logical OR.

Parentheses¶

You can use parentheses to group expressions, for example:

(n.attribute1 = "value1" or n.attribute2 = "value2") and n.attribute3 = "value3"

not can be used to negate an entire group, for example:

not (n.attribute1 = "value1" or n.attribute2 = "value2")

Examples¶

To filter for all needs that start with a case-insensitive string:

lower(n.title) starts with "my string"

To filter for all needs that are linked to a need of type requirement by a link of type needs:

l.type = "needs" and o.type = "requirement"

Python syntax¶

This syntax of a filter query uses a subset of the Python expression syntax.

See full syntax grammar

Written in pest, the grammar for the filter query language is as follows:

// A grammar for supporting a subset of python expressions

start = { SOI ~ ws* ~ or_expr ~ ws* ~ EOI }

or_expr = { and_expr ~ (ws+ ~ or_keyword ~ ws+ ~ and_expr)* }
or_keyword = _{ "or" }

and_expr = { (expr | not_expr) ~ (ws+ ~ and_keyword ~ ws+ ~ (expr | not_expr))* }
and_keyword = _{ "and" }

expr = { paren_expr | this_doc_check | search_check | var_field_op_expr | literal_in_var_field_expr }
not_expr = { not_keyword ~ ws+ ~ expr }
not_keyword = _{ "not" }

paren_expr = { "(" ~ ws* ~ or_expr ~ ws* ~ ")" }

literal_in_var_field_expr = { literal_single ~ ws+ ~ in_keyword ~ ws+ ~ var_field_with_func }

var_field_op_expr = {
  var_field_with_func ~
  (in_list_expr | not_in_list_expr | is_null_expr | is_not_null_expr | comparison_expr | str_predicate_method)?
}

comparison_expr = {
  ws* ~ equals_keyword ~ ws* ~ (literal | var_field_with_func) |
  ws* ~ not_equals_keyword ~ ws* ~ (literal | var_field_with_func) |
  ws* ~ less_than_keyword ~ ws* ~ (number_literal | var_field_with_func) |
  ws* ~ greater_than_keyword ~ ws* ~ (number_literal | var_field_with_func) |
  ws* ~ less_than_or_equals_keyword ~ ws* ~ (number_literal | var_field_with_func) |
  ws* ~ greater_than_or_equals_keyword ~ ws* ~ (number_literal | var_field_with_func)
}

equals_keyword = { "==" }
not_equals_keyword = { "!=" }
less_than_keyword = { "<" }
greater_than_keyword = { ">" }
less_than_or_equals_keyword = { "<=" }
greater_than_or_equals_keyword = { ">=" }

is_null_expr = { ws+ ~ is_keyword ~ ws+ ~ null_keyword }
is_not_null_expr = { ws+ ~ is_keyword ~ ws+ ~ not_keyword ~ ws+ ~ null_keyword }

in_list_expr = { ws+ ~ in_keyword ~ ws+ ~ (list_literal | var_field) }
not_in_list_expr = { ws+ ~ "not" ~ ws+ ~ in_keyword ~ ws+ ~ (list_literal | var_field) }

var_field = { symbolic_name_simple }

var_field_with_func = { var_field_with_len | var_field_with_upper | var_field_with_lower | var_field }
var_field_with_len = { ("len(") ~ var_field ~ (")") }
var_field_with_lower = { var_field ~ (".lower()") }
var_field_with_upper = { var_field ~ (".upper()") }

reserved = { "None" | "True" | "False" | "and" | "or" | "not" | "in" | "is" }
symbolic_name_simple = @{ !(reserved ~ (ws | EOI)) ~ id_start ~ id_part* }
id_start = @{ "_" | ASCII_ALPHA }
id_part = @{ id_start | ASCII_DIGIT }

literal_single = { null_literal | boolean_literal | number_literal | string_literal }
literal = { null_literal | boolean_literal | number_literal | string_literal | list_literal }

boolean_literal = { true_literal | false_literal }
null_literal = { "None" }
true_literal = { "True" }
false_literal = { "False" }

number_literal = { float_literal | decimal_literal | integer_literal }
integer_literal = @{ "-"? ~ ("0" | ASCII_NONZERO_DIGIT ~ ASCII_DIGIT*) }
decimal_literal = @{ integer_literal ~ "." ~ ASCII_DIGIT* }
float_literal = @{ integer_literal ~ exp | decimal_literal ~ exp? }
exp = @{ ^"E" ~ integer_literal }

string_literal = { string_single | string_double }
string_single = @{ "'" ~ string_single_char* ~ "'" }
string_single_char = @{ "\\" ~ ANY | !("'" | "\\") ~ ANY }
string_double = @{ "\"" ~ string_double_char* ~ "\"" }
string_double_char = @{ "\\" ~ ANY | !("\"" | "\\") ~ ANY }

list_literal = {
  "[" ~ ws* ~ (literal_single ~ ws* ~ ("," ~ ws* ~ literal_single ~ ws*)*)? ~ "]"
}

str_predicate_method = {"." ~ str_predicate_method_name ~ "(" ~ string_literal ~ ")"}
str_predicate_method_name = { "startswith" | "endswith" }

this_doc_check = { "c.this_doc()" }

search_check = { "search(" ~ string_literal ~ "," ~ ws* ~ var_field_with_func ~ ")" }

in_keyword = _{ "in" }
is_keyword = _{ "is" }
null_keyword = _{ "None" }

ws = _{ " " | "\t" | "\n" }

Variables¶

Variables relate to attributes of the need, e.g. id or title

In addition to node attributes, you can query by the source type of the node:

is_directive: True if the need originated from a directive.
is_external: True if the need originated from the external_needs configuration.
is_import: True if the need originated from a needimport directive.

You can also query if the need is modified by one or more needextend directives using the is_modified attribute.

Truthiness¶

Variables can be used directly in boolean context using Python truthiness rules:

Numbers: 0 and 0.0 are falsy, all other numbers are truthy
Strings: Empty string "" is falsy, all other strings (including whitespace) are truthy
Lists: Empty list [] is falsy, lists with any items are truthy
Null: None (null) values are falsy
Booleans: True is truthy, False is falsy

Examples:

title                    # True if title is not empty, False, 0, or None
not title                # True if title is empty, False, 0, or None
tags                     # True if tags list has items
not tags                 # True if tags list is empty

Values¶

Values can be:

Strings (in single or double quotes), e.g. "my value" or 'my value'.
Numbers, e.g. 42, 3.14, or 0.1e-3.
Booleans, e.g. True or False.
Null, e.g. None.
Other attributes, e.g. title.
Lists of values, e.g. ["value1", "value2"].

Comparison operators¶

The following boolean operators are available:

attribute == value: Equal to.
attribute != value: Not equal to.
attribute < value: Less than.
attribute <= value: Less than or equal to.
attribute > value: Greater than.
attribute >= value: Greater than or equal to.

Note that number types (integers and floats) can be compared with each other, and 1 == 1.0 evaluates to True. Mixed type comparisons (e.g., string to number) are strict and will return False for equality.

String operators¶

The following string mutators and comparators are available:

attribute.lower(): Convert string attribute to lower-case before comparison.
attribute.upper(): Convert string attribute to upper-case before comparison.
"value" in attribute: Check if string contains the value (substring match).
attribute.startswith("value"): Check if string starts with the value.
attribute.endswith("value"): Check if string ends with the value.

Note these can be combined, e.g. attribute.lower().startswith("value") or used with the in operator like "TEST" in attribute.upper().

String operations are case-sensitive unless you use .lower() or .upper() methods.

Built-in functions¶

The following built-in functions are available:

len(attribute): Get the length of a string or list attribute.
search(pattern, attribute): Test if the attribute matches a regular expression pattern.

Examples:

len(title) > 10          # Title has more than 10 characters
len(tags) == 0           # Tags list is empty
len(description) >= 100  # Description is at least 100 characters

Regular expression search¶

The search(pattern, attribute) function tests if a string attribute matches a regular expression pattern. It uses Rust regex syntax, which is compatible with most common regex features.

The function returns True if the pattern is found anywhere in the attribute value, False otherwise.

# Simple text search
search('test', title)                    # Title contains "test"

# Case-insensitive search
search('(?i)urgent', description)        # Description contains "urgent" (case-insensitive)

# Word boundaries
search('\\bREQ\\b', id)                  # ID contains "REQ" as a complete word

# Email pattern matching
search('[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}', author)

# URL pattern matching
search('https?://[^\\s]+', description)  # Description contains a URL

# Digit patterns
search('\\d{4}', id)                     # ID contains 4 consecutive digits

# Combined with other operations
search('^REQ-', id) and status == 'open' # ID starts with "REQ-" and status is open

# With string functions
search('test', title.lower())            # Search in lowercase version of title

# Negation
not search('deprecated', description)    # Description does not contain "deprecated"

Note

The search() function only works with string attributes. Lists and other types will result in an error. Use .lower() or .upper() with the attribute if you need case-insensitive matching across the entire string, or use the (?i) flag at the start of the regex pattern.

Warning

Complex regular expressions may impact performance when filtering large numbers of needs. Consider using simpler string operations like in, .startswith(), or .endswith() when possible.

Null operators¶

The following null operators are available:

attribute is None: Is null.
attribute is not None: Is not null.

List operators¶

The following list operators are available:

attribute in ["value1", "value2"]: Attribute is in the list.
"value" in attribute: Value is in a list type attribute.
"value" not in attribute: Value is not in a list type attribute.

Logical operators¶

The following logical operators are available:

not: Negate the following expression.
and: Logical AND.
or: Logical OR.

Parentheses¶

You can use parentheses to group expressions, for example:

(attribute1 == "value1" or attribute2 == "value2") and attribute3 == "value3"

not can be used to negate an entire group, for example:

not (attribute1 == "value1" or attribute2 == "value2")

Examples¶

Here are some comprehensive examples demonstrating various features:

Basic filtering:

# Filter by status
status == 'open'

# Filter by priority with number comparison
priority > 5

# Check if title is not empty
title

String operations:

# Case-insensitive search
type.lower() == 'requirement'

# Check if description contains specific text
'security' in description.lower()

# Title starts with specific prefix
title.startswith('REQ-')

List operations:

# Check if status is one of several values
status in ['open', 'in_progress', 'review']

# Check if tags contain a specific tag
'critical' in tags

# Exclude items with certain tags
'deprecated' not in tags

Complex conditions:

# Multiple conditions with AND
type == 'requirement' and status == 'open' and priority >= 7

# Multiple conditions with OR
priority >= 8 or 'critical' in tags or 'security' in description.lower()

# Grouping with parentheses
(status == 'open' or status == 'in_progress') and assignee != None

Length and null checks:

# Check for non-empty descriptions
description is not None and len(description) > 50

# Filter items with tags
len(tags) > 0

# Unassigned items
assignee is None

Real-world scenarios:

# High priority open requirements
type == 'requirement' and status == 'open' and priority >= 8

# Security-related items needing attention
('security' in title.lower() or 'CVE' in description) and status != 'closed'

# Requirements ready for review
type == 'requirement' and status == 'draft' and len(description) >= 100 and assignee is not None

Special functions¶

The following special functions are available:

c.this_doc(): True if the need is in the same document as the query originates from.