Queries and search¶

Lists and search read an index of the issues and people, kept in sqlite on the machine, or, for a board kept in PostgreSQL, in that database (ADR 0002, How the data flows). This page says where it lives, how it stays fresh, what a query asks, and how a program hears that entities changed.

Where it lives¶

For a board kept in git, in Markdown files or on the forges alone, the index is one sqlite file, <git common dir>/board/index.db. The git common dir is what git rev-parse --git-common-dir gives, so every worktree of a repository shares one index. The core creates the board directory when it is missing.

• It is never the source of truth. It holds only what the refs hold. Deleting it loses nothing: the next query builds it again.
• It is local. It lives in the git directory, so git never commits, fetches nor pushes it.
• Two processes can use it at once, like git-board's server and a command. The file is in WAL mode, so a read doesn't wait for a write, and a write waits up to 5 seconds for another (sqlite's busy_timeout).

For a board kept in PostgreSQL, the index is the same tables in the schema git_board_index_v<version> of the board's database (today git_board_index_v3), shared by every machine. Each version of the index's tables has its own schema, so two git-board versions sharing a database each keep their index; neither drops the other's. A schema no git-board reads any more stays until it is dropped by hand (DROP SCHEMA git_board_index_v1 CASCADE; git_board_index, before versions had their own). What differs from sqlite:

• A different rebuild: an index of another version is dropped and made again (DROP TABLE) under an advisory lock, not deleted as a file.
• A different text search: a tsvector with the simple configuration, the text tokenized as sqlite's unicode61 does, so a query finds the same issues; one issue's indexed text is cut at 300,000 bytes. The title, body and comments have their own weight (A, B, C), for in.
• The same order: Labels sorts byte for byte (COLLATE "C"), whatever the database's collation.
• Several writers: a refresh takes a table lock, so two machines refreshing at once write one after the other.

What it holds¶

Issues and persons. Sync entities aren't indexed.

• For each issue: its id, title, status, author, created, updated, labels and number of comments, from its state. For search, the text of its title, body and comments. For lists, the whole state, as JSON: a listing reads it instead of every issue's packs, and decodes only the issues it doesn't hold at that head.
• For each person: its id, name and accounts (server, id, login), from its state.
• For each entity: its head, the commit its ref points at.
• For each broken entity: its head and its error (see Freshness).

Freshness¶

A refresh brings the index up to the refs:

1. It lists refs/board/issue/* and refs/board/person/*. A ref whose name isn't a valid id isn't an entity, and is ignored (see Refs).
2. An entity that is new, or whose head differs from the stored one, is loaded again, and its rows replaced.
3. An entity whose ref is gone is dropped.
4. The writes are one transaction, which skips an entity whose stored head changed since step 2 read it: another refresh wrote it meanwhile, maybe a newer head than this one loaded. A head left behind is loaded again by the next refresh.

Every query refreshes first. So it answers from the refs as they are, whoever moved them: this process, another one, or a fetch.

An entity that fails to load is broken. The refresh drops its rows, so no query or lookup answers it, and still keeps the rest of its changes.

• The index records it, with its head and its error. Broken() lists the broken entities, each with its kind, id and error, after a refresh like every query.
• It is loaded again when its head moves, or at a rebuild. An entity whose ref is gone is dropped, broken or not.
• A query never fails because of one broken entity. It answers without it. A refresh that can't list the refs, or can't write the index, still fails the query.

Rebuild¶

The index holds its schema version in PRAGMA user_version. Opening it, the core deletes the file and builds it again from the refs when:

• the file is missing;
• sqlite can't read it as a database;
• its version isn't the code's.

Its -wal and -shm files go with it.

Queries¶

A query asks for issues. Each field is optional: an empty one filters nothing.

An issue must match every field given. The answer is ordered and cut by three more:

• Issues with the same time go by id, in the same direction.
• Another status, sort or text in in, or a negative limit, is an error.
• Each issue comes as its summary: what the index holds for it.

Text¶

The text is searched in each issue's title, body and comments, as their last edits left them.

• Words. The text splits on whitespace into words. An issue matches when every word matches, each anywhere in its title, body or comments.
• Prefixes. A word matches the start of a word: crash matches Crashes on save.
• Case and diacritics don't count: cafe matches Café. Words are cut as FTS5's default tokenizer, unicode61, cuts them: letters and digits make up a token, and anything else separates tokens.
• A word of several tokens, like title:crash, matches those tokens in a row, the last as a prefix.
• A word with no token, like -, is left out. A text with no word left matches every issue.
• FTS5's own syntax is never read. Each word is quoted, so ", *, -, :, AND, OR, NOT and parentheses are plain text.
• in narrows where. Each word must then match in one of the texts it names. Without the index, search.Match answers the same over an issue's texts: what git board ls --at searches with.
• On PostgreSQL (a board kept there, ADR 0002), an issue's text is indexed up to its first 300,000 bytes of tokens: a tsvector holds at most 1 MB, and one issue's text never fails a query. A word past that isn't found.

Labels¶

Labels answers every label in use: each label at least one issue has, sorted byte for byte, each once, whatever a PostgreSQL database's collation. A label no issue has anymore isn't one.

Prefixes¶

A prefix of an issue's or a person's id, as people type them, names the entity whose id starts with it.

• It is 1 to 40 lowercase hex digits. Anything else is an error, so no character acts as a pattern.
• One entity matches: it answers that id.
• None matches: it is an error that says none does.
• Several match: it is an error that says the prefix is ambiguous. A whole id never is.
• It reads the index, after a refresh like every query, so a broken entity matches nothing.

Accounts¶

A lookup by account, a server and an id, answers the person who owns it, by the rule in Who owns an account. When no person holds the account, it answers none.

Change events¶

Watch tells a program, like git-board's server or its TUI, that entities changed.

• It polls. Each poll lists refs/board/<kind>/* for each kind, and compares each head with the one it saw last. It ignores a ref whose name isn't a valid id.
• The first poll is the baseline. It runs at once, when Watch starts, and reports nothing.
• Each later poll runs one interval after the last, and reports what changed since. An interval of 0 or less means 1 second.
• A poll whose listing of the refs fails is skipped: it reports nothing, and the next interval tries again. A poll is skipped as a whole when listing any kind fails. The next poll compares with the last poll that succeeded, and a skipped baseline leaves the next poll as the baseline.
• A poll's changes come together: each entity that appeared, moved, or was removed, with its kind, id and new head. A removed one has the empty version, "", as its head (ADR 0002: a version is opaque). They are sorted by kind, then id. A poll with no change reports nothing.
• It reports heads, not writes. An entity written several times between two polls is reported once, with its head at that poll. One created and removed between two polls isn't reported.
• It sees every change, whoever made it: this process, another one, or a fetch.
• It stops only when its context ends, never on its own.