[Matthew Shaxted (mattshax)]

Search reads each library's index once for every kind of match: names, files below a matching folder, the Studio's words, the Studio's vectors, and external databases in the shared text and vector schema. The kinds are fused with reciprocal rank fusion. This follows the one-pass pattern GUFI's authors recommend: one gufi_query whose per-directory step adds every kind while the directory's tables are open, merged per thread and ranked once at the end.

What changed

• walkSearch (server/src/gufi.ts): one aggregate gufi_query per library, or per remote host, with these stages:

  • -I: a per-thread table, plus the query's embedding as a blob literal, one copy per thread. That works on every GUFI in use; --global-db would do the same on newer ones.

  • -E: four statements at most, in a fixed order:

    1. a trim that keeps each kind's best;
    2. names and folder-path matches in one insert;
    3. the Studio's words (fts5 words, by bm25);
    4. the Studio's vectors (vec0 gvec, by distance), with the query vector picked by the directory's gvecmodel.

    A directory without a words or vector table fails the step that needs it, and only the later steps are skipped there. No empty tables are written. gufi_query compiles every statement again in every directory, so there are few.

  • --no-print-sql-on-err: each such failure prints one short line, and only the last 16 KB of error output is kept. A GUFI from before 2026 that does not know it, or --external-copy, is asked again without them.

  • -G: ranks each kind, keeps each file once per kind and to the kind's limit, and fuses the kinds with RRF, k = 60.

• fuseRows: the same fusion across libraries and walks. Ties go to the more exact kind (words, name, folder, meaning), then the path, so the order is deterministic. Hits carry mode (the most exact kind, now including path), kinds (every way the file was found) and score.

• Folder-path matches: files below a folder whose path below the library root matches, with the same term rules as names. A file whose name matches counts as a name match only.

• External databases in the shared schema, read with --external-copy. Each file in an index directory other than db.db whose name matches STUDIO_EXTERNAL_DB_PATTERN (default \.db$; empty turns it off) is attached, and two tables are read:

  • ext_bm25_by_file: text by bm25.
  • ext_vectors_by_file: for a vector_model_name whose query vector the Studio has (all-MiniLM-L6-v2, which the bundled ONNX model quantizes), by cosine distance put on vec0's scale.

  Rows join the directory's entries on binode and bmtime, so a file changed since extraction is not matched. A library's "words not searched" note is withdrawn when its external databases answered.

• Remote: libraries on one host and account share one gufi_query with their roots stacked, one round trip per host. Before, each library ran one remote command for names and words, and none for meaning. Rows go back to their library by the root they are under. A walk past its deadline names every library in it.

• Local: libraries keep a walk each. Stacked local roots measured no faster; STUDIO_STACK_LOCAL=1 turns stacking on.

• Fallback for meaning: a library whose index has vectors but whose walk returns none (embedded with the earlier GGUF model) is searched by meaning the earlier way. The gufi_vt search setting keeps its earlier path.

• The assistant's search_kb uses searchScope's fused order directly.

• The Search page labels a folder-path match "folder", and the label's tooltip names every way a hit was found.

Measurements

On a synthetic index of 24,000 directories and 100,000 files, words in a quarter of the directories and vectors in a tenth, warm cache, median of five, on a shared host (load average 1 to 10):

Search	Before	After
Names, words and meaning	about 2.9 s (the name and word walk, 0.7 to 0.8 s, beside the meaning search, 2.9 s: a file walk for databases with vectors, then one attach each)	about 1.0 s in one walk
Names, words and folder paths, no vectors	0.7 s (names and words)	0.85 s
Same index rolled up (1 directory left to walk)	3.0 s	0.24 s, names and paths only (see below)
Four libraries of 6,000 directories	1.17 s	0.83 to 0.85 s, the same stacked or one walk each

What each part adds to the walk: folder-path matching about 0.15 s, vectors about 0.1 s, the external database lookup about 0.05 s.

Remote, by round trips: one per host per search, with meaning included where the remote index has vectors. Before, it was one per library, with no meaning.

Notes

• A rolled-up index's walk does not visit directories rolled into their parents, so their own tables are not read there; names and paths are. The Studio does not roll up the indexes it builds.
• Empty tables were measured and not written. An empty fts5 words table takes a 48 KB directory database to 68 KB, and empty vector tables take it to 108 KB. That is about 0.4 GB and 0.8 GB more on this test index.

Tests

• server/test/searchOnePass.test.mjs, with real GUFI where installed:
  • RRF order and ties;
  • statement order;
  • a tree with words in some directories only;
  • folder-path matches below the root only;
  • meaning in the same walk, and a model with no query vector failing alone;
  • the shared external schema by text and by model, with a stale mtime not matched;
  • several roots in one walk mapped back.
• server/test/searchStackedRemote.test.mjs: two libraries on one host give one remote gufi_query with both roots, another host has its own, and a failing host names its library.
• Updated fakes in libraryScope.test.mjs.
• Web: the folder label and the tooltip.
• Server suite: 619 passed, 9 skipped, including the restricted-host test (Libraries over ssh work where the login runs only gufi_query #594), whose search sends only gufi_query. Web suite: 278 passed.