diff --git a/deploy/mac/check_app.sh b/deploy/mac/check_app.sh index 59df2fc..a79e796 100755 --- a/deploy/mac/check_app.sh +++ b/deploy/mac/check_app.sh @@ -454,6 +454,20 @@ if [ "$RUN" = 1 ]; then } >"$T/vtdiag.txt" 2>&1 || true sed 's/^/ /' "$T/vtdiag.txt" fi + # The same with a words condition, whose FTS5 phrase quotes reach + # gufi_vt written as char(34): the OCR'd word in the main folder. + if [ "${ocr_found:-0}" = 1 ]; then + curl -fsS -m 60 -X POST -H 'content-type: application/json' \ + -d '{"library": "all", "builder": {"source": "files", "filters": [{"field": "text", "op": "match", "value": "heliotrope"}], "groupBy": "none", "sort": "size", "dir": "desc", "limit": 20}}' \ + "http://127.0.0.1:$PORT_FREE/api/query" >"$T/qwords.json" 2>"$T/qwords.err" || true + if python3 -c 'import json, sys; d = json.load(open(sys.argv[1])); found = any("ocr-sample" in str(r) for r in d.get("rows") or []); sys.exit(0 if d.get("engine") == "gufi_vt" and found and not d.get("failed") else 1)' "$T/qwords.json" 2>/dev/null; then + ok "and a query with a words condition runs through gufi_vt too" + else + bad "a query with a words condition did not run through gufi_vt:" + python3 -c 'import json, sys; d = json.load(open(sys.argv[1])); print("engine", d.get("engine"), "why", d.get("mergeReason"), "rows", len(d.get("rows") or []), "failed", d.get("failed"))' "$T/qwords.json" 2>&1 | sed 's/^/ /' + sed 's/^/ /' "$T/qwords.err" || true + fi + fi else bad "could not add a second library to query across" fi diff --git a/docs/REFERENCE.md b/docs/REFERENCE.md index 409199f..3c75f9b 100644 --- a/docs/REFERENCE.md +++ b/docs/REFERENCE.md @@ -58,7 +58,7 @@ The details behind the user guide ([`HELP.md`](HELP.md)), by part of the app. Th - Across all libraries, search runs each library in parallel and merges the results. Settings, General, Indexing, "Search across libraries" switches it to `gufi_vt`, as Query uses: name matches, and full-text matches where every search word is plain letters and digits, then run as one `gufi_vt` statement per kind, shown in the GUFI commands panel, while a word with punctuation, a library `gufi_vt` cannot reach, and results by meaning still run per library. Both ways return the same matches. `gufi_vt` answers its libraries one after another, so the parallel default is faster. The environment variable `STUDIO_SEARCH_ENGINE=gufi_vt` sets the default for a deployment. - In the parallel default each library has 20 seconds to answer (`STUDIO_SEARCH_DEADLINE_MS`). One that has not is left out and `GET /api/search` names it in `slow`; choosing that library alone waits for it. - A remote host whose login runs only `gufi_query` (`server/src/remoteExec.ts`): when the session loop is refused there (no bash, a restricted shell, a forced command), the site is marked direct and each command is its own ssh call over the shared connection, the program name bare and each other argument a POSIX shell word; only `gufi_query` and `gufi_sqlite3` are sent, and anything else answers 409 ` runs only gufi_query for this login (no shell), so this cannot run there.` The connection tests (`POST /api/user-libraries/ssh/test`, `.../pw/test`) answer `shell: false` there, the library is saved with `shell: false` and `gufiBin` `''` when `gufi_query` runs from the login's PATH, and a build check answers its ssh step with "This host runs only gufi_query: connect existing indexes read only; building an index there is not available." A connected index whose top levels have a `words` table is saved with `fullText`, so search reads its words. On macOS the run loop passes on a `KRB5CCNAME` the login shell sets, for ssh with Kerberos. Details in [`LIBRARIES.md`](LIBRARIES.md). -- No library drops out silently (`server/src/omissions.ts`). `GET /api/search` and `POST /api/query` carry `omitted` when any library in scope was left out, or searched only in part: one entry per library, `{ id, label, reason, partial }`, `partial` false when it added nothing. The reasons: `being built; searchable once its first pass is done`; `its build failed: ...`; `index not found`; `words not searched yet: its text is being read`, or `words not searched: its index has no text yet` (a library built at a depth that reads text whose index has no `words` table yet); `not searched by meaning: its index has no vectors yet`, for a search by meaning; `did not answer within 20 s`; `not reachable over ssh: ...`; `search failed: ...`; on macOS `macOS has not allowed ACTIVATE Studio to read ; allow it in System Settings, Privacy & Security, Files and Folders (or Full Disk Access)` (elsewhere `permission denied reading `), with `; results are from its last pass` when its index still answers; and `a folder could not be read (...)` or `N folders could not be read (...)` for folders inside it. Query names a library with no extracted text when the query reads the `words` table (`its index has no extracted text, so text conditions match nothing there`). A library with nothing to answer from is not asked: across libraries it is in `omitted` and in Query's `failed`; asked alone, Query answers 409 `