feat: an osw CLI and an MCP server for live OSL instances - #133
Conversation
Add an in-repo `osw[mcp]` extra and an `osw-mcp` stdio console script that wraps OswExpress and serves it over the Model Context Protocol for clients such as Claude Code. Tools: semantic/SPARQL/full-text search, category schema introspection, entity read + JSON-LD export, create/update/delete, full page-slot access, and file up/download. - Delete is provenance-guarded: a local JSON ledger records pages the server created/modified; deleting anything untracked requires confirm_external_delete=true. - Credentials resolve from env/.env and are validated up front (fail fast, never prompts, so stdio is never corrupted by an input() call). - osw stdout is redirected to stderr so it never leaks onto the JSON-RPC channel. - OSW_MCP_READ_ONLY hides all mutating tools.
b46f046 to
23db777
Compare
Release previewMerging this PR would release v2.6.0 (current: Changelog preview (truncated)## v2.6.0 (2026-09-21)
### Testing
- Decode subprocess output as UTF-8, not the locale codepage
([`55aab85`](https://github.com/OpenSemanticLab/osw-python/commit/55aab85fcc5c7c913222f62254e9ff24bfa5e49c))
Preview via python-semantic-release and conventional commits. |
|
Credential loading bypasses the
|
Multi-instance credentials: selecting an OSL instance per sessionFollowing on from the credential-file comment above. Pinning one The multi-account file already exists. One correction on the interactive part. Prompting on the CLI cannot work for Elicit the choice, never the secret. Elicited values pass through the MCP Concretely:
Multiple registrations already cover part of this, with no code change: claude mcp add osw-dev --env OSW_MCP_ENV_FILE=... -- uvx ...
claude mcp add osw-prod --env OSW_MCP_ENV_FILE=... --env OSW_MCP_READ_ONLY=true -- uvx ...That has two advantages over in-session switching: the instance is visible in the Supplying a |
LukasGold
changed the title
- FastMCP replaced by MCPServer, extra now requires mcp>=2 - mcp dropped from the all extra and the dev group: it needs anyio>=4.9, workflow pins anyio<4.7 (#139) - uv conflicts declare mcp exclusive with workflow and with dev - pytest stack moved to its own test group, so an environment with both pytest and mcp exists - src/osw/mcp excluded from ty, mcp tests guarded by importorskip
- OSW_MCP_CRED_FILEPATH configures it, OSL_CRED_FILEPATH is a fallback - an alternative to OSW_USERNAME/OSW_PASSWORD, so the password is not duplicated into a second plaintext file - existence and a matching domain entry are validated at startup - lookups use CredentialFallback.none, so osw never prompts and never blocks the stdio transport
- list_instances and select_instance tools, returning iris only and never any credential value - OSW_DOMAIN becomes optional when a credential file supplies the iris - auto-selects when OSW_DOMAIN is set or the file holds exactly one iri - switching rebuilds the connection and the per-domain provenance ledger - tools resolve the active domain and credentials at call time
|
All three addressed, one commit each: mcp>=1.2 resolves to 2.0.0Confirmed against the published wheel: The anyio clash you predicted is real, and it is a constraint the PR now has to Cost: the module is neither type-checked nor unit-tested in the default Credential loading bypasses the CredentialManager file patternAn iri-keyed credential file now configures the server, built into a Two things worth confirming:
Multi-instance credentialsAgreed on the constraint: elicited values pass through the client and into the Implemented as Note this piece was reversed later in the PR, see |
- move config/ledger/serialization from osw.mcp to osw.service - add errors.py (OpError), context.py (Context+Policy), registry.py - Operation validator rejects path-like params on the mcp surface - canonical OSW_* env names, OSW_MCP_*/OSL_* kept as aliases - config/ledger/serialization tests now run without the mcp extra
- add osw/service/ops/ with the search group as @operation functions - mcp/tools/search.py becomes a registry loop over bind() - add transitional legacy_context() so existing tests keep passing - bind() resolves annotations against the op module, not registry.py
- schema, status, entities and slots lifted as @operation functions - mcp/tools/*.py collapse to registry loops over bind() - error dicts become raised errors.*; ledger calls become records= hooks - normalize cli_name so every group reads as `osw <group> <verb>` - 172 passed in the dev env, 29 in the mcp extra env
- typer as a base dependency; osw = "osw.cli.main:app" console script - commands built from iter_operations(surface="cli"); Context built lazily - OpError.exit_code becomes the process exit status, no traceback - json_value parser lives in osw.service.params, so core never imports cli - set_slot coerces content per the sibling slot's content model
- add path-free get_file_info/read_file_text/write_file_text built on
WikiFileController.get()/.put(), never touching the local filesystem
- move file download/upload and ledger path to osw.cli.ops, the only
module allowed to name a path (surfaces={"cli"}); drop status's
ledger_path
- guard tests: no MCP-surfaced op names a path, and osw.mcp.server
never imports osw.cli
- read_file_text decodes incrementally so a byte cap splitting a
multi-byte character is not misreported as binary content
- delete list_instances/select_instance; each server process is pinned to one OSL instance and refuses to start if none resolves - map Operation's hints onto ToolAnnotations and _meta by explicit keyword, since the SDK silently absorbs a misspelled field name - set server instructions and version; make mcp.run(transport="stdio") explicit - add CLI --instance and osw instance list, returning iris only - retarget the "no instance selected" message at OSW_DOMAIN and --instance instead of the removed tool
Reversing in-session instance switching (
|
- delete osw/mcp/tools/ and connection.py; bodies live in osw.service.ops - fold registration.py into server.py, its sole consumer - replace test_mcp_tools.py with test_service_ops.py + test_mcp_server.py - rebuild the integration fixture on bind()/iter_operations - test_mcp_instances.py -> test_service_instances.py, no longer SDK-gated
- drop src/osw/mcp from [tool.ty.src] exclude - silence only the two unresolvable SDK imports inline (issue #139)
- add a Command line section: command tree, global options, exit behaviour - move credentials into a shared Configuration section with an alias table - state that no MCP tool takes a path, and where the path-based commands live - add stdio type, multi-instance and per-instance permission examples - give each CLI command group a one-line help string
- conflict in src/osw/wiki_tools.py: both sides appended after prefix_search, kept both additions - branch side: content_search - main side: _ask_results_as_dict, CONDITION_PATTERN, LIMIT_PARAM_PATTERN, get_query_limit
- cap_list only reports a cut it made itself, but all four search operations pass the same limit to the wiki, which never returns more, so `truncated` was always False - flag when the result set is as large as the effective limit - search_entities: a `limit=` inside the ask query wins over the limit argument, so compare against that one - sparql_query unchanged; its limit applies to the response, not the query
# Conflicts: # pyproject.toml # uv.lock
Docs
we should name it "Tools" in general in the nav bar including sidebar navigation to
Maybe like this (also adding CLI, MCP as their own sidebar elements 
CLI ImprovementsConfiguration Behaviorwhen following in OSW_CRED_FILEPATH='./accounts.pwd.yaml'
OSW_DOMAIN=wiki-dev.open-semantic-lab.org
Additionalswhen using CLI using osw instance wiki-dev.open-semantic-lab.orgbut status would be great when we use osw instance wiki-dev.open-semantic-lab.org statusor maybe better make instance a flag parameter in order to get status of multi instances osw status --instance wiki-dev.open-semantic-lab.org |
- replace the "CLI and MCP tools" tab with a "Tools" tab - split the single page into Overview, CLI, MCP and Configuration - leave room for a User Sync page from PR #174 - repoint the links in README.md and get-started.md
- status reports the username from a credential file, not only from OSW_USERNAME, and uses the precedence the login path uses - rename the "instance" group to "instances", unconfusable with --instance - add "osw instances status" to check every configured instance at once - name the correct form when a root option is typed after the command - extract config.get_credentials_for and Context.osw_for
- shared code hardcoded [osw] or [osw-mcp], so each adapter printed the other one's name about half the time - config.set_log_prefix/log_prefix; CLI sets osw, MCP server sets osw-mcp - covers the default delete and set_slot wiki edit comments, which are written permanently into page histories - add flush=True to the status connection-failure print, which CliRunner otherwise never captures because it flushes only stdout
- "instances" read as OSL instances, which is what osw instances lists - MCP tool name is the function name, so list_instances_of_category becomes search_entities and the ask query becomes search_ask - fix the help text, which documented [[HasType::Category:<category>]] while the real query for Category:Item is [[HasType::Category:Item]] - set the MCP prefix in the integration fixture that stands in for the server, so a future write test cannot tag a live wiki edit as [osw]
|
Addressed on DocsThe top-level tab is now "Tools", with the CLI and the MCP server as separate sidebar entries plus a page for the configuration they share:
User Sync is not included, because #174 is not in this branch. The section takes it as a fifth page with no restructuring once that PR merges. Username missing when it comes from
|
- README: keep both the Tools and the Logging sections - pyproject: take main's version 2.4.0, keep the mcp extra and osw-mcp script - uv.lock: regenerated
The tests assumed osw was imported after something had configured logging, which decides whether osw attaches its own handler on import. tests/conftest.py now imports osw, so that assumption no longer held and the hand-over test read the leftover import-time handler.
- six stderr prints in the shared code move to the osw logger - removes messages lost to an unflushed stderr write - the configuration source report stays a print, gated by --verbose - the MCP startup failure stays a print, so OSW_LOG_LEVEL=OFF cannot hide it
osw.cli.main subclasses click's Command and Group to report a misplaced root option, so click is imported directly. deptry flagged it as DEP003 and failed the quality job.
sys.stderr is line-buffered on every supported Python (>=3.10). The real reason is the caller-chosen stream: the MCP server passes an io.StringIO and click's CliRunner never flushes its stderr replacement.
- typer forces colour on when GITHUB_ACTIONS is set - rich styles an option's leading dash separately, so "--jsondata" is no longer a plain substring - rich wraps at the real terminal width, 80 in CI, 79 on Windows - strip the escape sequences and panel borders, then join the lines
- a redirected stream used cp1252 on Windows, so a German label reached the consumer as bytes no JSON parser could read, with exit code 0 - a character cp1252 cannot represent raised UnicodeEncodeError instead - the app callback reconfigures both streams before any output - errors= is passed too, or reconfigure would reset stderr to strict - --help and an unknown command name still bypass it; neither carries wiki content Closes #187
- osw and osw-mcp now start through the new src/osw_entry.py shim - the shim sets OSW_LOG_LEVEL before osw is imported, which is the only place that can, since importing any submodule imports the package first - setdefault leaves a value the caller already chose alone - a plain "import osw" still writes the notice, library behaviour is deliberately unchanged - the wheel ships osw_entry.py via only-include plus sources
- the osw-mcp console script had no test of its own, only osw did - it now runs as a subprocess and must leave stdout empty, which is the assertion that matters for a JSON-RPC transport - the level the shim sets moved into _DEFAULT_LEVEL, and a test ties it to osw.DEFAULT_LOG_LEVEL so the two cannot drift apart
* fix(config): reject a hostless domain and a relative state_dir - domain accepts a bare host or a full URL, so the check is on what _derive_domain yields rather than on the form of the value - a hostless domain used to fail later in OswExpress.validate_domain, quoting a regex instead of naming OSW_DOMAIN - state_dir "~/osw" created a directory literally named "~", because Ledger builds its path with Path() and never expands it - a relative state_dir resolves against a working directory the MCP client chooses, so the ledger landed in an unpredictable place - last open item from #143 * test(config): cover drive-relative state_dir and an unknown home - reject "\osw-state" and "C:osw-state", non-absolute on both platforms - assert the expanduser RuntimeError becomes a named config error - state in the docs that a domain without a host is rejected
- text=True alone decodes with locale.getencoding(), which ignores Python's UTF-8 mode, so a child writing UTF-8 broke the reader thread with UnicodeDecodeError and returned the stream as None - reproduced on Windows: byte 0x90 at position 1028 of osw --help - the osw_entry tests now share a _DECODE mapping - test_no_paths_on_mcp_surface gets the same kwargs as protection
3 participants
What this adds
Two ways to work against a live OpenSemanticLab instance without writing Python:
osw, a command line client, shipped with the base packageosw-mcp, a stdio MCP server (osw[mcp]extra) that exposes one instanceto agent clients such as Claude Code
Both are generated from a single declaration per operation, so a command and its
matching tool cannot drift apart in behaviour, argument names or help text.
Capabilities
search ask,search text,search instances,search sparqlsearch_entities,full_text_search,list_instances_of_category,sparql_queryschema getget_category_schemaentity get,put,export,deleteget_entity,create_or_update_entity,export_entity_jsonld,delete_entityslot list,get,setlist_page_slots,get_slot,set_slotfile info,cat,writeget_file_info,read_file_text,write_file_textfile download,file uploadstatus,instance list,ledger pathstatusBehaviour common to both:
--read-only,OSW_READ_ONLY) hides mutating tools ratherthan failing them when called, so an agent never sees a tool it cannot use.
normally; anything else requires
confirm_external_delete=true.OSW_MAX_RESULTS,OSW_MAX_CHARS) keep a broad query fromflooding an agent's context.
.envfile, or an iri-keyed oswcredential file. No tool ever returns one.
osw --jsonputs JSON on stdout and osw's ownprogress output on stderr. Failures exit non-zero with one line, no traceback.
Constraints that shaped it
Decisions that need agreement
1. No filesystem path reaches the MCP surface.
Rejected: documenting the server as local-use-only. That is a rule with no
enforcement behind it. Instead paths exist only in
osw.cli, and the operationmodel refuses to register a path-like parameter on an MCP-surfaced operation, so
the server fails at import rather than shipping such a tool.
2. One server process per instance, no switching at runtime.
This reverses
d393a66, which was agreed earlier in this PR. In-sessionswitching cannot keep the instance visible in the tool name, cannot make
read-only per instance, and is a prompt-injection target. Argument in full:
#133 (comment)
3. The server refuses to start unless an instance is named explicitly.
Rejected: failing at the first tool call, and inferring the sole iri of a
credential file. A server that cannot name its target would advertise tools that
all fail. The CLI still infers, because it resolves per invocation, reports what
it resolved, and
--instanceoverrides any single command.4.
typeras a base dependency, not an extra.Rejected: argparse plus a hand-written signature-to-parser translator (roughly 50
lines to own and keep honest), and click, which is decorator-per-option and would
mean writing every parameter twice. typer reads the same type hints and
docstrings the MCP SDK reads, which is what makes one declaration serve both. As
an extra it would let
pip install oswship a brokenoswconsole script.5.
mcpis an extra, not a base dependency.The SDK pulls in a server stack (starlette, uvicorn, sse-starlette) that nothing
in the Python API or the CLI needs, so only users who actually run the server pay
for it. It is included in
osw[all]and in thedevgroup, so the server and itstests share one environment with everything else. The earlier isolation, forced by
an anyio conflict with
osw[workflow]and declared through[tool.uv] conflicts,is gone: #139 is closed and
the pin is now
anyio>=4.9,<4.14.6. Canonical
OSW_*configuration names, old names kept as aliases.OSW_CRED_FILEPATHis already read bysrc/osw/express.py, so the previousOSW_MCP_CRED_FILEPATHdiverged from the library that owns the same setting.OSW_MCP_*andOSL_*remain accepted, so existing deployments keep working.Configuration
Required: an instance and credentials, either
OSW_DOMAINplusOSW_USERNAME/OSW_PASSWORD, orOSW_DOMAINplusOSW_CRED_FILEPATH..envhandling differs by adapter on purpose. The CLI searches upward from theworking directory. The server searches nowhere, because its working directory is
chosen by the client, and takes its settings from the
envblock of itsregistration. Both report the env file and credential file they resolved on
stderr before connecting.
Full reference, including registering one server per instance:
https://github.com/OpenSemanticLab/osw-python/blob/feat/mcp-server/docs/cli-and-mcp.md
Verification
One environment covers everything. That includes the 17 MCP tests in
tests/test_mcp_registration.py,tests/test_mcp_server.pyandtests/test_no_paths_on_mcp_surface.py, which CI now runs alongside the rest.They previously self-skipped in CI, because a plain
uv sync --frozenneverinstalled the
mcpextra, so the MCP code was effectively untested there.tests/integration/test_mcp_server.pyruns against a live instance.Follow-ups
Filed while planning this work.
Resolved:
Settingsisnow a frozen pydantic model with validators instead of a dataclass with
hand-rolled env parsing.
oswCLIwrites stdout and stderr as UTF-8 rather than the locale encoding, so a
non-ASCII label no longer reaches the consumer as cp1252 bytes on Windows.
that forced the separate MCP environment. Closed upstream of this branch.
fix: prevent silent page loss in store_entity parallel upload #132.
date. It reported that osw log records reach the JSON-RPC channel because
enable_logging()defaults tosys.stdout. On this branch it defaults tosys.stderr, naming the MCP stdio server as the reason. The test it alsoasked for is tracked in
No test asserts the MCP stdio server keeps stdout free of osw output #192.
fix: allow uploading a file from an in-memory stream #148 and already merged into
this branch. Whichever of the two lands first, the other carries the change as
an ancestor, so fix: allow uploading a file from an in-memory stream #148 collapses to an empty diff if this PR merges first.
Open, deliberately not blocking this PR:
fetch_schemarewriting the installed
src/osw/model/entity.pyas a side effect. This ispre-existing
osw.corebehaviour, but this PR adds two new entry points to it(
create_or_update_entity, andexport_entity_jsonldviaautofetch_schema),so merging widens the exposure. The MCP
get_category_schematool is notaffected, it reads the
jsonschemaslot directly. Documented indocs/cli-and-mcp.md; design discussion running on the issue.ToolAnnotationssilently ignoring a misspelled hint. The guard is already in this branch, typed
Optional[bool]fields plus keyword construction plus a test; only the upstreamreport is outstanding.
the MCP stdio server keeps stdout free of osw output. Both protections are in
place on this branch:
enable_logging()defaults to stderr, and the SDKdiverts file descriptor 1. Only the regression test is missing.
OSW_CRED_FILEPATHdoes not expand a leading
~, whileOSW_STATE_DIRnow does. The two pathsettings disagree, and a tilde credential path is reported as not existing
although the file exists. Introduced by the MCP Settings is a frozen dataclass with hand-rolled env parsing while the rest of the library is pydantic #143 work on this branch. Loud,
not silent: the run stops, so no credential is read from the wrong place.
osw-mcpwrites itsstartup report to stderr with the locale encoding. Same defect as
osw --json writes non-UTF-8 bytes on Windows for any non-ASCII text #187, which this branch
fixes for the CLI only. Low severity: stderr uses
backslashreplace, so thetext is mangled rather than raising, and the JSON-RPC channel is unaffected
because the SDK forces UTF-8 on it.
The
store_entityparallel-upload fix that once rode along on this branch is nowtracked in #132.