diff --git a/conformance/README.md b/conformance/README.md index f3120ea..6668af5 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -14,7 +14,7 @@ runner/ Conformance runner that replays fixtures against any prov * **Fixtures** (`fixtures/v1/`): one JSON file per scenario. Each scenario defines a session with request/response steps, optional CLI flags, and the expected exit code. Responses are compared after JSON parsing so key order does not matter. The directory contains LPP 1.0, LPP 1.1, LPP 1.2, LPP 1.3, LPP 1.4, and LPP 1.5 scenarios. `fixtures/v1/sessions/` holds the standard initialize/shutdown handshake once per protocol version; a scenario whose steps don't themselves exercise the handshake sets `"session": ""` and the runner splices those steps around its own. * **Mock provider** (`mock-provider/`): a small Rust binary implementing the full LPP v1 surface for a demonstration language distinct from OPY and OSTW. It runs over stdio so clients (like the Wright LPP client in wrightkit/wright#142) can test against it directly. -* **Runner** (`runner/`): spawns a fresh provider process per scenario, feeds requests over stdin, validates stdout responses against expectations, and checks the process exit code. +* **Runner** (`runner/`): spawns a fresh provider process per scenario, feeds requests over stdin, validates stdout responses against expectations, and checks the process exit code. Expected responses compare verbatim except at leaves a fixture marks provider-supplied (see the scenario format below). ## Running the suite @@ -42,7 +42,8 @@ LPP has no wire dependency on Rust. Any provider that reads newline-delimited JS 1. Build your provider as a stdio binary. 2. Run the protocol-scope scenarios: `lpp-conformance-runner --provider --scope protocol` -3. The semantic scenarios use `x-demo-lang` (an equation-puzzle DSL). To test a provider for another language, substitute your own language source texts and artifact payloads while keeping the protocol envelope and message sequence. + Scenarios that do not exercise `x-demo-lang` semantics pass out of the box: provider identity (`serverInfo`, `languages`), negotiated capability values, `supportedProtocolVersions`, and provider prose are asserted by contract shape, not by reference-adapter values. +3. The remaining scenarios are adapter-specific by design: `semantics` scenarios exercise `x-demo-lang` content, and `providerArgs` scenarios configure the reference mock's CLI. To cover those, substitute your own language source texts, artifact payloads, and adapter configuration while keeping the protocol envelope and message sequence. Passing this suite verifies wire protocol conformance. It does not check Workshop engine correctness or runtime performance. @@ -74,7 +75,11 @@ Passing this suite verifies wire protocol conformance. It does not check Worksho own steps call `lpp/initialize` or `lpp/shutdown`; session-behavior tests (mismatch, double-init, notifications) keep the handshake explicit. * `providerArgs`: optional extra command-line arguments for the provider - binary (used to exercise capability negotiation). + binary (used to exercise capability negotiation). These name the reference + mock's own CLI flags; a scenario carrying `providerArgs` is adapter-specific + and third-party providers are not expected to run it verbatim. Its + `expectResponse` keeps the negotiated capability map verbatim — the + `false` leaves are the assertion the scenario exists for. * `projectFiles`: optional relative path/content pairs that the runner writes to an isolated temporary project for entry-based project-loading scenarios. `${PROJECT_URI}` in requests and expected responses is replaced with that @@ -86,6 +91,26 @@ Passing this suite verifies wire protocol conformance. It does not check Worksho sides, and an `expectResponse` without `id` expects the request's id (null for `rawLine` steps). Scenarios that exercise a non-`"2.0"` or unusual id keep the fields explicit. +* Inside `expectResponse`, a leaf of the form `{ "$provider": "" }` + marks a value the spec leaves provider-supplied and asserts its contract + shape instead of a literal: + + | Marker | Asserted shape | + | --- | --- | + | `nonEmptyString` | any non-empty string | + | `boolean` | any boolean | + | `languageList` | non-empty array of `{ "id", "extensions" }` language entries, extensions lowercase without a leading dot | + | `protocolVersions` | non-empty array of `MAJOR.MINOR` strings | + + Marker positions are a whitelist — the provider-supplied positions named + in [§20.1](../docs/spec/conformance.md#201-provider-supplied-leaves), and + fixture validation enforces it, including the sibling `data.lpp.kind` and + negotiated `protocolVersion` a position can depend on. A marked + `capabilities` object must mark exactly the capability ids declared for + the negotiated version: a provider may advertise any subset (absent means + not offered), but an undeclared capability id still fails the closed-set + check. Every other leaf keeps verbatim JSON equality; markers are not + valid in `request`. * `expectExitCode`: the provider's exit status after stdin is closed (default 0). diff --git a/conformance/fixtures/v1/01-initialize-success.json b/conformance/fixtures/v1/01-initialize-success.json index f9a14f1..fcdeace 100644 --- a/conformance/fixtures/v1/01-initialize-success.json +++ b/conformance/fixtures/v1/01-initialize-success.json @@ -3,7 +3,7 @@ "description": "A full handshake: initialize with client info, then a clean shutdown.", "scope": "protocol", "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0","clientInfo":{"name":"wright","version":"0.2.0"}}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0","clientInfo":{"name":"wright","version":"0.2.0"}}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":2,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/02-initialize-protocol-mismatch.json b/conformance/fixtures/v1/02-initialize-protocol-mismatch.json index 79a040a..a290e20 100644 --- a/conformance/fixtures/v1/02-initialize-protocol-mismatch.json +++ b/conformance/fixtures/v1/02-initialize-protocol-mismatch.json @@ -3,6 +3,6 @@ "description": "The provider must reject an unsupported protocol version and list the versions it supports. The session ends with EOF on stdin.", "scope": "protocol", "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"0.9"}},"expectResponse":{"error":{"code":-32000,"message":"unsupported protocol version 0.9","data":{"lpp":{"kind":"protocolVersionMismatch","details":{"supportedProtocolVersions":["1.0"]}}}}}} + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"0.9"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"protocolVersionMismatch","details":{"supportedProtocolVersions":{"$provider":"protocolVersions"}}}}}}} ] } diff --git a/conformance/fixtures/v1/03-request-before-initialize.json b/conformance/fixtures/v1/03-request-before-initialize.json index ca03c55..a7ada81 100644 --- a/conformance/fixtures/v1/03-request-before-initialize.json +++ b/conformance/fixtures/v1/03-request-before-initialize.json @@ -3,8 +3,8 @@ "description": "No method except lpp/initialize may be processed before the handshake.", "scope": "protocol", "steps": [ - {"request":{"id":1,"method":"lpp/check","params":{"documents":{}}},"expectResponse":{"error":{"code":-32000,"message":"invalid request: session not initialized","data":{"lpp":{"kind":"invalidRequest","details":{"reason":"notInitialized"}}}}}}, - {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"request":{"id":1,"method":"lpp/check","params":{"documents":{}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidRequest","details":{"reason":"notInitialized"}}}}}}, + {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":3,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/04-initialize-twice.json b/conformance/fixtures/v1/04-initialize-twice.json index 141b63c..8ce53c2 100644 --- a/conformance/fixtures/v1/04-initialize-twice.json +++ b/conformance/fixtures/v1/04-initialize-twice.json @@ -3,8 +3,8 @@ "description": "A second lpp/initialize after a successful one is a session violation.", "scope": "protocol", "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, - {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"error":{"code":-32000,"message":"invalid request: already initialized","data":{"lpp":{"kind":"invalidRequest","details":{"reason":"alreadyInitialized"}}}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, + {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidRequest","details":{"reason":"alreadyInitialized"}}}}}}, {"request":{"id":3,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/05-unknown-method.json b/conformance/fixtures/v1/05-unknown-method.json index 44a7bb5..2a242cc 100644 --- a/conformance/fixtures/v1/05-unknown-method.json +++ b/conformance/fixtures/v1/05-unknown-method.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/frobnicate","params":{}},"expectResponse":{"error":{"code":-32601,"message":"Method not found"}}} + {"request":{"id":2,"method":"lpp/frobnicate","params":{}},"expectResponse":{"error":{"code":-32601,"message":{"$provider":"nonEmptyString"}}}} ] } diff --git a/conformance/fixtures/v1/06-malformed-json.json b/conformance/fixtures/v1/06-malformed-json.json index b7fc66a..99755ac 100644 --- a/conformance/fixtures/v1/06-malformed-json.json +++ b/conformance/fixtures/v1/06-malformed-json.json @@ -3,8 +3,8 @@ "description": "A line that is not valid JSON is answered with the standard JSON-RPC -32700 parse error (id null), and the session continues.", "scope": "protocol", "steps": [ - {"rawLine":"this is not json","expectResponse":{"error":{"code":-32700,"message":"Parse error"}}}, - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"rawLine":"this is not json","expectResponse":{"error":{"code":-32700,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":2,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/07-invalid-jsonrpc-version.json b/conformance/fixtures/v1/07-invalid-jsonrpc-version.json index 9dfd986..122ffa7 100644 --- a/conformance/fixtures/v1/07-invalid-jsonrpc-version.json +++ b/conformance/fixtures/v1/07-invalid-jsonrpc-version.json @@ -3,8 +3,8 @@ "description": "A message with a non-2.0 jsonrpc version is answered with the standard JSON-RPC -32600 invalid request error.", "scope": "protocol", "steps": [ - {"rawLine":"{\"jsonrpc\":\"1.0\",\"id\":1,\"method\":\"lpp/initialize\",\"params\":{\"protocolVersion\":\"1.0\"}}","expectResponse":{"error":{"code":-32600,"message":"Invalid Request"}}}, - {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"rawLine":"{\"jsonrpc\":\"1.0\",\"id\":1,\"method\":\"lpp/initialize\",\"params\":{\"protocolVersion\":\"1.0\"}}","expectResponse":{"error":{"code":-32600,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":2,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":3,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/08-notification-not-supported.json b/conformance/fixtures/v1/08-notification-not-supported.json index 80260fe..ffae975 100644 --- a/conformance/fixtures/v1/08-notification-not-supported.json +++ b/conformance/fixtures/v1/08-notification-not-supported.json @@ -3,8 +3,8 @@ "description": "LPP v1 defines no notifications: a message without an id is a protocol violation answered with an LPP invalidRequest error (id null in the response).", "scope": "protocol", "steps": [ - {"request":{"method":"lpp/shutdown"},"expectResponse":{"error":{"code":-32000,"message":"invalid request: LPP v1 defines no notifications","data":{"lpp":{"kind":"invalidRequest","details":{"reason":"notificationNotSupported"}}}}}}, - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"request":{"method":"lpp/shutdown"},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidRequest","details":{"reason":"notificationNotSupported"}}}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":2,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/09-batch-not-supported.json b/conformance/fixtures/v1/09-batch-not-supported.json index 48f9265..2d30d01 100644 --- a/conformance/fixtures/v1/09-batch-not-supported.json +++ b/conformance/fixtures/v1/09-batch-not-supported.json @@ -3,8 +3,8 @@ "description": "Batch messages are not supported in LPP v1: an array message is answered with the standard JSON-RPC -32600 error (id null).", "scope": "protocol", "steps": [ - {"request":[{"jsonrpc":"2.0","id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}}],"expectResponse":{"error":{"code":-32600,"message":"Invalid Request"}}}, - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"request":[{"jsonrpc":"2.0","id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}}],"expectResponse":{"error":{"code":-32600,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":{"$provider":"boolean"},"compile":{"$provider":"boolean"},"reconstruct":{"$provider":"boolean"},"symbols":{"$provider":"boolean"},"definition":{"$provider":"boolean"},"references":{"$provider":"boolean"},"rename":{"$provider":"boolean"},"editValidation":{"$provider":"boolean"}}}}}, {"request":{"id":2,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/10-capability-negotiation.json b/conformance/fixtures/v1/10-capability-negotiation.json index 5b46f72..883bc71 100644 --- a/conformance/fixtures/v1/10-capability-negotiation.json +++ b/conformance/fixtures/v1/10-capability-negotiation.json @@ -4,8 +4,8 @@ "scope": "protocol", "providerArgs": ["--without", "reconstruct"], "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":false,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, - {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/puzzle-eval-v1","content":"{}"}}},"expectResponse":{"error":{"code":-32000,"message":"capability 'reconstruct' is not available","data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"reconstruct","method":"lpp/reconstruct"}}}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.0"}},"expectResponse":{"result":{"protocolVersion":"1.0","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":true,"compile":true,"reconstruct":false,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true}}}}, + {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/puzzle-eval-v1","content":"{}"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"reconstruct","method":"lpp/reconstruct"}}}}}}, {"request":{"id":3,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}}}},"expectResponse":{"result":{"documents":[{"uri":"file:///project/puzzle.xdl","version":3,"diagnostics":[]}]}}}, {"request":{"id":4,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] diff --git a/conformance/fixtures/v1/14-invalid-language.json b/conformance/fixtures/v1/14-invalid-language.json index a70f525..ed1e115 100644 --- a/conformance/fixtures/v1/14-invalid-language.json +++ b/conformance/fixtures/v1/14-invalid-language.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-other-lang","version":1,"text":"not relevant"}}}},"expectResponse":{"error":{"code":-32000,"message":"language 'x-other-lang' is not served by this provider","data":{"lpp":{"kind":"invalidLanguage","details":{"languageId":"x-other-lang"}}}}}} + {"request":{"id":2,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-other-lang","version":1,"text":"not relevant"}}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidLanguage","details":{"languageId":"x-other-lang"}}}}}} ] } diff --git a/conformance/fixtures/v1/15-invalid-document.json b/conformance/fixtures/v1/15-invalid-document.json index 2203d1d..f444269 100644 --- a/conformance/fixtures/v1/15-invalid-document.json +++ b/conformance/fixtures/v1/15-invalid-document.json @@ -5,6 +5,6 @@ "session": "1.0", "steps": [ {"request":{"id":2,"method":"lpp/check","params":{"documents":{}}},"expectResponse":{"result":{"documents":[]}}}, - {"request":{"id":3,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":-1,"text":"puzzle clean {\n target = 40\n}"}}}},"expectResponse":{"error":{"code":-32000,"message":"document version must be a non-negative integer","data":{"lpp":{"kind":"invalidDocument","details":{"uri":"file:///project/puzzle.xdl","reason":"invalidVersion"}}}}}} + {"request":{"id":3,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":-1,"text":"puzzle clean {\n target = 40\n}"}}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidDocument","details":{"uri":"file:///project/puzzle.xdl","reason":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/16-invalid-position.json b/conformance/fixtures/v1/16-invalid-position.json index 4172d66..aa4a76f 100644 --- a/conformance/fixtures/v1/16-invalid-position.json +++ b/conformance/fixtures/v1/16-invalid-position.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/definition","params":{"document":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"},"position":{"line":0,"character":500}}},"expectResponse":{"error":{"code":-32000,"message":"position outside document","data":{"lpp":{"kind":"invalidPosition","details":{"uri":"file:///project/puzzle.xdl","position":{"line":0,"character":500}}}}}}} + {"request":{"id":2,"method":"lpp/definition","params":{"document":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"},"position":{"line":0,"character":500}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidPosition","details":{"uri":"file:///project/puzzle.xdl","position":{"line":0,"character":500}}}}}}} ] } diff --git a/conformance/fixtures/v1/19-compile-refusal-multi-document.json b/conformance/fixtures/v1/19-compile-refusal-multi-document.json index 88cffb3..97e372e 100644 --- a/conformance/fixtures/v1/19-compile-refusal-multi-document.json +++ b/conformance/fixtures/v1/19-compile-refusal-multi-document.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/a.xdl":{"uri":"file:///project/a.xdl","languageId":"x-demo-lang","version":1,"text":"puzzle a {\n target = 1\n start = 1\n ops {\n one: x => x * 1\n }\n solution = [ one ]\n}"},"file:///project/b.xdl":{"uri":"file:///project/b.xdl","languageId":"x-demo-lang","version":1,"text":"puzzle b {\n target = 1\n start = 1\n ops {\n one: x => x * 1\n }\n solution = [ one ]\n}"}}}},"expectResponse":{"error":{"code":-32000,"message":"compile requires exactly one document","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"compile.requiresSingleDocument"}}}}}} + {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/a.xdl":{"uri":"file:///project/a.xdl","languageId":"x-demo-lang","version":1,"text":"puzzle a {\n target = 1\n start = 1\n ops {\n one: x => x * 1\n }\n solution = [ one ]\n}"},"file:///project/b.xdl":{"uri":"file:///project/b.xdl","languageId":"x-demo-lang","version":1,"text":"puzzle b {\n target = 1\n start = 1\n ops {\n one: x => x * 1\n }\n solution = [ one ]\n}"}}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/21-reconstruct-unknown-format.json b/conformance/fixtures/v1/21-reconstruct-unknown-format.json index 10fe3e3..69b4119 100644 --- a/conformance/fixtures/v1/21-reconstruct-unknown-format.json +++ b/conformance/fixtures/v1/21-reconstruct-unknown-format.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/other-v9","content":"{}"}}},"expectResponse":{"error":{"code":-32000,"message":"artifact format not supported","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"reconstruct.artifactFormatUnsupported","format":"x-demo/other-v9"}}}}}} + {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/other-v9","content":"{}"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"format":"x-demo/other-v9"}}}}}} ] } diff --git a/conformance/fixtures/v1/22-reconstruct-invalid-content.json b/conformance/fixtures/v1/22-reconstruct-invalid-content.json index e1681f4..467bd3b 100644 --- a/conformance/fixtures/v1/22-reconstruct-invalid-content.json +++ b/conformance/fixtures/v1/22-reconstruct-invalid-content.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/puzzle-eval-v1","content":"not json"}}},"expectResponse":{"error":{"code":-32000,"message":"artifact content is not a valid puzzle evaluation sheet","data":{"lpp":{"kind":"invalidArtifact","details":{"reason":"malformedArtifactContent"}}}}}} + {"request":{"id":2,"method":"lpp/reconstruct","params":{"artifact":{"format":"x-demo/puzzle-eval-v1","content":"not json"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidArtifact","details":{"reason":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/24-definition.json b/conformance/fixtures/v1/24-definition.json index 2a77371..a25547a 100644 --- a/conformance/fixtures/v1/24-definition.json +++ b/conformance/fixtures/v1/24-definition.json @@ -5,6 +5,6 @@ "session": "1.0", "steps": [ {"request":{"id":2,"method":"lpp/definition","params":{"document":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"},"position":{"line":7,"character":16}}},"expectResponse":{"result":{"locations":[{"uri":"file:///project/puzzle.xdl","range":{"start":{"line":4,"character":4},"end":{"line":4,"character":10}}}]}}}, - {"request":{"id":3,"method":"lpp/definition","params":{"document":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"},"position":{"line":1,"character":11}}},"expectResponse":{"error":{"code":-32000,"message":"no symbol at position","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"definition.noSymbolAtPosition","uri":"file:///project/puzzle.xdl"}}}}}} + {"request":{"id":3,"method":"lpp/definition","params":{"document":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"},"position":{"line":1,"character":11}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"uri":"file:///project/puzzle.xdl"}}}}}} ] } diff --git a/conformance/fixtures/v1/27-rename-collision.json b/conformance/fixtures/v1/27-rename-collision.json index 08f068e..f45b857 100644 --- a/conformance/fixtures/v1/27-rename-collision.json +++ b/conformance/fixtures/v1/27-rename-collision.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":4,"character":6},"newName":"plus1"}},"expectResponse":{"error":{"code":-32000,"message":"new name collides with an existing symbol","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"rename.nameCollision","newName":"plus1"}}}}}} + {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":4,"character":6},"newName":"plus1"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"newName":"plus1"}}}}}} ] } diff --git a/conformance/fixtures/v1/28-rename-no-symbol.json b/conformance/fixtures/v1/28-rename-no-symbol.json index 87031ff..1b78142 100644 --- a/conformance/fixtures/v1/28-rename-no-symbol.json +++ b/conformance/fixtures/v1/28-rename-no-symbol.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":1,"character":11},"newName":"twice"}},"expectResponse":{"error":{"code":-32000,"message":"no symbol at position","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"rename.noSymbolAtPosition","uri":"file:///project/puzzle.xdl"}}}}}} + {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":1,"character":11},"newName":"twice"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"uri":"file:///project/puzzle.xdl"}}}}}} ] } diff --git a/conformance/fixtures/v1/29-rename-invalid-name.json b/conformance/fixtures/v1/29-rename-invalid-name.json index 7a106c4..d99aa4b 100644 --- a/conformance/fixtures/v1/29-rename-invalid-name.json +++ b/conformance/fixtures/v1/29-rename-invalid-name.json @@ -4,6 +4,6 @@ "scope": "protocol", "session": "1.0", "steps": [ - {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":4,"character":6},"newName":"2bad"}},"expectResponse":{"error":{"code":-32000,"message":"new name is not a valid identifier","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"rename.invalidName","newName":"2bad"}}}}}} + {"request":{"id":2,"method":"lpp/rename","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"positionDocumentUri":"file:///project/puzzle.xdl","position":{"line":4,"character":6},"newName":"2bad"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"newName":"2bad"}}}}}} ] } diff --git a/conformance/fixtures/v1/37-project-loading-failure.json b/conformance/fixtures/v1/37-project-loading-failure.json index 520e980..0e9e37c 100644 --- a/conformance/fixtures/v1/37-project-loading-failure.json +++ b/conformance/fixtures/v1/37-project-loading-failure.json @@ -6,6 +6,6 @@ "providerArgs": ["--protocol-version", "1.1"], "projectFiles": {"present.xdl":""}, "steps": [ - {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/missing.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":"project entry could not be loaded","data":{"lpp":{"kind":"projectLoadFailed","details":{"entryUri":"${PROJECT_URI}/missing.xdl","reason":"entryNotFound","uri":"${PROJECT_URI}/missing.xdl"}}}}}} + {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/missing.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"projectLoadFailed","details":{"entryUri":"${PROJECT_URI}/missing.xdl","reason":{"$provider":"nonEmptyString"},"uri":"${PROJECT_URI}/missing.xdl"}}}}}} ] } diff --git a/conformance/fixtures/v1/38-project-loading-capability-unavailable.json b/conformance/fixtures/v1/38-project-loading-capability-unavailable.json index 10fa0d2..f14c0e4 100644 --- a/conformance/fixtures/v1/38-project-loading-capability-unavailable.json +++ b/conformance/fixtures/v1/38-project-loading-capability-unavailable.json @@ -5,9 +5,9 @@ "providerArgs": ["--protocol-version", "1.1", "--without", "projectLoading"], "projectFiles": {"entry.xdl":""}, "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.1"}},"expectResponse":{"result":{"protocolVersion":"1.1","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":false}}}}, - {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":"capability 'projectLoading' is not available","data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"projectLoading","method":"lpp/check"}}}}}}, - {"request":{"id":3,"method":"lpp/compile","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":"capability 'projectLoading' is not available","data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"projectLoading","method":"lpp/compile"}}}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.1"}},"expectResponse":{"result":{"protocolVersion":"1.1","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":false}}}}, + {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"projectLoading","method":"lpp/check"}}}}}}, + {"request":{"id":3,"method":"lpp/compile","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"projectLoading","method":"lpp/compile"}}}}}}, {"request":{"id":4,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] } diff --git a/conformance/fixtures/v1/39-project-loading-invalid-params.json b/conformance/fixtures/v1/39-project-loading-invalid-params.json index 7d3a597..250451a 100644 --- a/conformance/fixtures/v1/39-project-loading-invalid-params.json +++ b/conformance/fixtures/v1/39-project-loading-invalid-params.json @@ -6,9 +6,9 @@ "providerArgs": ["--protocol-version", "1.1"], "projectFiles": {"entry.xdl":""}, "steps": [ - {"request":{"id":2,"method":"lpp/check","params":{"documents":{},"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":3,"method":"lpp/check","params":{}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":4,"method":"lpp/check","params":{"entry":{"uri":"untitled:entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":"project entry must be an absolute file URI","data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"untitled:entry.xdl","reason":"unsupportedUri"}}}}}}, - {"request":{"id":5,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":-1}}},"expectResponse":{"error":{"code":-32000,"message":"project entry version must be a non-negative integer","data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"${PROJECT_URI}/entry.xdl","reason":"invalidVersion"}}}}}} + {"request":{"id":2,"method":"lpp/check","params":{"documents":{},"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":3,"method":"lpp/check","params":{}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":4,"method":"lpp/check","params":{"entry":{"uri":"untitled:entry.xdl","languageId":"x-demo-lang","version":7}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"untitled:entry.xdl","reason":{"$provider":"nonEmptyString"}}}}}}}, + {"request":{"id":5,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}/entry.xdl","languageId":"x-demo-lang","version":-1}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"${PROJECT_URI}/entry.xdl","reason":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/42-project-loading-directory-version-gate.json b/conformance/fixtures/v1/42-project-loading-directory-version-gate.json index 33c4cb5..cf5155e 100644 --- a/conformance/fixtures/v1/42-project-loading-directory-version-gate.json +++ b/conformance/fixtures/v1/42-project-loading-directory-version-gate.json @@ -8,6 +8,6 @@ "entry.xdl": "puzzle entry {\n target = 2\n start = 1\n ops {\n plus1: x => x + 1\n }\n solution = [ plus1 ]\n}\n" }, "steps": [ - {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}","languageId":"x-demo-lang","version":7,"kind":"directory"}}},"expectResponse":{"error":{"code":-32000,"message":"directory project targets require protocol version 1.2","data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"${PROJECT_URI}","reason":"unsupportedKind"}}}}}} + {"request":{"id":2,"method":"lpp/check","params":{"entry":{"uri":"${PROJECT_URI}","languageId":"x-demo-lang","version":7,"kind":"directory"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidEntry","details":{"entryUri":"${PROJECT_URI}","reason":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/44-project-loading-source-identity-unavailable.json b/conformance/fixtures/v1/44-project-loading-source-identity-unavailable.json index 5b2e046..d7bff7a 100644 --- a/conformance/fixtures/v1/44-project-loading-source-identity-unavailable.json +++ b/conformance/fixtures/v1/44-project-loading-source-identity-unavailable.json @@ -8,7 +8,7 @@ "support.xdl": "puzzle support {\n target = 2\n start = 1\n ops {\n plus1: x => x + 1\n }\n solution = [ plus1 ]\n}\n" }, "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.3"}},"expectResponse":{"result":{"protocolVersion":"1.3","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":true,"sourceIdentity":false}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.3"}},"expectResponse":{"result":{"protocolVersion":"1.3","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":true,"sourceIdentity":false}}}}, {"request":{"id":2,"method":"lpp/compile","params":{"entry":{"uri":"${PROJECT_URI}","languageId":"x-demo-lang","version":7,"kind":"directory"}}},"expectResponse":{"result":{"diagnostics":[{"uri":"${PROJECT_URI}/entry.xdl","version":7,"diagnostics":[]},{"uri":"${PROJECT_URI}/support.xdl","version":7,"diagnostics":[]}],"artifact":{"format":"x-demo/puzzle-eval-v1","content":"{\"name\":\"entry\",\"ops\":[{\"arg\":1,\"name\":\"plus1\",\"op\":\"+\"}],\"solution\":[\"plus1\"],\"start\":1,\"target\":2,\"value\":2}"}}}}, {"request":{"id":3,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] diff --git a/conformance/fixtures/v1/47-compile-accepted-format-unsupported.json b/conformance/fixtures/v1/47-compile-accepted-format-unsupported.json index 785d7a1..9c404be 100644 --- a/conformance/fixtures/v1/47-compile-accepted-format-unsupported.json +++ b/conformance/fixtures/v1/47-compile-accepted-format-unsupported.json @@ -8,6 +8,6 @@ "1.4" ], "steps": [ - {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":["x-demo/unknown-v9"]}},"expectResponse":{"error":{"code":-32000,"message":"none of the accepted artifact formats is supported","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"compile.artifactFormatUnsupported"}}}}}} + {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":["x-demo/unknown-v9"]}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"}}}}}}} ] } diff --git a/conformance/fixtures/v1/48-compile-accepted-format-version-gate.json b/conformance/fixtures/v1/48-compile-accepted-format-version-gate.json index a95ca9d..a74db36 100644 --- a/conformance/fixtures/v1/48-compile-accepted-format-version-gate.json +++ b/conformance/fixtures/v1/48-compile-accepted-format-version-gate.json @@ -8,6 +8,6 @@ "1.3" ], "steps": [ - {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":["x-demo/puzzle-eval-v1"]}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}} + {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":["x-demo/puzzle-eval-v1"]}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}} ] } diff --git a/conformance/fixtures/v1/49-compile-accepted-format-invalid.json b/conformance/fixtures/v1/49-compile-accepted-format-invalid.json index 181c3ef..f27cabe 100644 --- a/conformance/fixtures/v1/49-compile-accepted-format-invalid.json +++ b/conformance/fixtures/v1/49-compile-accepted-format-invalid.json @@ -8,6 +8,6 @@ "1.4" ], "steps": [ - {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":[]}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}} + {"request":{"id":2,"method":"lpp/compile","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}},"acceptedArtifactFormats":[]}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}} ] } diff --git a/conformance/fixtures/v1/54-lookup-within-callable.json b/conformance/fixtures/v1/54-lookup-within-callable.json index fb1c45a..91427e1 100644 --- a/conformance/fixtures/v1/54-lookup-within-callable.json +++ b/conformance/fixtures/v1/54-lookup-within-callable.json @@ -6,7 +6,7 @@ "providerArgs": ["--protocol-version", "1.5"], "steps": [ {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"callable","value":"x-demo:operator/multiply"}}},"expectResponse":{"result":{"entries":[{"identity":"x-demo:operator/multiply/arg","kind":"parameter","spelling":"arg","displayName":"arg","parameter":{"type":"integer","required":true}}]}}}, - {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"callable","value":"x-demo:keyword/ops"}}},"expectResponse":{"error":{"code":-32000,"message":"within selector names no known scope","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"lookup.unknownWithin","within":{"kind":"callable","value":"x-demo:keyword/ops"}}}}}}}, - {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"callable","value":"x-demo:operator/floor"}}},"expectResponse":{"error":{"code":-32000,"message":"within selector names no known scope","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"lookup.unknownWithin","within":{"kind":"callable","value":"x-demo:operator/floor"}}}}}}} + {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"callable","value":"x-demo:keyword/ops"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"within":{"kind":"callable","value":"x-demo:keyword/ops"}}}}}}}, + {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"callable","value":"x-demo:operator/floor"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"within":{"kind":"callable","value":"x-demo:operator/floor"}}}}}}} ] } diff --git a/conformance/fixtures/v1/55-lookup-within-enum.json b/conformance/fixtures/v1/55-lookup-within-enum.json index cace945..2f5e45b 100644 --- a/conformance/fixtures/v1/55-lookup-within-enum.json +++ b/conformance/fixtures/v1/55-lookup-within-enum.json @@ -7,6 +7,6 @@ "steps": [ {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"enum","value":"x-demo:enum/operator"}}},"expectResponse":{"result":{"entries":[{"identity":"x-demo:enum/operator/+","kind":"enumMember","spelling":"+","displayName":"add"},{"identity":"x-demo:enum/operator/-","kind":"enumMember","spelling":"-","displayName":"subtract"},{"identity":"x-demo:enum/operator/*","kind":"enumMember","spelling":"*","displayName":"multiply"},{"identity":"x-demo:enum/operator//","kind":"enumMember","spelling":"/","displayName":"divide"}]}}}, {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"mu","within":{"kind":"enum","value":"x-demo:enum/operator"}}},"expectResponse":{"result":{"entries":[{"identity":"x-demo:enum/operator/*","kind":"enumMember","spelling":"*","displayName":"multiply"}]}}}, - {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"enum","value":"x-demo:setting/target"}}},"expectResponse":{"error":{"code":-32000,"message":"within selector names no known scope","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"lookup.unknownWithin","within":{"kind":"enum","value":"x-demo:setting/target"}}}}}}} + {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"enum","value":"x-demo:setting/target"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"within":{"kind":"enum","value":"x-demo:setting/target"}}}}}}} ] } diff --git a/conformance/fixtures/v1/56-lookup-within-settings.json b/conformance/fixtures/v1/56-lookup-within-settings.json index 43cd32a..59592ae 100644 --- a/conformance/fixtures/v1/56-lookup-within-settings.json +++ b/conformance/fixtures/v1/56-lookup-within-settings.json @@ -7,6 +7,6 @@ "steps": [ {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"settings","value":""}}},"expectResponse":{"result":{"entries":[{"identity":"x-demo:setting/target","kind":"setting","spelling":"target","displayName":"target","setting":{"type":"integer"}},{"identity":"x-demo:setting/start","kind":"setting","spelling":"start","displayName":"start","setting":{"type":"integer"}}]}}}, {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"targ","within":{"kind":"settings","value":""}}},"expectResponse":{"result":{"entries":[{"identity":"x-demo:setting/target","kind":"setting","spelling":"target","displayName":"target","setting":{"type":"integer"}}]}}}, - {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"settings","value":"target"}}},"expectResponse":{"error":{"code":-32000,"message":"within selector names no known scope","data":{"lpp":{"kind":"refusal","details":{"refusalCode":"lookup.unknownWithin","within":{"kind":"settings","value":"target"}}}}}}} + {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"settings","value":"target"}}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"refusal","details":{"refusalCode":{"$provider":"nonEmptyString"},"within":{"kind":"settings","value":"target"}}}}}}} ] } diff --git a/conformance/fixtures/v1/57-lookup-capability-unavailable.json b/conformance/fixtures/v1/57-lookup-capability-unavailable.json index 1479af5..0a44f58 100644 --- a/conformance/fixtures/v1/57-lookup-capability-unavailable.json +++ b/conformance/fixtures/v1/57-lookup-capability-unavailable.json @@ -4,8 +4,8 @@ "scope": "protocol", "providerArgs": ["--protocol-version", "1.5", "--without", "lookup"], "steps": [ - {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.5"}},"expectResponse":{"result":{"protocolVersion":"1.5","serverInfo":{"name":"lpp-mock-provider","version":"0.1.0"},"languages":[{"id":"x-demo-lang","extensions":["xdl"]}],"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":true,"sourceIdentity":true,"lookup":false}}}}, - {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":"capability 'lookup' is not available","data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"lookup","method":"lpp/lookup"}}}}}}, + {"request":{"id":1,"method":"lpp/initialize","params":{"protocolVersion":"1.5"}},"expectResponse":{"result":{"protocolVersion":"1.5","serverInfo":{"name":{"$provider":"nonEmptyString"},"version":{"$provider":"nonEmptyString"}},"languages":{"$provider":"languageList"},"capabilities":{"check":true,"compile":true,"reconstruct":true,"symbols":true,"definition":true,"references":true,"rename":true,"editValidation":true,"projectLoading":true,"sourceIdentity":true,"lookup":false}}}}, + {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"lookup","method":"lpp/lookup"}}}}}}, {"request":{"id":3,"method":"lpp/check","params":{"documents":{"file:///project/puzzle.xdl":{"uri":"file:///project/puzzle.xdl","languageId":"x-demo-lang","version":3,"text":"puzzle clean {\n target = 40\n start = 10\n ops {\n double: x => x * 2\n plus1: x => x + 1\n }\n solution = [ double, double ]\n}"}}}},"expectResponse":{"result":{"documents":[{"uri":"file:///project/puzzle.xdl","version":3,"diagnostics":[]}]}}}, {"request":{"id":4,"method":"lpp/shutdown","params":{}},"expectResponse":{"result":null}} ] diff --git a/conformance/fixtures/v1/58-lookup-version-gate.json b/conformance/fixtures/v1/58-lookup-version-gate.json index 508ce71..3c1febf 100644 --- a/conformance/fixtures/v1/58-lookup-version-gate.json +++ b/conformance/fixtures/v1/58-lookup-version-gate.json @@ -5,6 +5,6 @@ "session": "1.4", "providerArgs": ["--protocol-version", "1.4"], "steps": [ - {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":"capability 'lookup' is not available","data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"lookup","method":"lpp/lookup"}}}}}} + {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"capabilityUnavailable","details":{"capability":"lookup","method":"lpp/lookup"}}}}}} ] } diff --git a/conformance/fixtures/v1/59-lookup-invalid-language.json b/conformance/fixtures/v1/59-lookup-invalid-language.json index 553b2b4..adacfc6 100644 --- a/conformance/fixtures/v1/59-lookup-invalid-language.json +++ b/conformance/fixtures/v1/59-lookup-invalid-language.json @@ -5,6 +5,6 @@ "session": "1.5", "providerArgs": ["--protocol-version", "1.5"], "steps": [ - {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-other-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":"language 'x-other-lang' is not served by this provider","data":{"lpp":{"kind":"invalidLanguage","details":{"languageId":"x-other-lang"}}}}}} + {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-other-lang","query":"multiply"}},"expectResponse":{"error":{"code":-32000,"message":{"$provider":"nonEmptyString"},"data":{"lpp":{"kind":"invalidLanguage","details":{"languageId":"x-other-lang"}}}}}} ] } diff --git a/conformance/fixtures/v1/60-lookup-invalid-params.json b/conformance/fixtures/v1/60-lookup-invalid-params.json index 04bcc50..93e6f0b 100644 --- a/conformance/fixtures/v1/60-lookup-invalid-params.json +++ b/conformance/fixtures/v1/60-lookup-invalid-params.json @@ -5,10 +5,10 @@ "session": "1.5", "providerArgs": ["--protocol-version", "1.5"], "steps": [ - {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"a","limit":0}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"bogus","value":"x-demo:enum/operator"}}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"enum"}}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":5,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","limit":"3"}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}}, - {"request":{"id":6,"method":"lpp/lookup","params":{"query":"a"}},"expectResponse":{"error":{"code":-32602,"message":"Invalid params"}}} + {"request":{"id":2,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","query":"a","limit":0}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":3,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"bogus","value":"x-demo:enum/operator"}}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":4,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","within":{"kind":"enum"}}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":5,"method":"lpp/lookup","params":{"languageId":"x-demo-lang","limit":"3"}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}}, + {"request":{"id":6,"method":"lpp/lookup","params":{"query":"a"}},"expectResponse":{"error":{"code":-32602,"message":{"$provider":"nonEmptyString"}}}} ] } diff --git a/conformance/fixtures/v1/sessions/1.0.json b/conformance/fixtures/v1/sessions/1.0.json index 184c21e..67842d7 100644 --- a/conformance/fixtures/v1/sessions/1.0.json +++ b/conformance/fixtures/v1/sessions/1.0.json @@ -4,17 +4,17 @@ "expectResponse": { "result": { "protocolVersion": "1.0", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" } } } } diff --git a/conformance/fixtures/v1/sessions/1.1.json b/conformance/fixtures/v1/sessions/1.1.json index 2c408bb..db04097 100644 --- a/conformance/fixtures/v1/sessions/1.1.json +++ b/conformance/fixtures/v1/sessions/1.1.json @@ -4,18 +4,18 @@ "expectResponse": { "result": { "protocolVersion": "1.1", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true, - "projectLoading": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" }, + "projectLoading": { "$provider": "boolean" } } } } diff --git a/conformance/fixtures/v1/sessions/1.2.json b/conformance/fixtures/v1/sessions/1.2.json index c3e8f74..7c4ea8e 100644 --- a/conformance/fixtures/v1/sessions/1.2.json +++ b/conformance/fixtures/v1/sessions/1.2.json @@ -4,18 +4,18 @@ "expectResponse": { "result": { "protocolVersion": "1.2", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true, - "projectLoading": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" }, + "projectLoading": { "$provider": "boolean" } } } } diff --git a/conformance/fixtures/v1/sessions/1.3.json b/conformance/fixtures/v1/sessions/1.3.json index e5c79a2..dad5176 100644 --- a/conformance/fixtures/v1/sessions/1.3.json +++ b/conformance/fixtures/v1/sessions/1.3.json @@ -4,19 +4,19 @@ "expectResponse": { "result": { "protocolVersion": "1.3", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true, - "projectLoading": true, - "sourceIdentity": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" }, + "projectLoading": { "$provider": "boolean" }, + "sourceIdentity": { "$provider": "boolean" } } } } diff --git a/conformance/fixtures/v1/sessions/1.4.json b/conformance/fixtures/v1/sessions/1.4.json index a7c498e..7ded51d 100644 --- a/conformance/fixtures/v1/sessions/1.4.json +++ b/conformance/fixtures/v1/sessions/1.4.json @@ -4,19 +4,19 @@ "expectResponse": { "result": { "protocolVersion": "1.4", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true, - "projectLoading": true, - "sourceIdentity": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" }, + "projectLoading": { "$provider": "boolean" }, + "sourceIdentity": { "$provider": "boolean" } } } } diff --git a/conformance/fixtures/v1/sessions/1.5.json b/conformance/fixtures/v1/sessions/1.5.json index 06aa289..200db8c 100644 --- a/conformance/fixtures/v1/sessions/1.5.json +++ b/conformance/fixtures/v1/sessions/1.5.json @@ -4,20 +4,20 @@ "expectResponse": { "result": { "protocolVersion": "1.5", - "serverInfo": { "name": "lpp-mock-provider", "version": "0.1.0" }, - "languages": [{ "id": "x-demo-lang", "extensions": ["xdl"] }], + "serverInfo": { "name": { "$provider": "nonEmptyString" }, "version": { "$provider": "nonEmptyString" } }, + "languages": { "$provider": "languageList" }, "capabilities": { - "check": true, - "compile": true, - "reconstruct": true, - "symbols": true, - "definition": true, - "references": true, - "rename": true, - "editValidation": true, - "projectLoading": true, - "sourceIdentity": true, - "lookup": true + "check": { "$provider": "boolean" }, + "compile": { "$provider": "boolean" }, + "reconstruct": { "$provider": "boolean" }, + "symbols": { "$provider": "boolean" }, + "definition": { "$provider": "boolean" }, + "references": { "$provider": "boolean" }, + "rename": { "$provider": "boolean" }, + "editValidation": { "$provider": "boolean" }, + "projectLoading": { "$provider": "boolean" }, + "sourceIdentity": { "$provider": "boolean" }, + "lookup": { "$provider": "boolean" } } } } diff --git a/conformance/runner/src/main.rs b/conformance/runner/src/main.rs index 0acec09..92d7d9d 100644 --- a/conformance/runner/src/main.rs +++ b/conformance/runner/src/main.rs @@ -1,9 +1,11 @@ //! `lpp-conformance-runner`: replays LPP v1 fixtures against a provider //! binary. `scenario` owns the fixture schema and structural validation, //! `project_fixture` owns the materialized `projectFiles` tree and -//! `${PROJECT_URI}` substitution, and `session` owns the spawned provider -//! exchange. Fixtures must stay deterministic and self-contained. +//! `${PROJECT_URI}` substitution, `matcher` owns the expected-response +//! comparison including provider-owned leaves, and `session` owns the spawned +//! provider exchange. Fixtures must stay deterministic and self-contained. +mod matcher; mod project_fixture; mod scenario; mod session; diff --git a/conformance/runner/src/matcher.rs b/conformance/runner/src/matcher.rs new file mode 100644 index 0000000..52f8dce --- /dev/null +++ b/conformance/runner/src/matcher.rs @@ -0,0 +1,574 @@ +//! Expected-vs-actual response comparison. +//! +//! The fixture DSL marks leaves the spec declares provider-supplied as +//! `{ "$provider": "" }`. A marked leaf is asserted by its contract +//! shape instead of a literal value, so providers other than the reference +//! mock can satisfy protocol-scope scenarios. Unmarked leaves keep verbatim +//! JSON equality. +//! +//! The kinds mirror spec-declared provider-supplied positions: +//! +//! * `boolean` — a `capabilities` value; the key set in the fixture stays the +//! normative required-field assertion. +//! * `languageList` — the `languages` array: non-empty, each entry exactly +//! `{ "id": , "extensions": [] }`. +//! * `nonEmptyString` — provider prose or identity: `serverInfo` fields, +//! error `message`, `refusalCode`, provider-defined `reason`. +//! * `protocolVersions` — `supportedProtocolVersions`: a non-empty array of +//! `MAJOR.MINOR` strings. + +use serde_json::Value; + +/// The provider-leaf kinds the fixture DSL accepts. +const KINDS: &[&str] = &[ + "boolean", + "languageList", + "nonEmptyString", + "protocolVersions", +]; + +/// The `capabilities` object inside an `lpp/initialize` result. +const CAPABILITIES_PATH: &str = "$.result.capabilities"; + +/// Capability ids in introduction order. Per §7.3 the set is closed for each +/// LPP minor version: ids are added only through a new protocol version. +const CAPABILITY_IDS: &[&str] = &[ + "check", + "compile", + "reconstruct", + "symbols", + "definition", + "references", + "rename", + "editValidation", + "projectLoading", + "sourceIdentity", + "lookup", +]; + +/// The capability ids a negotiated protocol version declares. +fn declared_capability_ids(root: &Value) -> Option<&'static [&'static str]> { + let count = match root.pointer("/result/protocolVersion")?.as_str()? { + "1.0" => 8, + "1.1" | "1.2" => 9, + "1.3" | "1.4" => 10, + "1.5" => 11, + _ => return None, + }; + Some(&CAPABILITY_IDS[..count]) +} + +/// The `data.lpp.kind` of an expected error response. +fn expected_error_kind(root: &Value) -> Option<&str> { + root.pointer("/error/data/lpp/kind")?.as_str() +} + +/// Whether `{ "$provider": kind }` may appear at `path` inside an expected +/// response, per the §20.1 whitelist of spec-declared provider-supplied +/// positions. `root` is the whole `expectResponse`, so positions whose +/// legality depends on sibling fields (`data.lpp.kind`, the negotiated +/// `protocolVersion`) can be checked. +fn marker_position_legal(path: &str, kind: &str, root: &Value) -> bool { + match path { + "$.result.serverInfo.name" | "$.result.serverInfo.version" => kind == "nonEmptyString", + "$.result.languages" => kind == "languageList", + "$.error.message" => kind == "nonEmptyString", + "$.error.data.lpp.details.supportedProtocolVersions" => { + kind == "protocolVersions" + && expected_error_kind(root) == Some("protocolVersionMismatch") + } + "$.error.data.lpp.details.refusalCode" => { + kind == "nonEmptyString" && expected_error_kind(root) == Some("refusal") + } + // `reason` is provider-defined only for the kinds whose details the + // spec does not enumerate; `invalidRequest.reason` is a closed set. + "$.error.data.lpp.details.reason" => { + kind == "nonEmptyString" + && matches!( + expected_error_kind(root), + Some( + "invalidDocument" + | "invalidEntry" + | "projectLoadFailed" + | "invalidArtifact" + ) + ) + } + _ => match path + .strip_prefix(CAPABILITIES_PATH) + .and_then(|id| id.strip_prefix('.')) + { + Some(id) => { + kind == "boolean" + && declared_capability_ids(root).is_some_and(|ids| ids.contains(&id)) + } + None => false, + }, + } +} + +/// `Some(kind)` when `expected` is a well-formed provider leaf marker. +fn marker_kind(expected: &Value) -> Option<&str> { + expected + .as_object() + .filter(|object| object.len() == 1) + .and_then(|object| object.get("$provider")) + .and_then(Value::as_str) + .filter(|kind| KINDS.contains(kind)) +} + +/// Validates every `$provider` marker inside a fixture value. Markers are +/// legal only in expected responses, so requests are checked with +/// `allow_markers: false`. A marker's legality also depends on where it sits: +/// `value` doubles as the response root so marker paths can be checked +/// against the §20.1 whitelist. +pub(crate) fn validate_leaves(value: &Value, allow_markers: bool) -> Result<(), String> { + validate_at(value, value, "$", allow_markers) +} + +fn validate_at(root: &Value, value: &Value, path: &str, allow_markers: bool) -> Result<(), String> { + let is_marker = value + .as_object() + .is_some_and(|object| object.contains_key("$provider")); + if is_marker { + if !allow_markers { + return Err("'$provider' markers belong to 'expectResponse'".into()); + } + let Some(kind) = marker_kind(value) else { + return Err(format!( + "'$provider' marker must be the object's only key and one of: {}", + KINDS.join(", ") + )); + }; + return marker_position_legal(path, kind, root) + .then_some(()) + .ok_or_else(|| { + format!("'{kind}' marker is not a provider-supplied position at '{path}'") + }); + } + if path == CAPABILITIES_PATH { + validate_capability_object(root, value)?; + } + match value { + Value::Object(object) => object.iter().try_for_each(|(key, value)| { + validate_at(root, value, &format!("{path}.{key}"), allow_markers) + }), + Value::Array(values) => values.iter().enumerate().try_for_each(|(i, value)| { + validate_at(root, value, &format!("{path}[{i}]"), allow_markers) + }), + _ => Ok(()), + } +} + +/// A `capabilities` object that carries markers must mark the full declared +/// capability set for the negotiated version: every value a `boolean` marker, +/// every key a declared id, and no declared id missing. Verbatim capability +/// maps (adapter-specific `providerArgs` scenarios) are unaffected. +fn validate_capability_object(root: &Value, value: &Value) -> Result<(), String> { + let Some(object) = value.as_object() else { + return Ok(()); + }; + let marked = object.values().any(|value| { + value + .as_object() + .is_some_and(|o| o.contains_key("$provider")) + }); + if !marked { + return Ok(()); + } + if !object + .values() + .all(|value| marker_kind(value) == Some("boolean")) + { + return Err(format!( + "'{CAPABILITIES_PATH}': provider markers cannot mix with verbatim capability values" + )); + } + let declared = declared_capability_ids(root).unwrap_or(&[]); + if object.len() != declared.len() || !object.keys().all(|key| declared.contains(&key.as_str())) + { + return Err(format!( + "'{CAPABILITIES_PATH}': marked capabilities must cover exactly the declared ids for the negotiated version" + )); + } + Ok(()) +} + +/// Recursive expected-vs-actual comparison honoring provider-leaf markers. +/// On mismatch returns the failing path and what differed. +pub(crate) fn matches(expected: &Value, actual: &Value) -> Result<(), String> { + match_at(expected, actual, "$") +} + +fn match_at(expected: &Value, actual: &Value, path: &str) -> Result<(), String> { + if let Some(kind) = marker_kind(expected) { + return check_kind(kind, actual).map_err(|error| format!("{path}: {error}")); + } + match (expected, actual) { + (Value::Object(expected), Value::Object(actual)) => { + // §7.3: a provider may advertise any capability subset — an absent + // id means not offered, equivalent to `false`. The exemption only + // applies to a fully marker-checked capabilities object; a + // verbatim map (a `providerArgs` scenario asserting an explicit + // negotiated map, false keys included) keeps exact key matching. + let subset_mode = path == CAPABILITIES_PATH + && expected.values().all(|value| marker_kind(value).is_some()); + for (key, expected_value) in expected { + let Some(actual_value) = actual.get(key) else { + if subset_mode { + continue; + } + return Err(format!("{path}: missing key '{key}'")); + }; + match_at(expected_value, actual_value, &format!("{path}.{key}"))?; + } + for key in actual.keys() { + if !expected.contains_key(key) { + return Err(format!("{path}: unexpected key '{key}'")); + } + } + Ok(()) + } + (Value::Array(expected), Value::Array(actual)) => { + if expected.len() != actual.len() { + return Err(format!( + "{path}: expected {} elements, got {}", + expected.len(), + actual.len() + )); + } + for (i, (expected_value, actual_value)) in + expected.iter().zip(actual.iter()).enumerate() + { + match_at(expected_value, actual_value, &format!("{path}[{i}]"))?; + } + Ok(()) + } + _ if expected == actual => Ok(()), + _ => Err(format!("{path}: expected {expected}, got {actual}")), + } +} + +fn check_kind(kind: &str, actual: &Value) -> Result<(), String> { + let holds = match kind { + "boolean" => actual.is_boolean(), + "nonEmptyString" => actual.as_str().is_some_and(|text| !text.is_empty()), + "languageList" => is_language_list(actual), + "protocolVersions" => is_version_list(actual), + _ => unreachable!("validated kinds"), + }; + if holds { + Ok(()) + } else { + Err(format!("expected provider-supplied {kind}, got {actual}")) + } +} + +/// `languages`: a non-empty array of `{ id, extensions }` entries, where `id` +/// is a non-empty string and `extensions` a (possibly empty) array of file +/// extensions written without a leading dot and in lowercase (§7.2). +fn is_language_list(actual: &Value) -> bool { + let Some(entries) = actual.as_array() else { + return false; + }; + !entries.is_empty() + && entries.iter().all(|entry| { + entry.as_object().is_some_and(|entry| { + entry.len() == 2 + && entry + .get("id") + .and_then(Value::as_str) + .is_some_and(|id| !id.is_empty()) + && entry + .get("extensions") + .and_then(Value::as_array) + .is_some_and(|ext| { + ext.iter().all(|ext| ext.as_str().is_some_and(is_extension)) + }) + }) + }) +} + +fn is_extension(ext: &str) -> bool { + !ext.is_empty() && !ext.starts_with('.') && !ext.chars().any(char::is_uppercase) +} + +/// `supportedProtocolVersions`: a non-empty array of `MAJOR.MINOR` strings. +fn is_version_list(actual: &Value) -> bool { + let Some(versions) = actual.as_array() else { + return false; + }; + !versions.is_empty() + && versions.iter().all(|version| { + version.as_str().is_some_and(|version| { + let mut parts = version.split('.'); + matches!( + (parts.next(), parts.next(), parts.next()), + (Some(major), Some(minor), None) + if [major, minor].iter().all(|part| { + !part.is_empty() && part.chars().all(|c| c.is_ascii_digit()) + }) + ) + }) + }) +} + +#[cfg(test)] +mod tests { + use serde_json::json; + + use super::*; + + #[test] + fn verbatim_leaves_compare_exactly() { + assert!(matches(&json!({"a": [1, 2]}), &json!({"a": [1, 2]})).is_ok()); + let error = matches(&json!({"a": 1}), &json!({"a": 2})).unwrap_err(); + assert!(error.contains("$.a"), "unexpected error: {error}"); + } + + #[test] + fn provider_leaves_accept_their_shape() { + for (kind, actual) in [ + ("boolean", json!(false)), + ("nonEmptyString", json!("any text")), + ( + "languageList", + json!([{ "id": "opy", "extensions": ["opy"] }]), + ), + ("protocolVersions", json!(["1.0", "1.5"])), + ] { + let expected = json!({ "$provider": kind }); + assert!( + matches(&expected, &actual).is_ok(), + "{kind} should accept {actual}" + ); + } + } + + #[test] + fn provider_leaves_reject_wrong_shape() { + for (kind, actual) in [ + ("boolean", json!("true")), + ("nonEmptyString", json!("")), + ("nonEmptyString", json!(7)), + ("languageList", json!([])), + ("languageList", json!([{ "id": "opy" }])), + ("languageList", json!([{ "id": "", "extensions": [] }])), + // §7.2: extensions are lowercase, without a leading dot. + ( + "languageList", + json!([{ "id": "opy", "extensions": [".OPY"] }]), + ), + ( + "languageList", + json!([{ "id": "opy", "extensions": ["OPY"] }]), + ), + ("languageList", json!([{ "id": "opy", "extensions": [""] }])), + ("protocolVersions", json!([])), + ("protocolVersions", json!(["1x"])), + ("protocolVersions", json!(["1.0", 1])), + ] { + let expected = json!({ "$provider": kind }); + assert!( + matches(&expected, &actual).is_err(), + "{kind} should reject {actual}" + ); + } + } + + #[test] + fn marker_inside_result_reports_path() { + let expected = json!({"result": {"serverInfo": {"name": {"$provider": "nonEmptyString"}}}}); + let error = + matches(&expected, &json!({"result": {"serverInfo": {"name": 42}}})).unwrap_err(); + assert_eq!( + error, + "$.result.serverInfo.name: expected provider-supplied nonEmptyString, got 42" + ); + } + + #[test] + fn objects_still_require_exact_keys() { + let expected = json!({"capabilities": {"check": {"$provider": "boolean"}}}); + assert!( + matches( + &expected, + &json!({"capabilities": {"check": true, "lookup": true}}) + ) + .is_err() + ); + } + + #[test] + fn capabilities_may_advertise_a_subset() { + let expected = json!({"result": {"capabilities": {"check": {"$provider": "boolean"}, "lookup": {"$provider": "boolean"}}}}); + // Per §7.3 absent means not advertised; advertised values are checked. + assert!(matches(&expected, &json!({"result": {"capabilities": {}}})).is_ok()); + assert!( + matches( + &expected, + &json!({"result": {"capabilities": {"check": false}}}) + ) + .is_ok() + ); + assert!( + matches( + &expected, + &json!({"result": {"capabilities": {"check": true, "custom": true}}}) + ) + .is_err(), + "undeclared capability ids violate the closed set" + ); + assert!( + matches( + &expected, + &json!({"result": {"capabilities": {"check": "yes"}}}) + ) + .is_err() + ); + } + + #[test] + fn verbatim_capability_map_keeps_exact_keys() { + // A `providerArgs` scenario asserting `reconstruct: false` must not + // pass when the provider omits `reconstruct` entirely. + let expected = json!({"result": {"capabilities": {"check": true, "reconstruct": false}}}); + assert!( + matches( + &expected, + &json!({"result": {"capabilities": {"check": true}}}) + ) + .is_err() + ); + assert!( + matches( + &expected, + &json!({"result": {"capabilities": {"check": true, "reconstruct": false}}}) + ) + .is_ok() + ); + } + + fn init_result() -> Value { + json!({ + "result": { + "protocolVersion": "1.5", + "serverInfo": { + "name": {"$provider": "nonEmptyString"}, + "version": {"$provider": "nonEmptyString"} + }, + "languages": {"$provider": "languageList"}, + "capabilities": { + "check": {"$provider": "boolean"}, + "compile": {"$provider": "boolean"}, + "reconstruct": {"$provider": "boolean"}, + "symbols": {"$provider": "boolean"}, + "definition": {"$provider": "boolean"}, + "references": {"$provider": "boolean"}, + "rename": {"$provider": "boolean"}, + "editValidation": {"$provider": "boolean"}, + "projectLoading": {"$provider": "boolean"}, + "sourceIdentity": {"$provider": "boolean"}, + "lookup": {"$provider": "boolean"} + } + } + }) + } + + #[test] + fn validate_leaves_enforces_marker_form_and_position() { + assert!(validate_leaves(&init_result(), true).is_ok()); + // Marker form. + assert!(validate_leaves(&json!({"result": {"protocolVersion": "1.5", "capabilities": {"check": {"$provider": "unknown"}}}}), true).is_err()); + assert!(validate_leaves(&json!({"$provider": "boolean", "x": 1}), true).is_err()); + assert!(validate_leaves(&json!({"$provider": "boolean"}), false).is_err()); + // Plain objects without the key are never markers. + assert!(validate_leaves(&json!({"provider": "boolean"}), false).is_ok()); + } + + #[test] + fn markers_reject_contract_owned_positions() { + // Marker on the root or a contract-owned leaf. + for expected in [ + json!({"$provider": "nonEmptyString"}), + json!({"result": {"protocolVersion": {"$provider": "nonEmptyString"}}}), + json!({"error": {"code": {"$provider": "nonEmptyString"}}}), + json!({"error": {"data": {"lpp": {"kind": {"$provider": "nonEmptyString"}}}}}), + json!({"result": {"serverInfo": {"$provider": "nonEmptyString"}}}), + ] { + assert!( + validate_leaves(&expected, true).is_err(), + "marker should be rejected: {expected}" + ); + } + } + + #[test] + fn marker_kind_must_match_position() { + let mut expected = init_result(); + expected["result"]["languages"] = json!({"$provider": "nonEmptyString"}); + assert!(validate_leaves(&expected, true).is_err()); + expected["result"]["serverInfo"]["name"] = json!({"$provider": "boolean"}); + assert!(validate_leaves(&expected, true).is_err()); + } + + #[test] + fn details_markers_follow_the_error_kind() { + // `reason` is provider-defined under `invalidDocument` ... + let expected = json!({"error": {"data": {"lpp": { + "kind": "invalidDocument", + "details": {"reason": {"$provider": "nonEmptyString"}} + }}}}); + assert!(validate_leaves(&expected, true).is_ok()); + // ... but a closed set under `invalidRequest`. + let expected = json!({"error": {"data": {"lpp": { + "kind": "invalidRequest", + "details": {"reason": {"$provider": "nonEmptyString"}} + }}}}); + assert!(validate_leaves(&expected, true).is_err()); + // `refusalCode` only under `refusal`; `supportedProtocolVersions` only + // under `protocolVersionMismatch`. + let expected = json!({"error": {"data": {"lpp": { + "kind": "invalidRequest", + "details": {"refusalCode": {"$provider": "nonEmptyString"}} + }}}}); + assert!(validate_leaves(&expected, true).is_err()); + let expected = json!({"error": {"data": {"lpp": { + "kind": "refusal", + "details": {"supportedProtocolVersions": {"$provider": "protocolVersions"}} + }}}}); + assert!(validate_leaves(&expected, true).is_err()); + } + + #[test] + fn marked_capabilities_cover_the_declared_set() { + let mut expected = init_result(); + // A declared id left unmarked weakens the closed-set check. + expected["result"]["capabilities"] + .as_object_mut() + .unwrap() + .remove("lookup"); + assert!(validate_leaves(&expected, true).is_err()); + // An undeclared id is not a provider-supplied position. + let mut expected = init_result(); + expected["result"]["capabilities"] + .as_object_mut() + .unwrap() + .insert("custom".into(), json!({"$provider": "boolean"})); + assert!(validate_leaves(&expected, true).is_err()); + // Markers cannot mix with verbatim capability values. + let mut expected = init_result(); + expected["result"]["capabilities"]["check"] = json!(true); + assert!(validate_leaves(&expected, true).is_err()); + // The declared set follows the negotiated version. + let mut expected = init_result(); + expected["result"]["protocolVersion"] = json!("1.0"); + for key in ["projectLoading", "sourceIdentity", "lookup"] { + expected["result"]["capabilities"] + .as_object_mut() + .unwrap() + .remove(key); + } + assert!(validate_leaves(&expected, true).is_ok()); + } +} diff --git a/conformance/runner/src/scenario.rs b/conformance/runner/src/scenario.rs index 142c5a5..6c19071 100644 --- a/conformance/runner/src/scenario.rs +++ b/conformance/runner/src/scenario.rs @@ -7,6 +7,8 @@ use std::path::Path; use serde::Deserialize; use serde_json::{Value, json}; +use crate::matcher::validate_leaves; + #[derive(Deserialize)] #[serde(rename_all = "camelCase")] pub(crate) struct Scenario { @@ -193,6 +195,10 @@ fn validate_scenario(scenario: &Scenario) -> Result<(), String> { )); } validate_expected(&step.expect_response).map_err(|e| format!("step {i}: {e}"))?; + validate_leaves(&step.expect_response, true).map_err(|e| format!("step {i}: {e}"))?; + if let StepRequest::Request(request) = &step.request { + validate_leaves(request, false).map_err(|e| format!("step {i}: {e}"))?; + } } Ok(()) } diff --git a/conformance/runner/src/session.rs b/conformance/runner/src/session.rs index a770fbb..62c1c16 100644 --- a/conformance/runner/src/session.rs +++ b/conformance/runner/src/session.rs @@ -73,9 +73,9 @@ pub(crate) fn run_scenario( let actual: Value = serde_json::from_str(&response_line) .map_err(|e| format!("step {i}: response is not valid JSON: {e}"))?; let expected = substitute_project_uri(&step.expect_response, project.as_ref()); - if actual != expected { + if let Err(detail) = crate::matcher::matches(&expected, &actual) { return Err(format!( - "step {i}: response mismatch\n expected: {}\n actual: {}", + "step {i}: response mismatch at {detail}\n expected: {}\n actual: {}", serde_json::to_string_pretty(&expected).expect("serializes"), serde_json::to_string_pretty(&actual).expect("serializes"), )); diff --git a/docs/spec/conformance.md b/docs/spec/conformance.md index 870edb5..3a87d55 100644 --- a/docs/spec/conformance.md +++ b/docs/spec/conformance.md @@ -18,7 +18,7 @@ contract: source-identity revision, its LPP 1.4 artifact-format negotiation revision, and its LPP 1.5 name-lookup revision. * `conformance/runner/`: a runner that replays fixtures against any provider - binary and compares responses exactly. + binary and compares responses. * `conformance/mock-provider/`: the reference provider for the demonstration language `x-demo-lang` (a puzzle/equation DSL). The mock is spawnable as a stdio binary so client-side integration can use it @@ -29,6 +29,48 @@ Running the suite and interpreting results is documented in not prove Workshop semantic correctness, game runtime behavior, or performance. +### 20.1 Provider-supplied leaves + +An `expectResponse` distinguishes contract-owned leaves, compared verbatim, +from leaves the spec declares provider-supplied. A provider-supplied leaf is +written `{ "$provider": "" }` and asserts the leaf's contract shape +rather than a literal value, so providers other than the reference mock can +satisfy protocol-scope scenarios: + +| Marker | Asserted shape | Where it applies | +| --- | --- | --- | +| `nonEmptyString` | any non-empty string | `serverInfo` fields, error `message` prose, `refusalCode` under `refusal`, `details.reason` under `invalidDocument`, `invalidEntry`, `projectLoadFailed`, and `invalidArtifact` | +| `boolean` | any boolean | `capabilities` values in `lpp/initialize` results | +| `languageList` | non-empty array of `{ "id": , "extensions": [] }` | the `languages` array | +| `protocolVersions` | non-empty array of `MAJOR.MINOR` strings | `supportedProtocolVersions` under `protocolVersionMismatch` | + +The marker positions above are a whitelist, and the runner enforces it at +fixture-validation time — including the sibling context a position depends +on, such as `data.lpp.kind` or the negotiated `protocolVersion`. A marker on +any other leaf — for example `protocolVersion`, `error.code`, a +spec-enumerated `invalidRequest` reason, or an undeclared capability id — is +a fixture error, as is a marker inside `request`. + +Two positions carry extra fixture rules: + +* A `capabilities` object that carries markers must mark exactly the + capability ids declared for the negotiated `protocolVersion` — no missing + or undeclared ids, and no markers mixed with verbatim values. This keeps + the closed capability set (§7.3) a normative assertion: a response that + advertises an undeclared id still fails, while a response that omits ids + it does not offer passes, since absent means not advertised. A scenario + asserting a specific negotiated subset — such as one configured through + `providerArgs` — keeps its capability map verbatim instead, and a verbatim + map keeps exact key matching: asserting `reconstruct: false` fails when + the provider omits `reconstruct` entirely. +* `details.reason` may be marked only under the error kinds whose values the + spec does not enumerate; the `invalidRequest` reasons are a closed set and + stay verbatim. + +Every unmarked leaf, including a marker object's own siblings, keeps verbatim +JSON equality. `providerArgs` is an adapter-owned surface and makes a +scenario adapter-specific. + ## Appendix A: Message and type index Methods: diff --git a/docs/spec/project-loading.md b/docs/spec/project-loading.md index 6cfe817..547f6fa 100644 --- a/docs/spec/project-loading.md +++ b/docs/spec/project-loading.md @@ -53,7 +53,7 @@ Initialization and capability negotiation. The client MUST send | `protocolVersion` | string | The protocol version the provider will speak: `"1.0"`, `"1.1"`, `"1.2"`, `"1.3"`, `"1.4"`, or `"1.5"`. | | `serverInfo` | object | `{ "name": string, "version": string }` identifying the provider. | | `languages` | array | One entry per source language the provider serves. | -| `capabilities` | object | One boolean field per capability. LPP 1.0 requires the eight fields listed below; LPP 1.1 and later additionally require `projectLoading`; LPP 1.3 and later also define `sourceIdentity`; LPP 1.5 and later also define `lookup`. | +| `capabilities` | object | The capabilities the provider advertises, as one boolean field per capability id. The object itself is required; its keys are drawn from the capability ids declared for the negotiated version (Section 7.3): LPP 1.0 declares the eight listed below, LPP 1.1 adds `projectLoading`, LPP 1.3 adds `sourceIdentity`, and LPP 1.5 adds `lookup`. A provider MAY advertise any subset — an absent id means not offered, equivalent to `false`. | Each language entry: `{ "id": string, "extensions": [string] }`. `extensions` is the list of file extensions the provider associates with the language,