Check servers with what REPLy and neorepl send

README.md

@@ -75,8 +75,8 @@ get a warning instead.
 
 Some of the checks send what a particular client sends, the way it sends
 it, e.g. `cider.connect` sends the requests CIDER sends while connecting.
-That way the report tells you directly whether CIDER, Calva, Conjure and
-vim-fireplace will work with your server.
+That way the report tells you directly whether CIDER, Calva, Conjure,
+vim-fireplace, REPLy and neorepl will work with your server.
 
 ## Checking Clients
 
@@ -128,8 +128,8 @@ the replies). `-like jank` turns on all of jank's scenarios at once, and
 
 proof is still in its early days. Right now it covers the core of the
 protocol (`describe`, unknown ops, `eval`, sessions, `stdin` and the wire format),
-what CIDER, Calva, Conjure and vim-fireplace send while connecting and
-evaluating code, the requests
+what CIDER, Calva, Conjure, vim-fireplace, REPLy and neorepl send while
+connecting and evaluating code, the requests
 clients send and the server differences clients have to deal with, and
 `proof list` will show you all the checks.
 

doc/design.md

@@ -132,8 +132,8 @@ it. Each situation becomes a check that sends those requests, the way
 the client sends them, and fails at the first reply the client couldn't
 use. Replies that only bother the client (e.g. an error the client just
 shows to the user) get a note. Right now there are profiles for CIDER,
-Calva, Conjure and vim-fireplace, built from what they send with their
-default settings.
+Calva, Conjure, vim-fireplace, REPLy and neorepl, built from what they
+send with their default settings.
 
 A new profile starts with a recording, as `proof proxy -record` saves
 what a client sent as a profile with a check for each connection. What

doc/profiles.md

@@ -121,7 +121,7 @@ snippet is missing, all the checks that need it are skipped.
 
 | Snippet | Options | What it should do | Checks that use it |
 |---|---|---|---|
-| `value` | `code`, `value` | evaluate to `value` | `eval.value`, `eval.survives-error`, `eval.ns`, `eval.unknown-ns`, `session.ephemeral`, `session.unknown`, `session.closed`, `cider.eval`, `cider.repl`, `calva.eval`, `conjure.eval`, `fireplace.eval` |
+| `value` | `code`, `value` | evaluate to `value` | `eval.value`, `eval.survives-error`, `eval.ns`, `eval.unknown-ns`, `session.ephemeral`, `session.unknown`, `session.closed`, `cider.eval`, `cider.repl`, `calva.eval`, `conjure.eval`, `fireplace.eval`, `neorepl.eval` |
 | `stdout` | `code`, `out` | print `out` to the standard output | `eval.stdout`, `eval.stdout-order` |
 | `stderr` | `code`, `err` | print `err` to the standard error | `eval.stderr` |
 | `throw` | `code` | raise an error | `eval.error-status`, `eval.error-report`, `eval.survives-error` |

doc/usage.md

@@ -89,16 +89,18 @@ eval
          see: nrepl/nrepl#147 https://github.com/nrepl/nrepl/issues/147
 ```
 
-The checks in the `cider`, `calva`, `conjure` and `fireplace` groups send
-what those clients send while connecting and evaluating code, with all
-the extra fields they put in their requests. If one of them fails, the
-client won't work properly with your server, and the report says what
-the client does with the reply it got (e.g. "CIDER gives up
-connecting"). The code they evaluate for the user is the `value`
-snippet of your profile. Before evaluating code from a source file,
-CIDER, Calva and Conjure evaluate an `ns` form for the file's namespace,
-so those checks also need the `clojure` capability. vim-fireplace finds the classpath
-with Java interop while connecting, so its checks need `java`.
+The checks in the `cider`, `calva`, `conjure`, `fireplace`, `reply` and
+`neorepl` groups send what those clients send while connecting and
+evaluating code, with all the extra fields they put in their requests.
+If one of them fails, the client won't work properly with your server,
+and the report says what the client does with the reply it got (e.g.
+"CIDER gives up connecting"). The code they evaluate for the user is the
+`value` snippet of your profile. (REPLy sends nothing but the code and
+the session, so `eval.value` covers it.) Before evaluating code from a
+source file, CIDER, Calva and Conjure evaluate an `ns` form for the
+file's namespace, so those checks also need the `clojure` capability.
+vim-fireplace finds the classpath with Java interop while connecting, so
+its checks need `java`.
 
 Each check gets one of the following verdicts:
 

internal/checks/checks_test.go

@@ -48,8 +48,9 @@ func TestWellBehavedServerPassesEverything(t *testing.T) {
 // other check and rule must still pass.
 func TestChecksCatchMisbehaviour(t *testing.T) {
 	F, W, S := check.Failed, check.Warned, check.Skipped
-	// What every client goes through first.
-	connects := map[string]check.Verdict{"cider.connect": F, "calva.connect": F, "conjure.connect": F, "fireplace.connect": F}
+	// What every client but REPLy goes through first.
+	connects := map[string]check.Verdict{"cider.connect": F, "calva.connect": F, "conjure.connect": F, "fireplace.connect": F,
+		"neorepl.connect": F}
 	cases := []struct {
 		name string
 		q    quirks
@@ -71,7 +72,8 @@ func TestChecksCatchMisbehaviour(t *testing.T) {
 		{"no session-closed", quirks{noSessionClosed: true}, map[string]check.Verdict{"session.close": F}},
 		// Only clients put dicts and lists in their requests.
 		{"flat fields only", quirks{flatFields: true}, map[string]check.Verdict{
-			"cider.connect": F, "cider.eval": F, "cider.repl": F, "calva.connect": F, "calva.eval": F, "conjure.eval": F}},
+			"cider.connect": F, "cider.eval": F, "cider.repl": F, "calva.connect": F, "calva.eval": F, "conjure.eval": F,
+			"neorepl.eval": F}},
 		{"any session accepted", quirks{acceptAnySession: true}, map[string]check.Verdict{"session.unknown": F, "session.closed": F}},
 		{"shared session state", quirks{sharedState: true}, map[string]check.Verdict{"session.isolated": F}},
 		{"sessions tied to sockets", quirks{socketSessions: true}, map[string]check.Verdict{"session.across-connections": W}},
@@ -82,6 +84,8 @@ func TestChecksCatchMisbehaviour(t *testing.T) {
 		{"no eval-error", quirks{noEvalError: true}, map[string]check.Verdict{"eval.error-status": F}},
 		{"no ex", quirks{noEx: true}, map[string]check.Verdict{"eval.error-report": W}},
 		{"no no-code", quirks{noNoCode: true}, map[string]check.Verdict{"eval.no-code": W}},
+		// REPLy evaluates nothing before its first prompt.
+		{"empty code never answered", quirks{emptyCodeHangs: true}, map[string]check.Verdict{"reply.connect": F}},
 		{"no ns", quirks{noNs: true}, map[string]check.Verdict{"eval.ns": W, "fireplace.connect": F}},
 		{"ns fallback", quirks{nsFallback: true}, map[string]check.Verdict{"eval.unknown-ns": F}},
 		{"two dones", quirks{twoDones: true}, map[string]check.Verdict{"wire.one-done": W}},
@@ -93,7 +97,8 @@ func TestChecksCatchMisbehaviour(t *testing.T) {
 			"wire.id": W, "eval.stdout": F, "eval.stderr": F, "eval.error-report": W}},
 		{"integer value", quirks{intValue: true}, map[string]check.Verdict{
 			"wire.field-types": F, "eval.value": F, "eval.survives-error": F, "eval.multiple-forms": F,
-			"session.ephemeral": F, "session.persistent": F, "cider.eval": F, "cider.repl": F, "calva.eval": F, "conjure.eval": F, "fireplace.eval": F}},
+			"session.ephemeral": F, "session.persistent": F, "cider.eval": F, "cider.repl": F, "calva.eval": F, "conjure.eval": F, "fireplace.eval": F,
+			"neorepl.eval": F}},
 		{"unsorted keys", quirks{unsortedKeys: true}, map[string]check.Verdict{"wire.canonical": W}},
 		{"invalid UTF-8", quirks{badUTF8: true}, map[string]check.Verdict{"wire.utf8": W, "eval.stdout": F}},
 		{"no stdin op", quirks{noStdinOp: true}, map[string]check.Verdict{

internal/checks/client_profiles_test.go

@@ -22,7 +22,8 @@ func TestClientProfiles(t *testing.T) {
 	pinned := regexp.MustCompile(`^https://github\.com/[^/]+/[^/]+/blob/[0-9a-f]{40}/`)
 	// The links in refs.go to the same clients.
 	bases := map[string]string{"CIDER": ciderBase, "Calva": strings.TrimSuffix(calvaBase, "nrepl/index.ts"),
-		"Conjure": strings.TrimSuffix(conjureBase, "client/clojure/nrepl/server.fnl"), "vim-fireplace": fireplaceBase}
+		"Conjure": strings.TrimSuffix(conjureBase, "client/clojure/nrepl/server.fnl"), "vim-fireplace": fireplaceBase,
+		"REPLy": strings.TrimSuffix(replyBase, "eval_modes/nrepl.clj")}
 	for _, p := range clientProfiles() {
 		if p.Name == "" || !pinned.MatchString(p.Code) {
 			t.Errorf("%q needs a name and links pinned to a commit, got %q", p.Name, p.Code)

internal/checks/clients/neorepl.toml

@@ -0,0 +1,54 @@
+# What neorepl's REPL sends while it starts and when it evaluates code,
+# with its default settings and without a terminal. In a terminal it also
+# looks things up and completes names as the user types, but it carries
+# on when that fails. The links point at the code that needs what the
+# steps check.
+name = "neorepl"
+code = "https://github.com/nrepl/neorepl/blob/462a182c885ff56d65acfc73ad1fa92f02fc7181/src/neorepl/"
+
+[[checks]]
+id = "neorepl.connect"
+title = "neorepl can start its REPL"
+why = "neorepl asks for describe, clones a session and asks for describe again, in that session, before it shows a prompt, so a server that gets any of it wrong can't be used from neorepl at all."
+
+# The ops tell neorepl what the server can do, e.g. look things up.
+[[checks.steps]]
+send = { op = "describe" }
+dicts = [["ops"], ["versions"]]
+why = "neorepl takes the server for one that isn't running, and says so or starts a server of its own for the project"
+refs = ["nrepl.clj#L153-L168", "discovery.clj#L38-L45"]
+
+[[checks.steps]]
+send = { op = "clone" }
+new-session = "repl"
+why = "neorepl can't start its REPL"
+refs = ["main.clj#L735-L737", "nrepl.clj#L185-L188"]
+
+# For the namespace of the first prompt, which is user when the reply
+# doesn't say.
+[[checks.steps]]
+send = { op = "describe", session = "$repl" }
+why = "neorepl can't start its REPL"
+refs = ["repl.clj#L194", "nrepl.clj#L199"]
+
+[[checks]]
+id = "neorepl.eval"
+title = "neorepl can evaluate code"
+why = "neorepl sends what the user types with options for pretty printing the values, so a server that can't handle them leaves neorepl users without the results of their code."
+
+[[checks.steps]]
+send = { op = "clone" }
+new-session = "repl"
+why = "neorepl can't start its REPL"
+refs = ["main.clj#L735-L737", "nrepl.clj#L185-L188"]
+
+[[checks.steps]]
+snippet = "value"
+why = "users don't see the result of their code"
+refs = ["repl.clj#L16-L24", "repl.clj#L270", "evaluation.clj#L183-L200"]
+
+[checks.steps.send]
+op = "eval"
+session = "$repl"
+"nrepl.middleware.print/print" = "nrepl.util.print/pprint"
+"nrepl.middleware.print/options" = { right-margin = 80 }