Vault from a newer MorphGate panicked upon running with old version of the app

CHANGELOG.md

@@ -39,6 +39,8 @@ prompt. Write them for users.
 - Two devices refreshing the same cloud source at once no longer make duplicate hosts or
   "(conflict)" copies.
 - About links to the documentation (morphgate.com/docs) and the source code on GitHub.
+- A vault saved by a newer MorphGate (after going back to an older version) no longer stops the app
+  from opening: its window says to install the latest version, and the vault is left as it is.
 - Copy buttons (a public key, the recovery key, an invite code, host key details) fall back to
   another way of copying when the system refuses the first, and say "Couldn't copy" instead of
   doing nothing when neither works.

apps/desktop/e2e/README.md

@@ -4,7 +4,7 @@ These run the real AppImage, deb and rpm binaries, not a dev build, and drive th
 
 | Scenario | What it checks |
 | --- | --- |
-| `appimage.mjs` | Creates a vault, which isn't offered sync then but is a week later (the page reloaded with the time moved on), once; SSHes to a throwaway local sshd and runs a command; imports two hosts and their key from a `~/.ssh/config` (one behind the other as its jump host) and connects through it; imports a PuTTY session with a new key in a `.ppk` file, a Termius-style CSV file with a group and a Royal TS document, and signs in with the PuTTY key; connects Vallic Cloud through a stand-in console (`vallic-console.mjs`: device sign-in, rotating tokens, an environment that is this sshd) and signs in to its host; adds AWS and DigitalOcean through stand-in CLIs (`clouds/`); adds a feed (`host-feed.mjs`) whose host connects with the host key it lists, whose wrongly listed host is refused and whose changed snippet asks before it runs; signs in with a key that only has a certificate from the run's user CA (which sshd trusts), from the vault and from an agent; reaches a host by a command its jump host runs (with a login command instead of the shell), and another by a proxy command on this computer (both the relay in `crates/mg-ssh/tests/relay.py`); selects two hosts, moves them into a group and deletes them; signs in to a host with one key from an `ssh-agent` (its own socket) and forwards it (`ssh-add -l` on the host lists it); splits a tab into three panes (beside and below, one through a jump host), broadcasts a command to all of them and then types into one, drags a border and closes them; opens a `morphgate://connect` link from a second launch (which hands it over and exits) and connects from the filled-in quick connect, and refuses an unknown link; refuses a tampered update twice without touching the AppImage; installs the signed update, restarts, unlocks the same vault as the new version, and finds nothing newer |
+| `appimage.mjs` | Creates a vault, which isn't offered sync then but is a week later (the page reloaded with the time moved on), once; SSHes to a throwaway local sshd and runs a command; imports two hosts and their key from a `~/.ssh/config` (one behind the other as its jump host) and connects through it; imports a PuTTY session with a new key in a `.ppk` file, a Termius-style CSV file with a group and a Royal TS document, and signs in with the PuTTY key; connects Vallic Cloud through a stand-in console (`vallic-console.mjs`: device sign-in, rotating tokens, an environment that is this sshd) and signs in to its host; adds AWS and DigitalOcean through stand-in CLIs (`clouds/`); adds a feed (`host-feed.mjs`) whose host connects with the host key it lists, whose wrongly listed host is refused and whose changed snippet asks before it runs; signs in with a key that only has a certificate from the run's user CA (which sshd trusts), from the vault and from an agent; reaches a host by a command its jump host runs (with a login command instead of the shell), and another by a proxy command on this computer (both the relay in `crates/mg-ssh/tests/relay.py`); selects two hosts, moves them into a group and deletes them; signs in to a host with one key from an `ssh-agent` (its own socket) and forwards it (`ssh-add -l` on the host lists it); splits a tab into three panes (beside and below, one through a jump host), broadcasts a command to all of them and then types into one, drags a border and closes them; opens a `morphgate://connect` link from a second launch (which hands it over and exits) and connects from the filled-in quick connect, and refuses an unknown link; refuses a tampered update twice without touching the AppImage; installs the signed update, restarts, unlocks the same vault as the new version, and finds nothing newer; and with the vault's schema raised past what it knows (as a newer MorphGate leaves it), the window says to install the latest version and the vault stays as it was |
 | `package.mjs` (deb, rpm) | The binary knows how it was installed, finds the update, and warns that installing asks for an administrator password (installing needs root, so it stops there) |
 | `teams.mjs` | Two apps side by side (Alice and Bob) with a local reference sync server: both turn on sync; Alice creates an organization and a vault and invites Bob; Bob joins only after the owner's fingerprint matches, and Alice confirms him only after his matches; a host saved in the shared vault reaches Bob, his edit reaches Alice; Bob publishes his SSH key; Alice revokes him (the vault leaves his device) and rotates the vault key; members may create vaults; the activity log tells the story. Then Alice offboards Bob against the local sshd: removal and key rotation, his key found and removed from `authorized_keys`, and the host's shared key replaced (she still logs in) |
 | `layout.mjs` | Turns sync on from the offer a vault without sync gets a week on, fills every page (hosts in a group, a key, a snippet, a tunnel, a team with an invite to a long address), and at 860, 1024, 1280 and 1600 px wide checks that nothing on a page overlaps or runs out of its column. Closing an editor returns to the page it was opened from, and closing a page to the session under it; with no session open, a page takes the tab bar's place |

apps/desktop/e2e/android-monkey.mjs

@@ -49,6 +49,8 @@ async function until(what, fn, ms = 90000) {
   throw new Error(`timed out waiting for ${what}`);
 }
 
+const CRASHED = "the page crashed";
+
 /** The app's page over the DevTools protocol: `evaluate` runs an expression (awaiting a promise),
  * `thrown` collects what the page throws (during `phase`: the app's start, the setup, the walk),
  * `gone` says why the connection ended. */
@@ -83,7 +85,7 @@ async function devtools(pid) {
       const e = msg.params.exceptionDetails;
       d.thrown.push(`${d.phase}: ${(e.exception?.description ?? e.text).split("\n")[0]}`);
     } else if (msg.method === "Inspector.targetCrashed") {
-      d.gone = "the page crashed";
+      d.gone = CRASHED;
     } else if (msg.method === "Inspector.detached") {
       d.gone = `the page went away (${msg.params.reason})`;
     }
@@ -109,7 +111,14 @@ async function devtools(pid) {
     ws.close();
     tryAdb("forward", "--remove", `tcp:${port}`);
   };
-  await send("Inspector.enable");
+  // Chromium answers Inspector.enable with targetCrashed while the page's renderer hasn't started
+  // yet (a WebView just made, before its page loads): ask again until it has, for up to 30 s.
+  for (let tries = 60; ; tries--) {
+    await send("Inspector.enable");
+    if (d.gone !== CRASHED || !tries) break;
+    d.gone = null;
+    await sleep(500);
+  }
   await send("Runtime.enable");
   return d;
 }
@@ -213,7 +222,7 @@ async function fail(dir, round, why, extra = []) {
   const report = [`Android monkey: ${why}`, `seed ${SEED}, round ${round} of ${ROUNDS} (MONKEY_SEED=${SEED} repeats it)`, ...extra].join("\n");
   writeFileSync(`${out}/failure.txt`, `${report}\n`);
   writeFileSync(`${dir}/end.png`, execFileSync(ADB, ["exec-out", "screencap", "-p"], { maxBuffer: 64 << 20 }));
-  throw new Error(report);
+  throw Object.assign(new Error(report), { reported: true });
 }
 
 // From no data: round 1 creates the vault (so not a phone whose MorphGate you use).
@@ -232,9 +241,10 @@ for (let round = 1; round <= ROUNDS; round++) {
   const logcat = spawn(ADB, ["logcat", "-v", "threadtime"], { stdio: ["ignore", "pipe", "ignore"] });
   logcat.stdout.pipe(createWriteStream(`${dir}/logcat.txt`));
   let d = null;
+  let pid = "";
   try {
     adb("shell", "monkey", "-p", PKG, "-c", "android.intent.category.LAUNCHER", "1");
-    const pid = await until("the app's process", () => tryAdb("shell", "pidof", PKG));
+    pid = await until("the app's process", () => tryAdb("shell", "pidof", PKG));
     d = await devtools(pid);
     d.phase = "setup";
     await unlock(d);
@@ -299,6 +309,15 @@ for (let round = 1; round <= ROUNDS; round++) {
     writeFileSync(`${dir}/end.png`, execFileSync(ADB, ["exec-out", "screencap", "-p"], { maxBuffer: 64 << 20 }));
     const slow = anrs ? `; ${anrs} ANR${anrs > 1 ? "s" : ""} noted${stuck.length ? " (anr.txt)" : ""}, then it drew again` : "";
     console.log(`  round ${round}: ${injected} events, nothing crashed or threw${slow}; in front: ${focus}`);
+  } catch (e) {
+    // Anything else that ended the round (the page never came, a screen never showed): a failure
+    // too, with the round's log once it has caught up.
+    if (e.reported) throw e;
+    await sleep(2000);
+    logcat.kill();
+    await sleep(500);
+    const problems = pid ? logProblems(readFileSync(`${dir}/logcat.txt`, "utf8"), pid) : [];
+    await fail(dir, round, `${e.message} (during the ${d?.phase ?? "launch"})`, problems);
   } finally {
     logcat.kill();
     d?.close();

apps/desktop/e2e/appimage.mjs

@@ -1088,3 +1088,35 @@ try {
 } finally {
   await s.close();
 }
+
+// A vault a newer MorphGate wrote (a schema this version doesn't know): the window says to
+// install the latest version instead of the app quitting, and the vault stays as it was. The
+// last session's app outlives it: stop it, or the launch hands over to it.
+for (const pid of appImagePids()) process.kill(pid, "SIGTERM");
+for (let i = 0; i < 50 && appImagePids().length > 0; i++) await new Promise((r) => setTimeout(r, 200));
+{
+  const file = `${process.env.XDG_DATA_HOME}/com.morphgate.desktop/vault.sqlite3`;
+  const { DatabaseSync } = await import("node:sqlite");
+  const schema = () => {
+    const db = new DatabaseSync(file);
+    try {
+      return db.prepare("PRAGMA user_version").get().user_version;
+    } finally {
+      db.close();
+    }
+  };
+  const newer = schema() + 1;
+  const db = new DatabaseSync(file);
+  db.exec(`PRAGMA user_version = ${newer}`);
+  db.close();
+  s = await session(app);
+  try {
+    const text = await s.waitText("newer version of MorphGate", "body", 60000);
+    assert.match(text, /morphgate\.com\/download/);
+    await s.screenshot(`${shots}/newer-vault.png`);
+  } finally {
+    await s.close();
+  }
+  assert.equal(schema(), newer);
+  ok("a vault a newer MorphGate wrote: the window says to install the latest version, and the vault is left as it was");
+}

apps/desktop/src-tauri/src/lib.rs

@@ -99,10 +99,20 @@ pub fn run() {
             }
             let data_dir = app.path().app_data_dir()?;
             let changed = std::sync::Arc::new(tokio::sync::Notify::new());
-            app.manage(vault::VaultState::open(
-                &data_dir.join("vault.sqlite3"),
-                changed.clone(),
-            )?);
+            // A vault that won't open (one a newer MorphGate wrote, say) leaves the app running
+            // to say why: the window shows what vault_status returns. Nothing else starts.
+            let opened =
+                match vault::VaultState::open(&data_dir.join("vault.sqlite3"), changed.clone()) {
+                    Ok(v) => {
+                        app.manage(v);
+                        true
+                    }
+                    Err(e) => {
+                        tracing::error!("opening the vault: {e}");
+                        app.manage(vault::Unopened(e.to_string()));
+                        false
+                    }
+                };
             app.manage(ssh::SshState::new(data_dir));
             app.manage(sync::SyncState::new(changed));
             app.manage(vpn::VpnManager::default());
@@ -128,7 +138,9 @@ pub fn run() {
                     links::received(app.handle(), urls);
                 }
             }
-            sync::spawn_loop(app.handle().clone());
+            if opened {
+                sync::spawn_loop(app.handle().clone());
+            }
             Ok(())
         })
         .invoke_handler(tauri::generate_handler![

apps/desktop/src-tauri/src/vault.rs

@@ -14,6 +14,9 @@ use crate::util::expand_home;
 
 pub struct VaultState(pub(crate) Arc<Mutex<LocalVault>>, Arc<Notify>);
 
+/// Why the vault file didn't open (then there's no [`VaultState`]): the window says so.
+pub struct Unopened(pub String);
+
 impl VaultState {
     /// `changed` is notified after every local edit (and unlock) so sync can run soon.
     pub fn open(path: &Path, changed: Arc<Notify>) -> mg_store::Result<Self> {
@@ -67,8 +70,14 @@ pub(crate) fn save(
 // ---------------------------------------------------------------- lock / unlock
 
 #[tauri::command]
-pub fn vault_status(vault: State<'_, VaultState>) -> Result<Status, String> {
-    vault.guard()?.status().map_err(err)
+pub fn vault_status(app: tauri::AppHandle) -> Result<Status, String> {
+    use tauri::Manager;
+    match app.try_state::<VaultState>() {
+        Some(vault) => vault.guard()?.status().map_err(err),
+        None => Err(app
+            .try_state::<Unopened>()
+            .map_or_else(|| "the vault didn't open".to_owned(), |u| u.0.clone())),
+    }
 }
 
 /// Returns the recovery code; the UI shows it once and never stores it.

apps/desktop/src/App.tsx

@@ -107,7 +107,7 @@ export function App() {
     return () => void off.then((f) => f());
   }, [loadTermSettings]);
 
-  if (fatal) return <div className="gate"><p className="error">{fatal}</p></div>;
+  if (fatal) return <div className="gate"><p className="error gate-error">{fatal}</p></div>;
   if (status === null) return <div className="gate" />;
   if (status === "uninitialized") return <SetupScreen erased={erased} onDone={(fresh) => (setErased(null), fresh ? syncOfferTime.schedule() : syncOfferTime.forget(), setStatus("unlocked"))} />;
   if (status === "locked") return <UnlockScreen onUnlocked={() => setStatus("unlocked")} />;

apps/desktop/src/styles.css

@@ -406,6 +406,18 @@ p.muted {
   text-align: center;
 }
 
+/* Why the vault didn't open (a newer MorphGate's, say), in place of the app. Errors start in
+ * lower case: this one is a sentence on its own. */
+.gate-error {
+  max-width: 440px;
+  text-align: center;
+  text-wrap: balance;
+}
+
+.gate-error::first-letter {
+  text-transform: uppercase;
+}
+
 /* "or with the master password", between two rules. */
 .divider-text {
   display: flex;

crates/mg-store/src/error.rs

@@ -30,6 +30,11 @@ pub enum Error {
     Untrusted(String),
     #[error("vault data is corrupt: {0}")]
     Corrupt(String),
+    /// The vault file has a schema this app doesn't know: a newer MorphGate wrote it.
+    #[error(
+        "this vault was saved by a newer version of MorphGate (vault format {found}; this version reads up to {known}). Install the latest version from morphgate.com/download to open it; the vault is unchanged."
+    )]
+    NewerVault { found: usize, known: usize },
     /// A signed copy of an item older than one this device has seen: a server serving an old
     /// envelope again.
     #[error(

crates/mg-store/src/schema.rs

@@ -159,9 +159,10 @@ pub(crate) fn migrate(conn: &mut Connection) -> Result<()> {
     let current: i64 = conn.pragma_query_value(None, "user_version", |r| r.get(0))?;
     let current = usize::try_from(current).unwrap_or(0);
     if current > MIGRATIONS.len() {
-        return Err(crate::Error::Corrupt(format!(
-            "vault was written by a newer MorphGate (schema {current}); please update the app"
-        )));
+        return Err(crate::Error::NewerVault {
+            found: current,
+            known: MIGRATIONS.len(),
+        });
     }
     for (i, sql) in MIGRATIONS.iter().enumerate().skip(current) {
         let tx = conn.transaction()?;
@@ -216,4 +217,21 @@ mod tests {
         )
         .unwrap();
     }
+
+    /// A vault from a newer MorphGate is refused as such (not as corrupt), and left as it was.
+    #[test]
+    fn newer_schema_is_refused() {
+        let mut conn = Connection::open_in_memory().unwrap();
+        let newer = i64::try_from(MIGRATIONS.len() + 1).unwrap();
+        conn.pragma_update(None, "user_version", newer).unwrap();
+        let e = migrate(&mut conn).unwrap_err();
+        assert!(
+            matches!(e, crate::Error::NewerVault { found, known } if found == MIGRATIONS.len() + 1 && known == MIGRATIONS.len())
+        );
+        assert!(e.to_string().contains("newer version of MorphGate"));
+        let after: i64 = conn
+            .pragma_query_value(None, "user_version", |r| r.get(0))
+            .unwrap();
+        assert_eq!(after, newer);
+    }
 }

docs/architecture.md

@@ -43,7 +43,7 @@ How the MorphGate app is put together: the webview, the Tauri commands behind it
 | Reference server | `server/` (`vault-sync-server`) | A self-hostable `vault-sync/v1` server. See [Reference server](features/reference-server.md) |
 | Conformance suite | `tools/vault-conform` | Checks any `vault-sync/v1` server against the protocol, endpoint by endpoint. See [Development](development.md#conformance-suite) |
 
-The whole repository is one Cargo workspace (`Cargo.toml`, edition 2024, Rust 1.91 or newer, one version for every crate: 0.2.0). Argon2 and BLAKE2 are compiled optimized even in debug builds, because unoptimized Argon2 makes tests unusable.
+The whole repository is one Cargo workspace (`Cargo.toml`, edition 2024, Rust 1.91 or newer, one version for every crate: 0.2.1). Argon2 and BLAKE2 are compiled optimized even in debug builds, because unoptimized Argon2 makes tests unusable.
 
 ## The webview and the shell
 
@@ -91,7 +91,7 @@ Everything lives in the app's data folder (Tauri's `app_data_dir` for the identi
 
 ### The SQLite schema
 
-`crates/mg-store/src/schema.rs` migrates by `PRAGMA user_version` (7 migrations). A vault written by a newer app is refused ("please update the app") instead of being misread.
+`crates/mg-store/src/schema.rs` migrates by `PRAGMA user_version` (8 migrations). A vault written by a newer app is refused instead of being misread, and the window says to install the latest version.
 
 | Table | Holds |
 | --- | --- |

docs/features/vault-and-unlock.md

@@ -12,7 +12,7 @@ The vault is one SQLite file on the device that holds every item as a signed, en
 | macOS | `~/Library/Application Support/com.morphgate.desktop/` |
 | Windows | `%APPDATA%\com.morphgate.desktop\` |
 
-`LocalVault::open` creates the file with mode 0600 on Unix, turns on `foreign_keys` and `secure_delete` (deleted rows are overwritten, not only unlinked), and migrates by `PRAGMA user_version` (7 migrations in `schema.rs`). A file from a newer app is refused: "vault data is corrupt: vault was written by a newer MorphGate (schema N); please update the app". On desktops a second launch hands over to the running app (single instance per data folder), so only one process opens the file. The tables are listed in [App architecture](../architecture.md#the-sqlite-schema).
+`LocalVault::open` creates the file with mode 0600 on Unix, turns on `foreign_keys` and `secure_delete` (deleted rows are overwritten, not only unlinked), and migrates by `PRAGMA user_version` (8 migrations in `schema.rs`). A file from a newer app is refused, and left as it is (`Error::NewerVault`): the app still starts, without the vault, and its window says "this vault was saved by a newer version of MorphGate (vault format N; this version reads up to M). Install the latest version from morphgate.com/download to open it; the vault is unchanged." (`vault_status` returns it; nothing else runs). On desktops a second launch hands over to the running app (single instance per data folder), so only one process opens the file. The tables are listed in [App architecture](../architecture.md#the-sqlite-schema).
 
 ### What is in the clear