Secrets contract, c op / c secrets commands, and c secrets init

.eslintrc.json

@@ -3,5 +3,20 @@
     "parserOptions": {
         "ecmaVersion": 2017
     },
+    "overrides": [
+        {
+            "files": ["test/**/*.js", "vitest.config.mjs"],
+            "parserOptions": {
+                "ecmaVersion": 2020,
+                "sourceType": "module"
+            },
+            "rules": {
+                "node/no-unsupported-features/es-syntax": "off",
+                "node/no-unsupported-features/node-builtins": "off",
+                "node/no-missing-import": "off",
+                "node/no-unpublished-import": "off"
+            }
+        }
+    ],
     "root": true
 }

.github/workflows/test.yaml

@@ -0,0 +1,32 @@
+name: Test
+
+on:
+    push:
+    pull_request:
+
+jobs:
+    test:
+        runs-on: ubuntu-latest
+
+        steps:
+            - uses: actions/checkout@v4
+
+            - uses: actions/setup-node@v4
+              with:
+                  cache: npm
+                  node-version: 22
+
+            # The real 1Password CLI for the op stdin-detection tests - no
+            # session or account is needed (op validates stdin before
+            # authenticating). Pinned so the detection-error expectations
+            # stay stable; bump deliberately alongside a local check.
+            - name: Install 1Password CLI
+              run: |
+                  curl -sSfL "https://cache.agilebits.com/dist/1P/op2/pkg/v2.39.0/op_linux_amd64_v2.39.0.zip" -o op.zip
+                  unzip -o -d op op.zip
+                  install -m 755 op/op /usr/local/bin/op
+                  op --version
+
+            - run: npm ci
+
+            - run: npm test

.npmignore

@@ -1,2 +1,5 @@
 .editorconfig
 .eslintrc
+.github
+test
+vitest.config.mjs

CHANGELOG.md

@@ -2,6 +2,26 @@
 
 This file is a history of the changes made to @idearium/cli.
 
+## v6.2.0 - 2026-09-30
+
+### Added
+
+-   The secrets contract: a `secrets` block in `c.js` (services, a sibling `shared` block for keys whose GSM ids carry no service segment, and `dev` build-time consumers) that becomes the single source of truth for secret-shaped manifests. With a contract present, `c kc manifests|apply|start` generate SecretProviderClasses, SecretSyncs, the secret-sync ServiceAccount (deployed environments) and local Secrets (1Password references); a conflicting committed file is a hard error. Without a contract, manifest behaviour is unchanged.
+-   `c op get|set` and `c secrets get|push|add|verify|ls|consumers|init`. Values are only ever displayed as truncated hashes (`--plaintext` is terminal-gated and agent-blocked with a discouraged `--force`); writes flow through stdin pipes; `push` verifies its GSM readback by hash; `add` is ordered, idempotent and rollback-safe across c.js, 1Password and GSM and never touches the cluster; `consumers` gates deletions and annotates each consuming function with the env parsed from its name; `init` drafts a contract from existing declarations (warm 1Password session required), binding `shared.opItem` and removing legacy duplicates by comparing value hashes.
+-   `opItem` references (`services.<name>`/`dev`, one hop) so each 1Password item id appears exactly once in the contract. Services resolve (or create) their item lazily by title; the `shared` block never auto-creates.
+-   Friendly credential pre-flights: commands that need 1Password, gcloud or kubectl fail fast with actionable re-authentication errors instead of raw tool output.
+-   The cli owns the 1Password session lifecycle: `c op session` signs in and stores the raw session token at `~/.local/state/idearium/op-session` (`0600`, enforced on overwrite), and `c op cmd [command...]` runs any `op` command with that session passed via `--session` (exit code propagates). Every internal `op` invocation (`c op get|set`, `c secrets *`, `c kc` template injection) passes the session the same way, so no `OP_SESSION_<account>` env var name is guessed anywhere; an exported `OP_SESSION_idearium` still takes precedence for `eval $(op signin)` users.
+-   `c secrets verify` prints a headed section per check phase with output streaming as each check runs, prefetches its data in bulk (one 1Password item fetch per item, one GSM list, bounded-concurrency GSM reads, one kubectl fetch per secret), and reports function bindings referencing deleted GSM secrets (`MISSING`), contract drift (`STALE`) and cross-env misbindings (`MISBOUND`).
+
+### Fixed
+
+-   c.js contract edits are anchored to `secrets.services` and verified after writing: `c op set` and `c secrets add` previously matched the first `name: {` block anywhere in `c.js`, silently editing a same-named docker location instead of the contract service. Every edit is now re-parsed from disk and rolled back when it does not land in the contract.
+-   Generating local Secret manifests for unbound owners (a service without an `opItem`, or an unbound `shared` block) embedded a literal `null` as the secret value; generation now fails with a remedy instead.
+-   `c op get` no longer creates 1Password items for unbound services (reads never create; write paths still resolve-or-create).
+-   A bare or invalid `--length`/`--type` on `c op set --generate` and `c secrets add --generate` failed with a raw crypto error (`NaN`); both now fail with usage hints.
+-   `c secrets verify` now compares each synced Kubernetes Secret's value hash against GSM (hashes were previously displayed with a hard-coded `ok`, so drifted values still reported `VERIFIED`).
+-   `c op cmd` and `c kc cmd` now dispatch with the verbatim argv: commander's git-style dispatch re-parsed subcommand args at every layer first, silently dropping `--` and unknown options (breaking e.g. `c op cmd run -- sh -c ...` and `c kc cmd` commands carrying kubectl flags).
+
 ## v6.1.0 - 2026-09-16
 
 ### Added

README.md

@@ -37,7 +37,9 @@ The following is a summary of the top level commands.
 -   `c mk` is for everything Minikube.
 -   `c mongo` is for MongoDB connections.
 -   `c npm` is for everything NPM.
+-   `c op` manages 1Password secret values for the secrets contract.
 -   `c project` is for project management.
+-   `c secrets` manages project secrets across their stores (1Password, GSM, Kubernetes).
 
 Be aware that `kubectl` uses a global configuration, but the `c kc` command supercedes those where possible and ensures kubectl runs in the context of the project you're within.
 
@@ -271,6 +273,8 @@ The `c kc apply` will automatically provide the values for `namespace`, `prefix`
 
 ##### Secret templates
 
+> With a secrets contract in `c.js` (see [Secrets configuration](#secrets-configuration) below), secret-shaped templates are generated automatically — you don't commit them. This section applies to repositories without a contract.
+
 Any template (`.yaml.tmpl`) can contain [1Password secret references](https://www.1password.dev/connect/knowledge-base/secrets-references/) (i.e. `op://vault/item/section/field`). The template is rendered (placeholders) first, and then any references are resolved with the 1Password cli (`op inject`) - only when present. Rendering first means secret values never flow through the template engine (a value containing curly braces could otherwise be mangled). As a consequence, references must be **bare** (i.e. `op://vault/item/section/field`, not wrapped in `{{ }}`): placeholders are gone by the time references are resolved, and a wrapped reference would be blanked as an unknown placeholder. When references are present, the cli first ensures the 1Password cli is installed and authenticated (via `op whoami`), halting with a friendly message if not (`eval $(op signin)`). Templates without secret references never invoke `op`.
 
 This makes it possible to commit a template containing secret references, and have the compiled manifest (within `.compiled`, which should be gitignored) contain the actual secrets, ready to be deployed to Kubernetes.
@@ -292,6 +296,60 @@ When the template is compiled, each `stringData` value will be base64 encoded an
 
 Compiled secret manifests contain plaintext secrets, so they are treated as sensitive: they're written with `0600` permissions, and removed once they've been applied to Kubernetes. `c kc start` and `c kc apply` remove them after applying, and `c skaffold dev` removes them (via `c kc secrets-clean`) when it exits. `c kc manifests` intentionally leaves them in place, as its compiled output is its purpose. This only applies to the local environment; other environments are unaffected.
 
+### Secrets configuration
+
+The `secrets` block of `c.js` is the secrets contract: the single source of truth for every secret-shaped manifest and value reference. With a contract present, the cli generates SecretProviderClasses, SecretSyncs and the secret-sync ServiceAccount for deployed environments, and local Secrets (1Password references) for the local environment, whenever manifests are compiled (`c kc manifests`, `c kc apply`, `c kc start`). A committed manifest file that the contract also generates is a hard error - the contract is the one true source. Without a contract, nothing changes.
+
+```js
+secrets: {
+    dev: {
+        opItem: '{dev-item-id}', // {org}-{repo}/dev
+        consumers: {
+            'site-build': { keys: ['FONTAWESOME_AUTH_TOKEN'] },
+        },
+    },
+    services: {
+        site: {
+            opItem: '{item-id}',
+            keys: ['CACHE_URL', 'RAS_API_KEY'], // owned
+            sharedKeys: ['IMGIX_TOKEN'], // borrowed
+        },
+        'prismic-published-workflow': {
+            sharedKeys: ['prismic-published-algolia.BASIC_AUTH'], // dotted owner.KEY
+        },
+    },
+    shared: {
+        // keys whose GSM ids carry no service segment
+        // (org-repo-env-key); opItem points at the item storing the
+        // values, usually a reference - never auto-created.
+        opItem: 'services.site',
+        keys: ['IMGIX_TOKEN'],
+    },
+},
+```
+
+Rules (enforced at load):
+
+-   `opItem` values are 1Password item ids or `services.<name>`/`dev` references (one hop - each id appears exactly once). Services resolve (or create) their item lazily by title; the `shared` block never auto-creates.
+-   A bare `sharedKeys` name resolves to its unique owner across `services` and `shared`; dotted `'owner.KEY'` disambiguates.
+-   GSM secret ids derive as `{org}-{repo}-{env}[-{service}]-{key}` - no literal ids anywhere.
+-   Region comes from `gcloud.region`; project from `project.gcpProjectId`.
+
+Commands (values are only ever displayed as truncated hashes; `--plaintext` requires a terminal and is blocked in agent/CI contexts unless forced):
+
+-   `c secrets init` - draft a contract from existing declarations (requires a warm 1Password session; no-op when a contract exists). Binds `shared.opItem` by comparing value hashes and removes value-verified legacy duplicates.
+-   `c secrets add <service> <env> <KEY>` - add a key end to end (1Password, GSM, c.js). Never touches the cluster.
+-   `c secrets push <service> <env> <KEY>` - push the 1Password value to GSM, verified by hash readback.
+-   `c secrets get <service> <env> <KEY>` / `c op get` - value summaries (hash, reference, length).
+-   `c op set <service> <env> <KEY>` - upsert a value from stdin (or `--generate`).
+-   `c secrets verify <env>` - reconcile structure + 1Password/GSM/Kubernetes hashes, stale 1Password fields (`op:EXTRA`), deployed-functions drift (`STALE`, `MISSING`) and cross-env misbindings (`MISBOUND`).
+-   `c secrets ls <env>` - the environment's contract-derived GSM inventory.
+-   `c secrets consumers <secretId>` - everything referencing a GSM secret id, with each consuming function's name-parsed env. Gate every deletion on this.
+
+Commands that need 1Password, gcloud or kubectl fail fast with friendly re-authentication instructions when their credentials have expired.
+
+**A note on the 1Password session.** `c op session` stores the session token at `~/.local/state/idearium/op-session` (`0600`), and every `op` invocation the cli makes passes it via `--session` (as does `c op cmd`). That argument is visible in the local process list - the same exposure any `op --session` or `OP_SESSION_*` environment carries. It is machine-local and expires, and secret values themselves never appear in argv: they move via stdin pipes only.
+
 ### MongoDB configuration
 
 The Idearium cli supports a MongoDB configuration. The MongoDB configuration can be used to access local and remote databases.

bin/c-kc-apply.js

@@ -1,4 +1,3 @@
-#!/usr/bin/env -S node
 'use strict';
 
 const program = require('commander');
@@ -18,6 +17,7 @@ const {
     renderServicesTemplates,
     setLocalsForServices,
 } = require('./lib/c-kc');
+const { generateSecretManifests } = require('./lib/c-secrets-manifests');
 
 program
     .arguments('<location>')
@@ -101,32 +101,31 @@ return Promise.all([loadConfig(), loadState()])
                     return reject(e);
                 }
 
-                return resolve([services, path, state]);
+                return resolve([services, path, state, config, namespace]);
             })
     )
-    .then(
-        ([services, path, state]) =>
-            new Promise(async (resolve, reject) => {
-                try {
-                    await ensureServiceFilesExist(path, services);
-                } catch (e) {
-                    reject(e);
-                }
-
-                return resolve([services, path, state]);
-            })
+    .then(([services, path, state, config, namespace]) =>
+        generateSecretManifests({
+            config,
+            env: state.env,
+            namespace,
+            path,
+            services,
+        }).then(() => [services, path, state])
     )
-    .then(
-        ([services, path, state]) =>
-            new Promise(async (resolve, reject) => {
-                try {
-                    await renderServicesTemplates(path, services);
-                } catch (e) {
-                    reject(e);
-                }
-
-                return resolve([services, path, state]);
-            })
+    .then(([services, path, state]) =>
+        ensureServiceFilesExist(path, services).then(() => [
+            services,
+            path,
+            state,
+        ])
+    )
+    .then(([services, path, state]) =>
+        renderServicesTemplates(path, services).then(() => [
+            services,
+            path,
+            state,
+        ])
     )
     .then(
         ([services, path, state]) =>

bin/c-kc-cmd.js

@@ -1,10 +1,14 @@
-#!/usr/bin/env -S node
 'use strict';
 
 const program = require('commander');
 const getPropertyPath = require('get-value');
-const { spawn } = require('child_process');
-const { loadConfig, loadState, reportError } = require('./lib/c');
+
+const {
+    loadConfig,
+    loadState,
+    reportError,
+    spawnWithExitCode,
+} = require('./lib/c');
 const { formatProjectPrefix } = require('./lib/c-project');
 
 program
@@ -26,26 +30,20 @@ return Promise.all([loadState('env'), loadConfig()])
                 `kubernetes.environments.${env}.namespace`
             ) || formatProjectPrefix(organisation, name, env, true, true);
 
-        // Remove the first two entries.
-        const parts = program.rawArgs.slice(2);
-
-        // The command starts with kubectrl.
-        parts.unshift('kubectl');
-
-        // Mandatory namespace and context.
-        parts.push(
-            '--namespace',
-            namespace,
-            '--context',
-            environments[env].context
-        );
-
-        // The rest of the command.
-        const cmd = parts.join(' ');
-
-        spawn(cmd, {
-            shell: true,
-            stdio: 'inherit',
+        // The context and namespace flags precede the user's command:
+        // anything after kubectl's -- would otherwise be executed inside
+        // the pod, letting kubectl fall back to the ambient context and
+        // namespace. The argv is an array (never a shell string), so
+        // nothing is re-parsed or interpolated, and the child's exit code
+        // propagates.
+        return spawnWithExitCode({
+            args: [
+                '--context',
+                environments[env].context,
+                '--namespace',
+                namespace,
+            ].concat(program.rawArgs.slice(2)),
+            command: 'kubectl',
         });
     })
     .catch((err) => reportError(err, false, true));

bin/c-kc-manifests.js

@@ -1,4 +1,3 @@
-#!/usr/bin/env -S node
 'use strict';
 
 const program = require('commander');
@@ -15,6 +14,7 @@ const {
     renderServicesTemplates,
     setLocalsForServices,
 } = require('./lib/c-kc');
+const { generateSecretManifests } = require('./lib/c-secrets-manifests');
 
 program
     .description('This command will compile any found template manifests.')
@@ -41,10 +41,10 @@ return Promise.all([loadState(), loadConfig()])
                 `kubernetes.environments.${state.env}.namespace`
             ) || formatProjectPrefix(organisation, name, state.env, true, true);
 
-        return [locations, prefix, namespace, path, state];
+        return [config, locations, prefix, namespace, path, state];
     })
     .then(
-        ([kubernetesLocations, prefix, namespace, path, state]) =>
+        ([config, kubernetesLocations, prefix, namespace, path, state]) =>
             new Promise((resolve, reject) => {
                 const services =
                     kubernetesLocationsToObjects(kubernetesLocations);
@@ -55,33 +55,22 @@ return Promise.all([loadState(), loadConfig()])
                     return reject(e);
                 }
 
-                return resolve([services, path]);
+                return resolve([config, namespace, path, services, state]);
             })
     )
-    .then(
-        ([services, path]) =>
-            new Promise(async (resolve, reject) => {
-                try {
-                    await ensureServiceFilesExist(path, services);
-                } catch (e) {
-                    reject(e);
-                }
-
-                return resolve([services, path]);
-            })
+    .then(([config, namespace, path, services, state]) =>
+        generateSecretManifests({
+            config,
+            env: state.env,
+            namespace,
+            path,
+            services,
+        }).then(() => [path, services])
     )
-    .then(
-        ([services, path]) =>
-            new Promise(async (resolve, reject) => {
-                try {
-                    await renderServicesTemplates(path, services);
-                } catch (e) {
-                    reject(e);
-                }
-
-                return resolve();
-            })
+    .then(([path, services]) =>
+        ensureServiceFilesExist(path, services).then(() => [path, services])
     )
+    .then(([path, services]) => renderServicesTemplates(path, services))
     .catch((err) => {
         if (err.code === 'ENOENT') {
             return reportError(