Add the inbox and freeagent plugins; inbox indexes into the knowledge base

.github/workflows/ci.yml

@@ -32,7 +32,7 @@ jobs:
   plugins:
     uses: rocketflare-dev/rocketflare/.github/workflows/plugin-ci.yml@main
     with:
-      plugin_subdirs: '["plugins/analytics", "plugins/web-knowledge", "plugins/connectors", "plugins/m365", "plugins/issue-reports"]'
+      plugin_subdirs: '["plugins/analytics", "plugins/web-knowledge", "plugins/connectors", "plugins/m365", "plugins/issue-reports", "plugins/inbox", "plugins/freeagent"]'
       # On a kit-released dispatch, prove against the ref that was just cut — used verbatim, with
       # the range NOT consulted. That is what makes the notification worth sending: resolving the
       # range again would pick the same versions as the last run and prove nothing new. Empty on

README.md

@@ -20,6 +20,8 @@ sees what and its known gaps. They are also published at
 | [**connectors**](plugins/connectors/README.md) | Organisation connections: admin consent once, then directory and calendar sync every 15 minutes; Settings → Connections, My calendar, CLI and the `connectors` setup skill | kit 0.13.0 and a provider |
 | [**m365**](plugins/m365/README.md) | Microsoft 365 as a `connectors` provider: Entra admin consent, app-only Graph delta sync of users, groups and calendars | kit 0.12.0 and `connectors` |
 | [**issue-reports**](plugins/issue-reports/README.md) | Ideas & issues: anyone talks a bug or an idea through with an agent that asks until it is clear, checks for a duplicate, and logs it as a GitHub issue (or a comment) once they approve the draft | kit 0.21.0 and a GitHub token |
+| [**inbox**](plugins/inbox/README.md) | Inbound email through Cloudflare Email Routing: per-organisation address and sender allow-list, SPF/DKIM/DMARC checks, parsed bodies and attachments, Gmail/Proton forwarding-confirmation codes, an Inbox page, CLI, read-only agent tools, and an `inbox:consumers` extension other plugins build on | kit 0.22.0 and Email Routing on the domain |
+| [**freeagent**](plugins/freeagent/README.md) | FreeAgent (UK accounting) per organisation: OAuth connection, a 30-minute incremental mirror of bank transactions, explanations, categories, contacts, bills and invoices, Settings → FreeAgent and a FreeAgent page, read tools plus approval-gated write tools for agents, analytics cubes, CLI, a service API and a `freeagent:sync-listeners` extension for plugins built on top | kit 0.21.0 and a FreeAgent app |
 
 ## Installing one
 

plugins/freeagent/README.md

@@ -0,0 +1,103 @@
+# freeagent
+
+> Connect each organisation's FreeAgent company (UK accounting) and get its books mirrored into
+> your app — for pages, agents, analytics and the plugins you build on top.
+
+An admin approves your FreeAgent app once, over OAuth. Every 30 minutes the plugin syncs the
+company's bank accounts, bank transactions, explanations, categories, contacts, bills and invoices
+into tenant-scoped `freeagent_*` tables, incrementally. Agents read that mirror and can propose an
+explanation or a bill for a person to approve. Other plugins — a receipts matcher, say — explain
+transactions and attach receipts through an exported service API, and hear about every sync through
+the `freeagent:sync-listeners` extension.
+
+## What it adds
+
+| Area | What you get |
+|---|---|
+| Connection | One FreeAgent company per organisation. OAuth approval through `POST /api/freeagent/connect` and the public callback `/api/hooks/freeagent/callback` (signed state). Refresh and access tokens stored sealed. A refused refresh token marks the connection **needs reconnecting** |
+| Sync | `freeagent.sync`, queued every 30 minutes per connected organisation, after connecting, and by **Sync now**. Incremental by `updated_since`, paged, across every bank account. The first sync reaches back about 400 days for transactions |
+| Client | One typed wrapper: always `X-Api-Version: 2026-09-01`, token refresh 2 minutes before expiry (single-flight), bounded back-off on 429 `Retry-After`, `Link` pagination, typed `FreeAgentError`, and the token never sent off the FreeAgent API host |
+| UI | **Settings → FreeAgent**: connect, reconnect, disconnect, environment, company, last sync, errors, Sync now. **FreeAgent** page: unexplained transactions and invoices (overdue filter) |
+| Agent tools | Read: `freeagent_company`, `freeagent_search_transactions`, `freeagent_list_categories`, `freeagent_list_invoices`, `freeagent_list_bills`. Write, approval required, admin requesters only: `freeagent_explain_transaction`, `freeagent_create_bill`. None offered until the organisation is connected |
+| Service API | `freeagentClientFor`, `explainTransaction`, `attachToExplanation`, `deleteExplanation`, `createBill`, `listUnexplainedTransactions` — each write updates the mirror row it touched |
+| Extension | `freeagent:sync-listeners`: a plugin contributes `{ id, onSync(summary, ctx) }`, called after every successful sync |
+| Analytics | When the `analytics` plugin is installed: cubes `FreeAgentTransactions`, `FreeAgentExplanations` (+ `FreeAgentCategories`), `FreeAgentInvoices`, `FreeAgentBills` — spend and income by month and category, unexplained count and amount, outstanding and overdue. Inert without analytics |
+| Data | `freeagent_connections`, `freeagent_sync_state` and seven mirror tables, all with row-level security; money as `numeric` |
+| CLI | `rocketflare freeagent status` and `rocketflare freeagent sync` |
+| Skill | `/freeagent`: registering the app (sandbox and production), secrets, connecting, verifying, troubleshooting |
+
+## Requirements
+
+- Rocketflare kit **0.21.0** or later.
+- A FreeAgent app registered at [dev.freeagent.com](https://dev.freeagent.com) with the redirect
+  URI `{APP_URL}/api/hooks/freeagent/callback`.
+- `OAUTH_ENCRYPTION_KEY` (the kit's bootstrap sets it) and the kit's `JOBS_QUEUE`.
+
+## Install
+
+```bash
+pnpm plugin add https://github.com/rocketflare-dev/rocketflare-plugins.git@<version> --subdir plugins/freeagent
+pnpm plugin add https://github.com/rocketflare-dev/rocketflare-plugins.git@<version> --subdir plugins/freeagent --apply
+pnpm db:generate --name plugin-freeagent-<version> && pnpm db:migrate
+```
+
+Then set the three keys — `FREEAGENT_CLIENT_ID`, `FREEAGENT_CLIENT_SECRET`, `FREEAGENT_ENVIRONMENT`
+(`sandbox` | `production`, unset means `sandbox`) — in `apps/web/.dev.vars` locally and with
+`wrangler secret put` deployed. All three are declared as secrets so no toml edit is needed; the
+environment is not sensitive and may go under `[vars]` in both tomls instead. `pnpm provision
+cloudflare <env>` writes the `*/30 * * * *` cron. Run `/freeagent` for the walk-through.
+
+## Building on it
+
+```ts
+import { freeagentExtensions } from '@rocketflare/shared/plugins/freeagent/index'
+import {
+  attachToExplanation,
+  explainTransaction,
+  type FreeAgentSyncListener,
+  listUnexplainedTransactions,
+} from '@/plugins/freeagent'
+
+const rematch: FreeAgentSyncListener = {
+  id: 'receipts',
+  async onSync(summary, ctx) {
+    await ctx.enqueue({ type: 'receipts.match', payload: { tenantId: summary.tenantId } })
+  },
+}
+
+// in a job:
+const open = await listUnexplainedTransactions(ctx, tenantId, { from: '2026-04-06' })
+const explanation = await explainTransaction(ctx, tenantId, {
+  bankTransactionUrl: open[0].url,
+  categoryUrl, // from freeagent_categories
+  grossValue: open[0].unexplainedAmount, // signed like the transaction
+  datedOn: open[0].datedOn,
+  description: 'Train ticket',
+})
+await attachToExplanation(ctx, tenantId, explanation.url, {
+  bytes, filename: 'receipt.pdf', contentType: 'application/pdf',
+})
+
+export const receiptsServer = {
+  // …
+  extensions: freeagentExtensions({ syncListeners: [rematch] }),
+}
+```
+
+Declare `requires: { plugins: ["freeagent"] }` in your `plugin.json`. Writes throw `FreeAgentError`;
+branch on `err.code` (`needs_reconnect`, `rate_limited`, `validation`, …).
+
+## Who sees what
+
+Every member can see the FreeAgent page and the mirrored books, and their agent runs get the read
+tools. Owners and admins connect, disconnect and sync, and only their runs are offered the two write
+tools — each call still waits for a person to approve it.
+
+## Known gaps
+
+- One FreeAgent company per organisation; no accountancy-practice (multi-client) API.
+- Polling only (FreeAgent has no webhooks): changes appear within 30 minutes, or on Sync now.
+- The first sync mirrors about 400 days of transactions and explanations; older history is not fetched.
+- A deleted contact, bill or invoice stays in the mirror (FreeAgent's `updated_since` does not report deletions). Explanations deleted in FreeAgent are reconciled through their transaction.
+- No per-user rate-limit budgeting beyond the bounded 429 back-off; a very large first sync may span several queue retries.
+- Disconnecting deletes the mirror as well as the tokens.

plugins/freeagent/apps/cli/src/plugins/freeagent/index.ts

@@ -0,0 +1,59 @@
+/**
+ * `rocketflare freeagent …` — the plugin's CLI half.
+ *
+ *   rocketflare freeagent status [--json]   GET  /api/freeagent/status
+ *   rocketflare freeagent sync [--json]     POST /api/freeagent/sync   (queues a sync; admin)
+ */
+import {
+  FREEAGENT_ID,
+  freeagentShared,
+  freeagentStatusSchema,
+  freeagentSyncResponseSchema,
+} from '@rocketflare/shared/plugins/freeagent/index'
+import type { CliPlugin, CommandContext } from '../api'
+import { formatDate, requireClient } from '../api'
+
+export async function runFreeagentStatus(ctx: CommandContext): Promise<void> {
+  const { data, raw } = await requireClient(ctx).request('GET', '/api/freeagent/status', {
+    schema: freeagentStatusSchema,
+  })
+  const connection = data.connected
+    ? 'connected'
+    : data.status === 'needs_reconnect'
+      ? 'needs reconnecting (Settings → FreeAgent)'
+      : 'not connected'
+  ctx.out.data(raw, () =>
+    [
+      `FreeAgent:     ${data.configured ? `configured (${data.environment})` : 'not configured on this deployment'}`,
+      `Connection:    ${connection}`,
+      `Company:       ${data.companyName ?? '-'}${data.companyCurrency ? ` (${data.companyCurrency})` : ''}`,
+      `Last sync:     ${formatDate(data.lastSyncAt)}`,
+      `Mirrored:      ${data.counts.transactions} transactions (${data.counts.unexplained} unexplained), ${data.counts.invoices} invoices, ${data.counts.bills} bills`,
+      ...(data.lastError
+        ? [`Last error:    ${data.lastError} (${formatDate(data.lastErrorAt)})`]
+        : []),
+    ].join('\n')
+  )
+}
+
+export async function runFreeagentSync(ctx: CommandContext): Promise<void> {
+  const { data, raw } = await requireClient(ctx).request('POST', '/api/freeagent/sync', {
+    schema: freeagentSyncResponseSchema,
+  })
+  ctx.out.data(raw, () => `Sync queued (job ${data.jobId}).`)
+}
+
+export const freeagentCli: CliPlugin<typeof freeagentShared> = {
+  shared: freeagentShared,
+  register(program, action) {
+    const root = program.command(FREEAGENT_ID).description('FreeAgent accounting (a plugin)')
+    root
+      .command('status', { isDefault: true })
+      .description('show the connection, company and last sync')
+      .action(action(ctx => runFreeagentStatus(ctx)))
+    root
+      .command('sync')
+      .description('queue a sync of the FreeAgent books now')
+      .action(action(ctx => runFreeagentSync(ctx)))
+  },
+}

plugins/freeagent/apps/web/src/plugins/freeagent/CLAUDE.md

@@ -0,0 +1,25 @@
+# freeagent plugin — rules for this directory
+
+An organisation's FreeAgent books: OAuth (`api/routes.ts` connect → FreeAgent → `api/hooks.ts`
+callback) → sealed tokens on `freeagent_connections` → `freeagent.sync` (cron `*/30`, "Sync now",
+the callback) → `services/sync.ts` → the `freeagent_*` mirror → agent tools, the page, cubes, and
+every `freeagent:sync-listeners` extension.
+
+- **Every FreeAgent call goes through `FreeAgentClient`** (`services/client.ts`): it sends
+  `X-Api-Version: 2026-09-01`, refreshes tokens, waits out 429s (bounded), follows `Link`
+  pagination and refuses any URL off the connection's API host. Never `fetch` FreeAgent directly.
+- **`fetch` is injected.** Services take `deps` (default `freeagentHttp`); tests pass a fake, and
+  route tests swap `freeagentHttp.fetch` for one test. No test reaches FreeAgent.
+- **Tokens are sealed** (`sealSecret`) in `*_enc` columns, never returned, never logged, nor are
+  approval codes. Log a `FreeAgentError.code`, not a body.
+- **`invalid_grant` → `needs_reconnect`**, recorded on the row; the cron skips it, tools vanish.
+- **Money is a decimal string** end to end (`numeric` columns, zod `decimalStringSchema`). Never
+  `Number()` an amount.
+- **Mirror rows are keyed `(tenant_id, url)`** and written only through `services/mirror.ts`
+  (`upsertMirror`), by the sync and by every write — a write updates the row it touched.
+- **The one cross-tenant read** is the cron fan-out in `api/scheduled.ts` (tenant ids only). It
+  needs no `unscopedAllowlist` entry: the kit scan sees the `tenantId` it selects.
+- **Cubes are plain data** contributed under `analytics:*` keys (`cubes/index.ts`): no import of
+  analytics or drizzle-cube, inert without it. A new cube needs an isolation case beside it.
+- **Listeners** (`FreeAgentSyncListener`) are called after a successful sync; a throw is logged and
+  does not fail the sync. They must be quick and idempotent.

plugins/freeagent/apps/web/src/plugins/freeagent/api/hooks.ts

@@ -0,0 +1,108 @@
+/**
+ * `/api/hooks/freeagent` — this plugin's PUBLIC mount (no `authMiddleware`).
+ *
+ *   GET /callback   FreeAgent's approval redirect lands here with `code` and `state`
+ *
+ * What makes an unauthenticated route safe is that it trusts nothing the request says except what
+ * it can VERIFY: `state`, signed by `POST /api/freeagent/connect` for one tenant and one admin, for
+ * this purpose only, expiring in fifteen minutes (FreeAgent's own code lifetime). A missing,
+ * tampered or expired state is a 401 and nothing else — no hint which part failed. Only after that
+ * does the handler know which organisation it is acting for.
+ *
+ * Then: exchange the code (Basic auth with the operator's app), read `GET /v2/company` with the
+ * new token, store both tokens SEALED, enqueue the first sync and send the admin back to
+ * Settings → FreeAgent. The code and the tokens are never logged; a failure is logged by code.
+ */
+import {
+  FREEAGENT_CONNECTION_ENTITY,
+  FREEAGENT_SUBJECT,
+  FREEAGENT_SYNC_JOB,
+} from '@rocketflare/shared/plugins/freeagent/index'
+import { z } from 'zod'
+import type { PublicCtx } from '@/plugins/api'
+import { createRouter, publicCtx, recordActivity, verifyState } from '@/plugins/api'
+import { FreeAgentClient } from '../services/client'
+import { saveConnection } from '../services/connection'
+import {
+  appCredentials,
+  FREEAGENT_HOSTS,
+  freeagentConfig,
+  freeagentHttp,
+  redirectUri,
+  settingsReturnUrl,
+} from '../services/env'
+import { FreeAgentError } from '../services/errors'
+import { mapCompany } from '../services/mapping'
+import { exchangeCode } from '../services/tokens'
+
+export const CONNECT_STATE_PURPOSE = 'freeagent.connect'
+/** FreeAgent's approval code lives 15 minutes; the state need not outlive it. */
+export const CONNECT_STATE_TTL_SECONDS = 15 * 60
+
+export const connectStateSchema = z.object({
+  tenantId: z.string().uuid(),
+  userId: z.string().uuid(),
+})
+
+export const freeagentHooksRouter = createRouter()
+
+freeagentHooksRouter.get('/callback', async c => {
+  const ctx: PublicCtx = publicCtx(c)
+  const query = new URL(c.req.url).searchParams
+  // Parsed even though it is signed: a signature proves who wrote it, not what shape it has.
+  const state = connectStateSchema.safeParse(
+    await verifyState(ctx.config, CONNECT_STATE_PURPOSE, query.get('state') ?? '')
+  )
+  if (!state.success) ctx.unauthorized('Invalid or expired FreeAgent approval')
+  const { tenantId, userId } = state.data
+  const back = (outcome: { connected: true } | { error: string }) =>
+    c.redirect(settingsReturnUrl(ctx.appUrl, outcome), 302)
+
+  // The admin clicked "Deny" (or FreeAgent refused): FreeAgent sends `error`, no code.
+  const denied = query.get('error')
+  if (denied)
+    return back({ error: denied === 'access_denied' ? 'access_denied' : 'approval_failed' })
+  const code = query.get('code')
+  if (!code) return back({ error: 'missing_code' })
+  const credentials = appCredentials(ctx.config)
+  if (!credentials) return back({ error: 'not_configured' })
+
+  const environment = freeagentConfig(ctx.config).FREEAGENT_ENVIRONMENT
+  const deps = freeagentHttp
+  let connectionId: string
+  let companyName: string | null
+  try {
+    const tokens = await exchangeCode(
+      environment,
+      credentials,
+      { code, redirectUri: redirectUri(ctx.appUrl) },
+      deps
+    )
+    const client = new FreeAgentClient({
+      baseUrl: FREEAGENT_HOSTS[environment],
+      accessToken: async () => tokens.accessToken,
+      deps,
+    })
+    const company = mapCompany(await client.get('/v2/company'))
+    companyName = company.companyName
+    const row = await saveConnection(ctx, { tenantId, userId, environment, tokens, company })
+    connectionId = row.id
+  } catch (err) {
+    const codeName = err instanceof FreeAgentError ? err.code : 'internal'
+    ctx.logger.warn({ tenantId, code: codeName }, 'freeagent: completing the connection failed')
+    return back({ error: codeName === 'invalid_grant' ? 'code_expired' : 'exchange_failed' })
+  }
+
+  await ctx.enqueue({ type: FREEAGENT_SYNC_JOB, payload: { tenantId, requestedBy: userId } })
+  // Awaited rather than deferred: both are one cheap write, and the admin is waiting on a redirect.
+  await recordActivity(ctx.db, {
+    tenantId,
+    userId,
+    type: 'freeagent.connected',
+    subjectType: FREEAGENT_SUBJECT,
+    subjectId: connectionId,
+    metadata: { companyName, environment },
+  }).catch(() => {})
+  await ctx.nudge(tenantId, FREEAGENT_CONNECTION_ENTITY).catch(() => {})
+  return back({ connected: true })
+})