Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,26 @@ the kit's move, copy them again rather than editing them in place.

## Releasing

`node scripts/release.mjs X.Y.Z` — the kit's own release script. It detects a
`rocketflare-plugin.json` with no `.rocketflare.json`, stamps that manifest's version and
`package.json`, folds `docs/upgrades/unreleased.md` into `docs/upgrades/X.Y.Z.md` and prepends the
`CHANGELOG.md` section.
Every plugin ships at one version, so a release covers all of `plugins/*`, on a branch:

1. Give every plugin a non-empty `docs/upgrades/unreleased.md`. A plugin with no change of its own
gets a "version stamps only" note (`plugins/<id>/docs/upgrades/3.7.0.md` is the pattern);
`release.mjs` refuses an empty one.
2. Run the kit's release script with every plugin named. Without `--plugin` it finds no manifest at
the root and exits 1:

```bash
args=(); for p in plugins/*/; do args+=(--plugin "${p%/}"); done
node scripts/release.mjs X.Y.Z "${args[@]}"
```

It folds each `unreleased.md` into `X.Y.Z.md`, stamps every `rocketflare-plugin.json`, its
anchor `plugin.json` and the root `package.json`, and prepends the `CHANGELOG.md` section.
3. Move the `rocketflare-plugins.git@<previous>` install refs in `README.md` and
`plugins/*/README.md` to `@X.Y.Z`; the script does not touch them.
4. `node scripts/release-check.mjs --tag X.Y.Z` must report `release-check ok`.
5. Commit `Release X.Y.Z`, open a PR, and once it merges tag the merge commit `X.Y.Z` and push the
tag.

**Released history is never rewritten.** Every app pins a commit of this repository; a force-push
to a released tag orphans them.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ analyticsPagesRouter.post('/templates/recreate', async c => {
analyticsPagesRouter.get('/facts/status', async c => {
const ctx: RequestCtx = requestCtx(c)
guardFactTables(ctx, 'read fact-table status')
return c.json({ items: await checkFactTableFreshness(ctx.db) })
return c.json({ items: await checkFactTableFreshness(ctx.db, ctx.tenantId) })
})

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ read by the `TenantActivityDaily` cube.
`DELETE … WHERE tenant_id = $1` then `INSERT INTO t (<columns from getTableColumns>) <select>`;
errors isolated per tenant. `refreshAllFactTables(db)` = every table. Full rebuild, not
incremental; `fact_refreshed_at` is a stamp.
- `freshness.ts` — `checkFactTableFreshness(db)`: `lagSeconds` = newest source row minus last
build (0 if the build is newer); `stale` = lag > 2× interval.
- `freshness.ts` — `checkFactTableFreshness(db, tenantId)`: one tenant's newest source row minus
its last build (0 if the build is newer) is `lagSeconds`; `stale` = lag > 2× interval. Both
`MAX()`s filter on `tenant_id`, so every `source.table` must carry it.
- Raw `execute` results go through `../sql-result.ts` (`rowsOf`, `affectedRows`), never indexed
or `.count`-read directly: the kit's driver is postgres.js (a `RowList` array with `.count`) up
to 0.14.x and the Neon serverless driver (`{ rows, rowCount }`) from 0.15.0, and both must work.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
/**
* Fact-table freshness (D19): for each registry entry compare the newest source row
* (`MAX(source.timestampColumn)`) with the newest rebuild (`MAX(fact_refreshed_at)`). `lagSeconds`
* Fact-table freshness (D19): for each registry entry compare one tenant's newest source row
* (`MAX(source.timestampColumn)`) with its newest rebuild (`MAX(fact_refreshed_at)`), both
* filtered on `tenant_id` — never across tenants, which would show one organisation another's
* activity and let a busy tenant hide a stale one. `lagSeconds`
* is how long the newest source row has waited for a refresh — 0 when the fact table is newer than
* the source; `stale` when that exceeds 2× the table's refresh interval (one missed cron is fine,
* two is not). A table never built while the source has rows is measured against `now`.
* Read by `GET /api/analytics/facts/status` (admin+) and `scripts/check-fact-table-freshness.ts`.
* Read by `GET /api/analytics/facts/status` (admin+) and so `rocketflare analytics check-facts`.
*/
import type { FactTableStatus } from '@rocketflare/shared/plugins/analytics/index'
import { sql } from 'drizzle-orm'
Expand Down Expand Up @@ -51,13 +53,17 @@ export function computeFreshness(

export async function checkFactTableFreshness(
db: Database,
tenantId: string,
now: Date = new Date()
): Promise<FactTableStatus[]> {
const out: FactTableStatus[] = []
const ofTenant = sql`where tenant_id = ${tenantId}`
for (const def of factTables()) {
const { table, source } = def
const newest = source.timestampColumn
const [refreshedAt, sourceMaxAt] = await Promise.all([
maxOf(db, sql`select max(fact_refreshed_at) as max_ts from ${def.table}`),
maxOf(db, sql`select max(${def.source.timestampColumn}) as max_ts from ${def.source.table}`),
maxOf(db, sql`select max(fact_refreshed_at) as max_ts from ${table} ${ofTenant}`),
maxOf(db, sql`select max(${newest}) as max_ts from ${source.table} ${ofTenant}`),
])
out.push(computeFreshness(def, refreshedAt, sourceMaxAt, now))
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
* Fact-table registry (D19, D31) — the ONE list that `refresh.ts`, `freshness.ts` and the `:15`
* cron task (`api/scheduled.ts`) iterate. Adding a fact table = a schema file under
* `db/schema/facts/`, a `queries/<name>.ts` SELECT builder, and one entry here. Every table
* carries `tenant_id` (rebuilt per tenant) and `fact_refreshed_at` (the freshness watermark).
* carries `tenant_id` (rebuilt per tenant) and `fact_refreshed_at` (the freshness watermark), and
* every `source.table` carries `tenant_id` too: freshness filters both on it.
*
*/
import type { SQL } from 'drizzle-orm'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
} from '@testkit/integration'
import { eq } from 'drizzle-orm'
import { describe, expect, it } from 'vitest'
import { activityEvents } from '@/db/schema/kit'
import { analyticsPages } from '../../db/schema'

const db = setupTestDatabase()
Expand Down Expand Up @@ -263,4 +264,19 @@ describe('templates and fact status', () => {
})
expect(denied.status).toBe(403)
})

it("GET /facts/status reports the caller's organisation, not another one's backlog", async () => {
const o = await owner()
const other = await createTestTenantWithUser(db, 'owner')
// Another organisation has a source row five hours old that was never built.
await db.insert(activityEvents).values({
tenantId: other.tenant.id,
userId: other.user.id,
type: 'other',
createdAt: new Date(Date.now() - 5 * 60 * 60 * 1000),
})
const res = await request('/api/analytics/facts/status', { headers: o.cookie })
const body = await json<{ items: Array<{ lagSeconds: number; stale: boolean }> }>(res)
expect(body.items[0]).toMatchObject({ lagSeconds: 0, stale: false })
})
})
Original file line number Diff line number Diff line change
Expand Up @@ -180,11 +180,36 @@ describe('freshness', () => {
const { user, tenant } = await createTestTenantWithUser(db, 'owner')
await db.insert(activityEvents).values({ tenantId: tenant.id, userId: user.id, type: 'f' })
await refreshFactTable(db, TABLE, { tenantId: tenant.id })
const statuses = await checkFactTableFreshness(db)
const statuses = await checkFactTableFreshness(db, tenant.id)
expect(statuses.map(s => s.table)).toEqual(factTables().map(t => t.name))
const s = statuses[0]
expect(s?.refreshedAt).toBeInstanceOf(Date)
expect(typeof s?.lagSeconds).toBe('number')
expect(typeof s?.stale).toBe('boolean')
})

it("checkFactTableFreshness sees only the caller's tenant", async () => {
const a = await createTestTenantWithUser(db, 'owner')
const b = await createTestTenantWithUser(db, 'owner')
await db.insert(activityEvents).values({
tenantId: a.tenant.id,
userId: a.user.id,
type: 'a',
createdAt: new Date(Date.now() - DAY),
})
await refreshFactTable(db, TABLE, { tenantId: a.tenant.id })
const [aRow] = await factRows(a.tenant.id)
// B's newest source row is later than A's build, and B is rebuilt after A.
await db.insert(activityEvents).values({
tenantId: b.tenant.id,
userId: b.user.id,
type: 'b',
createdAt: new Date(Date.now() + 60 * 60 * 1000),
})
await refreshFactTable(db, TABLE, { tenantId: b.tenant.id })

const [status] = await checkFactTableFreshness(db, a.tenant.id)
expect(status?.refreshedAt?.getTime()).toBe(aRow?.factRefreshedAt.getTime())
expect(status).toMatchObject({ lagSeconds: 0, stale: false })
})
})
20 changes: 17 additions & 3 deletions plugins/analytics/docs/upgrades/unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,33 @@ previous: 3.8.0
date: null
breaking: false
migrations: []
areas: []
areas: [api, tests]
touches_surfaces: []
requires_surfaces: []
manual: false
---

## What changed

_Nothing yet. Add an entry here in the same pull request as the change — see `README.md` beside this
file for the fields and for what "How to apply" has to say._
`GET /api/analytics/facts/status` reports only the caller's organisation. It used to read `MAX()` across every tenant, so an organisation's admin saw when any other organisation last had activity, and a busy tenant could hide another's stale fact table.

- `services/fact-tables/freshness.ts`: `checkFactTableFreshness(db, tenantId, now?)` filters both the fact table and its source on `tenant_id`.
- `api/routes/analytics-pages.ts`: the route passes `ctx.tenantId`.
- `services/fact-tables/registry.ts`: a fact table's `source.table` must carry `tenant_id`, as the fact table already does.
- `tests/api/`: freshness and the route are each tested against a second organisation's newer activity.
- `rocketflare analytics check-facts` reads the same route, so it now reports the signed-in organisation only.

## How to apply

1. Run `pnpm plugin upgrade analytics --apply`. It replaces the three source files, the two test files and `services/fact-tables/CLAUDE.md`.
2. If app code calls `checkFactTableFreshness` itself, pass the tenant id as the second argument.
3. If the app contributes a fact table through `analyticsExtensions({ factTables })`, check its `source.table` has a `tenant_id` column. Without one, `/facts/status` fails with `column "tenant_id" does not exist`.

## Conflicts to expect

None.

## Verify

1. `pnpm gate` passes, including `tests/api/fact-table-refresh.test.ts` and `tests/api/analytics-pages.test.ts`.
2. As one organisation's admin, `GET /api/analytics/facts/status` reports no lag when only another organisation has unbuilt activity.
Loading