diff --git a/README.md b/README.md index cbe663d5..b3f3a9fa 100644 --- a/README.md +++ b/README.md @@ -148,7 +148,9 @@ The tab has five views and a page per project: turns can be changed in place or with `PATCH /api/v1/settings`. A team's lead can take just their team's rows, and copy them as text for a meeting's notes. It weighs every project, whatever the filter bar narrows the - other views to. The tab's label counts what's left. + other views to. The tab's label counts what's left. With automatic + calls on (below), it leaves out a reason only when Pulldasher makes a + call that answers it, and asks the rest as before. - **Roadmap**: the plan set against everything actually in flight. Across the top, how many projects were in flight each week (from their PRs) and, for the weeks ahead, the plans plus every project big enough to owe @@ -208,6 +210,69 @@ The tab has five views and a page per project: row opens the PRs that took the most days and who spent them. It all copies as text for the retro's notes. +Pulldasher makes the roadmap calls that only say what already happened, so +the roadmap follows the work without anyone answering Decide. After each +sync of the projects' work (a webhook on a project's issue, or the hourly +pass), it: + +- picks a plan up again: a parked plan whose project gets a PR opened, or + an open PR a person labeled into it, after it was parked goes back in progress + ("Picked up again by owner/repo#12 on 2026-10-20"). A done or dropped + plan takes 2 such PRs, since one fix after a finish isn't a return. PRs + that were already open when someone marked it done don't count, and + neither does an old PR the labeling job files later: a job's PR counts + by when it opened, not when its label went on. A plan whose project + issue closed after the plan last changed isn't picked up: Decide asks + about that. +- puts a plan down: a plan in progress is parked when, for `idleDays` (28), + its project had no PR opened, merged or pushed to, nobody posted an + update, and no person changed it. +- finishes a plan: one under way whose issue closed, or all of whose issues + closed, with no PRs open, is done, or dropped when they were closed as not + planned. One past its hard end with none open, its status untouched + since, is done. +- plans a project: one with 3 or more PRs open or merged in the last 14 + days that never had a plan gets one, in progress from its first open + PR's week (or its first PR's, when all have merged) with no end, named + by its issue or its label. A burst that already merged counts too, and + put down parks it once it goes quiet. A plan someone removed isn't made + again, and Not tracked, ongoing and closed projects get none. Decide's + own question about a new project still needs a PR open. +- names a lead: a plan planned or in progress with no lead, whose status + it isn't changing in the same pass, gets its project issue's assignee, + else the author with the most PRs in its project over the last 60 days. + A lead someone set is never replaced, and a plan a person changed since + its project's PRs last moved keeps an empty lead, since they may have + cleared it. + +Each call is a roadmap write like a person's, recorded as `pulldasher`, +with why in the plan's `auto_note` and when in `auto_at` (a lead fills in +without counting as a call). Decide lists the week's under "Changed on its +own". A person's change always wins: the rules only read what happened +after a plan's status last changed, and a call that finds its plan's +status changed since it read it doesn't land. Decide, the API's and the +board's, works out the calls Pulldasher makes now and leaves out only +the reasons those answer (`pending_calls` on `/api/v1/decide`). A plan +being finished is asked nothing more. + +The labeling job files PRs into projects and re-sorts its own filings at +each sweep, so a fresh label of its own is still a guess. A PR the job +labeled (a bot's label, or one inside a `machineLabelRuns` window) counts +toward picking a plan up or making a new one only once that label is +`settleDays` old (3 by default). A PR a person labeled, or one that joins +by an issue link, counts at once. + +It's off until `autoCalls: true` is set in config.js's `projects`, and +while it's off Decide asks those calls. To turn it on: + +1. Deploy. +2. On the server, run `bin/auto-calls` to see what it would do now. It + writes nothing. +3. Run `bin/auto-calls --at ` to see the plans + that will be put down once a burst of new labels goes quiet. It runs + the rules as of that day on today's PRs and plans. +4. Set `autoCalls: true`. + To put the plans where people already look, `bin/sync-issue-fields` copies each plan under way into its project issue's Start date, Target date and Priority fields: the plan's first Monday and last day, and Urgent for a @@ -225,8 +290,12 @@ bin/sync-issue-fields --apply # write it Everything the tab shows, and every change the roadmap takes, is in the API, Bearer-authed with the caller's own GitHub token like `/api/v1/pulls`, so a script or a Claude session can read the projects and run the roadmap -without a browser. `GET /api/v1` lists every route with what it does, and -the rules a roadmap write is checked against: +without a browser. A job reads it with a read key instead: the labeling job +sends one as its Bearer value. config.js's `apiReadKeys` lists the SHA-256 +hex digest of each key, never the key itself (`printf %s "$KEY" | sha256sum`). +A read key is let in for `GET` only, and any write with one is refused, since +it has no login to record. `GET /api/v1` lists every route with what it does, +and the rules a roadmap write is checked against: ```sh curl -s -H "Authorization: Bearer $(gh auth token)" https://pulldasher.example.com/api/v1 @@ -235,7 +304,15 @@ curl -s -H "Authorization: Bearer $(gh auth token)" https://pulldasher.example.c - `GET /api/v1/projects` and `GET /api/v1/people`: each project and each person with where they stand today and their numbers for a window. Both take `start` and `end` as `YYYY-MM-DD` days (default: the last 30 days, at - most 400), and `project=` narrows the numbers to one project. + most 400), and `project=` narrows the numbers to one project. Each + project also says what the labeling job reads: `status`, its plan's + status (null with none), `last_activity`, the team's day + (`projects.timeZone`) its newest PR opened, merged or was pushed to, the + same activity a plan is put down by, and `examples`, up to 5 of its labeled PRs (`repo`, `number`, + `title`, `merged_at`, `by`), open or merged, the ones a person labeled (`by: "person"`) + before the job's (`"job"`), newest first. A bot's label, or one a script + added with a person's token inside a `projects.machineLabelRuns` window, + is the job's. - `GET /api/v1/decide`: the calls owed now, worst first, each with its project, its roadmap item if it has one, and its reasons. `GET /api/v1` says what each reason means and which write clears it. It also names who diff --git a/app.js b/app.js index 79f66035..471b1d7f 100644 --- a/app.js +++ b/app.js @@ -24,6 +24,7 @@ import { syncRepoIssues } from './lib/project-issue-sync.js'; import { loadSettingsWithRetry } from './lib/settings.js'; import { issueRepos, projectSettings } from './lib/projects.js'; import { setOnWorkSynced, syncWork } from './lib/work.js'; +import { runAutoCalls } from './lib/auto-calls.js'; import { missingMigrations } from './lib/schema-check.js'; import apiAuth from './lib/api-auth.js'; import Debug from './lib/debug.js'; @@ -116,7 +117,15 @@ app.use(PROJECT_WRITES, function (req, res, next) { } next(); }); -setOnWorkSynced(projectsChanged); +// once the work is read again, Pulldasher makes the roadmap calls that only +// say what happened (lib/auto-calls.js), then open boards fetch again +setOnWorkSynced(function () { + runAutoCalls(projectSettings()) + .catch(function (err) { + console.error('Roadmap calls failed: %s', (err && err.message) || err); + }) + .finally(projectsChanged); +}); app.post('/project-issues', canWrite, projectsController.attachIssue); app.delete('/project-issues', canWrite, projectsController.detachIssue); // the roadmap is the one part of the Projects tab people edit here: reads are diff --git a/bin/auto-calls b/bin/auto-calls new file mode 100755 index 00000000..085e85f1 --- /dev/null +++ b/bin/auto-calls @@ -0,0 +1,37 @@ +#!/usr/bin/env node +/** + * What the roadmap calls Pulldasher makes on its own after each sync of the + * projects' work (lib/auto-calls.js) would be now: a plan picked up again, + * put down, finished, planned or given a lead. It only says so and writes + * nothing, whether or not config.js projects.autoCalls is on: the server is + * the one writer, so two processes never make the same call. With --at, it + * runs the rules as of that day (lib/auto-calls.js asOf), on today's PRs + * and plans: what nothing new would bring, such as the put-downs idleDays + * after a burst of labels. + * + * Usage: bin/auto-calls [--at YYYY-MM-DD] + */ +import db from '../lib/db.js'; +import dbManager from '../lib/db-manager.js'; +import pullManager from '../lib/pull-manager.js'; +import { asOf, describe, runAutoCalls } from '../lib/auto-calls.js'; +import { projectSettings } from '../lib/projects.js'; + +async function main() { + const settings = projectSettings(); + if (!settings) throw new Error('this Pulldasher has no projects config'); + const now = asOf(process.argv); + // the board's pulls, as the server holds them: what's open now is read off them + const pulls = await dbManager.getRecentPulls(pullManager.getOldestAllowedPullTimestamp()); + for (const pull of pulls) pullManager.updatePull(pull); + const calls = await runAutoCalls(settings, { apply: false, now }); + for (const call of calls) console.log(describe(call)); + console.log(`${calls.length} call(s). Nothing was written.`); +} + +main() + .catch(err => { + console.error(err); + process.exitCode = 1; + }) + .finally(() => db.end()); diff --git a/config.example.js b/config.example.js index d0e62066..1b81c14f 100644 --- a/config.example.js +++ b/config.example.js @@ -67,6 +67,12 @@ module.exports = { secret: 'secret for signing session cookies', }, + // Read-only keys for /api/v1, for a job with no person's token (the + // labeling job sends one as its Bearer value): the SHA-256 hex digest of + // each key, never the key (printf %s "$KEY" | sha256sum). A key reads + // (GET) and never writes. + apiReadKeys: [], + // When true (default), GitHub PR reviews with state APPROVED count as CR // signoffs without requiring a CR :emoji: tag in the review body. useGithubApprovalForCr: true, @@ -145,6 +151,36 @@ module.exports = { // leaves Decide, the counts and capacity. One PR in another repo ends // it. Omit or [] for none. // untrackedRepos: ['owner/ops'], // or just 'ops' + // Pulldasher makes the roadmap calls that only say what already + // happened: a plan picked up again by a new PR, put down after + // idleDays with no PR opened, merged or pushed, finished when its + // issues close or its hard end passes with nothing open, a plan for a + // project with 3 or more PRs and none, and a lead for a plan with + // none (its issue's assignee, else its busiest author lately). Each + // is recorded as 'pulldasher' with why, and a person's change always + // wins. Off unless true: deploy, run bin/auto-calls on the server to + // see what it would do, then set it true. Off, Decide asks them. + autoCalls: false, + idleDays: 28, + // Days a PR the labeling job filed waits before it counts toward + // picking a plan up or making a new one: the job re-sorts its own + // filings, so a fresh one is still a guess. A person's label counts + // at once. + settleDays: 3, + // When a script added project labels with a person's token: a label + // `actor` added between `from` and `to` (ISO times, UTC) counts as the + // labeling job's, not theirs: in /api/v1/projects `examples`, and it + // waits `settleDays` before counting toward an automatic call. + machineLabelRuns: [ + { actor: 'jarstelfox', from: '2026-09-30T06:57:00Z', to: '2026-09-30T07:00:00Z' }, + { actor: 'jarstelfox', from: '2026-10-10T07:55:00Z', to: '2026-10-10T08:05:00Z' }, + { actor: 'jarstelfox', from: '2026-10-10T18:14:00Z', to: '2026-10-10T18:51:00Z' }, + { actor: 'jarstelfox', from: '2026-10-10T19:28:00Z', to: '2026-10-10T19:41:00Z' }, + { actor: 'jarstelfox', from: '2026-10-10T22:50:00Z', to: '2026-10-11T00:11:00Z' }, + { actor: 'jarstelfox', from: '2026-10-11T02:35:00Z', to: '2026-10-11T03:10:00Z' }, + { actor: 'jarstelfox', from: '2026-10-13T22:23:00Z', to: '2026-10-13T22:59:00Z' }, + { actor: 'jarstelfox', from: '2026-10-13T23:32:00Z', to: '2026-10-13T23:42:00Z' }, + ], }, // The usual MySQL stuff. Like every other MySQL webapp, basically. diff --git a/controllers/api-routes.js b/controllers/api-routes.js index ab6cfd93..6b76a03a 100644 --- a/controllers/api-routes.js +++ b/controllers/api-routes.js @@ -42,7 +42,8 @@ export const PROJECT_WRITES = [ * GET /api/v1 lists it, so a script (or an agent) reading the list can't * miss a route. Each takes the caller's own GitHub token as * `Authorization: Bearer `; a write also needs a JSON body and is - * recorded as that token's login. + * recorded as that token's login. A read key (config.js apiReadKeys) reads + * only (lib/api-auth.js). */ export const API_ROUTES = [ { @@ -68,7 +69,11 @@ export const API_ROUTES = [ 'A PR counts toward its project label, or with none the project whose issue it links; ' + '`double_labeled` holds the open PRs in two. Each project says its `initiative` (slug, or null for ' + 'Unsorted), whether it is `tracked`, and whether it `grew` while not tracked. A project Not tracked because ' + - 'its PRs are all in projects.untrackedRepos has `initiative: null` with `tracked: false`', + 'its PRs are all in projects.untrackedRepos has `initiative: null` with `tracked: false`. For the ' + + 'labeling job: `status`, its plan’s status or null; `last_activity`, the team’s day (projects.timeZone) ' + + 'its newest PR opened, merged or was pushed to, the activity a plan is put down by; `examples`, up to 5 ' + + 'of its labeled PRs, open or merged, {repo, number, title, merged_at, by}, `by` "person" ' + + '(listed first) or "job" (a bot, or a projects.machineLabelRuns window), newest first', }, { method: 'get', @@ -151,8 +156,11 @@ export const API_ROUTES = [ handlers: [projectsController.getDecide], does: 'The decisions owed now, worst first, each with its project, roadmap item, `initiative` (its home, or ' + - 'the one asked about), and reasons (see `decide`); days are the team’s (config.js projects.timeZone, ' + - 'default America/Los_Angeles). Not-tracked projects are left out, but for the ones that grew', + 'the one asked about), and reasons (see `decide`). `auto_calls`: whether Pulldasher makes the calls ' + + 'that only say what happened itself (see `decide.made_by_pulldasher`); `pending_calls`, the ones it ' + + 'makes this pass, and the reasons they answer are left out. Days ' + + 'are the team’s (config.js projects.timeZone, default America/Los_Angeles). Not-tracked projects are ' + + 'left out, but for the ones that grew', }, { method: 'get', @@ -316,7 +324,9 @@ export const API_ROUTES = [ * against, so a caller can learn the API from the API. */ export function apiIndex(req, res) { res.json({ - auth: 'Authorization: Bearer , from a member of the org', + auth: + 'Authorization: Bearer , from a member of the org. A read key (what the ' + + 'labeling job sends; config.js apiReadKeys holds its SHA-256) reads (GET) and never writes', endpoints: API_ROUTES.map(({ method, path, does }) => ({ method: method.toUpperCase(), path, @@ -345,6 +355,8 @@ export function apiIndex(req, res) { }, done_when: `one line on what finished looks like, up to ${DONE_WHEN_MAX} characters; line breaks become spaces`, notes: 'up to 2000 characters', + auto_note: + 'read only: why Pulldasher last changed it on its own, and `auto_at` when; null when it never has', waits_on: `ids of other items this one waits on, at most ${WAITS_ON_MAX}, never a loop`, }, initiative: { @@ -370,7 +382,7 @@ export function apiIndex(req, res) { off_track: 'its latest update says off track, and the plan hasn’t changed since', at_risk: 'its latest update says at risk, and the plan hasn’t changed since', issue_closed: - 'its project issue was closed (`as` done or dropped, `on` a day) after the plan last changed, and the plan is still under way', + 'its project issue was closed (`as` done or dropped, `on` a day) after the plan last changed, and the plan is still under way; `open` = its PRs still open', reopened: 'marked done or dropped a week ago or more (`by` the roadmap or by closing the issue), but `open` PRs are still open', moving: 'parked, but its PRs changed after it was parked', @@ -385,6 +397,22 @@ export function apiIndex(req, res) { reopened_in_closed: 'its `initiative` is closed (`as` done or dropped), but it has open PRs or a plan under way again: the same rule that refuses a close. On the project’s own row when it has one', }, + made_by_pulldasher: + 'With config.js projects.autoCalls true (off by default), Pulldasher makes these calls itself after each ' + + 'work sync, recorded as `pulldasher` with why in the item’s `auto_note`. Decide leaves a reason out ' + + 'only when a call this pass answers it: `new` (a plan in progress with no end, made for any project ' + + 'with 3 or more PRs open or merged in the last 14 days, merged bursts too, unless one was ever ' + + 'removed), `reopened` and `moving` (back in progress once a PR opens, or an open PR is labeled in, ' + + 'after the status was set: 1 for a parked plan, 2 for a done or dropped one, and never once its issue ' + + 'closed after its last change), `issue_closed` and `issues_done` with no PRs open (done, or dropped when closed as ' + + 'not planned), and `ended` while nobody changed its status since its end; a plan being finished is ' + + 'asked nothing more. A PR the labeling job filed counts toward a pick-up or a new plan once its label ' + + 'is projects.settleDays old (default 3), and toward a pick-up only by when it opened; a person’s ' + + 'label counts at once. An empty lead a person may have cleared (they changed the plan since its PRs ' + + 'last moved) stays empty. It also parks a plan in progress after projects.idleDays ' + + '(default 28) with no PR opened, merged or pushed, no update and no change by a person, and fills an ' + + 'empty lead on a plan under way with its issue’s assignee, else the author of the most PRs in the ' + + 'last 60 days. A person’s write wins: change it back as usual', to_clear: 'Make the call as a roadmap write. With no item: POST /api/v1/roadmap {name, project, status, ' + 'start, weeks, end_kind: "hard"}. With one: PATCH /api/v1/roadmap/:id with {status: "active", weeks, ' + diff --git a/controllers/api.js b/controllers/api.js index a06808de..2e3bdde5 100644 --- a/controllers/api.js +++ b/controllers/api.js @@ -70,8 +70,10 @@ export default { }, /** GET /api/v1/me -- the authenticated caller's login, so a skill can - * confirm its token works and resolve the default reviewer. */ + * confirm its token works and resolve the default reviewer. A read key has + * none: `read_key` says it reads only. */ getMe: function (req, res) { - res.json({ login: req.apiUser.login }); + const { login, readKey } = req.apiUser; + res.json(readKey ? { login, read_key: true } : { login }); }, }; diff --git a/controllers/projects.js b/controllers/projects.js index de18008a..f2b01c6f 100644 --- a/controllers/projects.js +++ b/controllers/projects.js @@ -12,6 +12,14 @@ import { import { listItems } from '../lib/roadmap.js'; import { trackedOnly } from '../lib/initiatives.js'; import { trackedNow } from './roadmap.js'; +import { + callsFrom, + callsOrNone, + loadDecide, + loadProjectPulls, + pendingCalls, + projectActivity, +} from '../lib/auto-calls.js'; import { attachIssue, detachIssue, @@ -24,7 +32,6 @@ import { addWeeks, closedIssues, decideProjects, - decideQueue, decideTurn, endOf, needsDecision, @@ -39,6 +46,8 @@ import { windowStats, prStage, MISC_SLUG, + planFor, + withoutAnswered, PROJECT_SLUG, STALL_DAYS, } from '../shared/dist/index.js'; @@ -58,11 +67,6 @@ function issueOfRequest(from) { return typeof from?.issue === 'string' ? parseIssueRef(from.issue) : null; } -/** Projects that run with no end: marked on the board, or by the `ongoing` - * label on their issue. */ -function ongoingSlugs(settings, projects) { - return new Set([...settings.ongoing, ...projects.filter(p => p.ongoing).map(p => p.slug)]); -} /** * Validate the projects config and the requested window, then load the @@ -108,9 +112,18 @@ function load(req, res) { * then quiet ones, then the rest (done, dropped, or only in the window's * history) by slug. Each says its initiative (its home's slug, or null), * whether it's tracked, and whether it grew while not tracked - * (lib/initiatives.js tracking). + * (lib/initiatives.js tracking). With `plans` and `pulls` (every project's + * PRs, lib/auto-calls.js loadProjectPulls), its plan's status and what the + * labeling job reads (projectActivity) too. */ -function projectRecords(projects, today, stats, prefix, { homes, untracked, grown }) { +function projectRecords( + projects, + today, + stats, + prefix, + { homes, untracked, grown }, + { plans = [], pulls = new Map() } = {} +) { const bySlug = new Map(projects.map(p => [p.slug, p])); const groups = new Map(); const standing = new Map(); @@ -162,6 +175,8 @@ function projectRecords(projects, today, stats, prefix, { homes, untracked, grow people: g ? g.people : [], idle_days: g ? g.idleDays : null, flags: g ? g.flags : [], + status: planFor(slug, plans)?.status ?? null, + ...projectActivity(pulls.get(slug) ?? []), // its open PRs by where they stand, in the project page's words stages: stageCounts(g ? g.open : []), window: stats.projects[slug] || null, @@ -201,7 +216,9 @@ export default { if (!loaded) return; respondOrError( res, - loaded.then(({ settings, projects, linked, tracked, stats }) => { + loaded.then(async ({ settings, projects, linked, tracked, stats }) => { + // what Pulldasher's calls this pass answer, for the board's Decide + const calls = await pendingCalls(); // the PRs Today is built from: the board's open and recently closed const onBoard = new Set(pullManager.getPulls().map(p => key(p.data).toLowerCase())); return { @@ -211,6 +228,7 @@ export default { teams_from: settings.teamsFrom, decide_rotation: settings.decideRotation, ongoing: settings.ongoing, + pending_calls: calls, untracked: [...tracked.untracked].sort(), grew: [...tracked.grown.keys()].sort(), grew_counts: Object.fromEntries(tracked.grown), @@ -239,7 +257,11 @@ export default { if (!loaded) return; respondOrError( res, - loaded.then(({ settings, projects, linked, tracked, stats }) => { + loaded.then(async ({ settings, projects, linked, tracked, stats }) => { + const [plans, history] = await Promise.all([ + listItems(), + loadProjectPulls(settings, linked), + ]); const now = Date.now() / 1000; const pulls = pullManager.getPulls(); const today = todayFromBoard(pulls, projects, settings.prefix, now, linked); @@ -248,7 +270,10 @@ export default { start: stats.start, end: stats.end, label_prefix: settings.prefix, - projects: projectRecords(projects, today, stats, settings.prefix, tracked), + projects: projectRecords(projects, today, stats, settings.prefix, tracked, { + plans, + pulls: history, + }), misc: { open: today.misc.map(d => key(d.data)), window: stats.projects[MISC_SLUG] || null, @@ -507,54 +532,31 @@ export default { res.status(404).json({ error: 'projects are not set up on this Pulldasher' }); return; } - const load = Promise.all([ - loadProjects(settings), - listItems(), - loadPullLinks(settings), - trackedNow(), - ]).then(([projects, plans, linked, tracked]) => - loadWork(settings, { plans, projects }).then(work => [projects, plans, work, linked, tracked]) - ); + const now = Date.now() / 1000; + // the calls Pulldasher makes this pass, so Decide leaves out what they answer + const load = trackedNow().then(async tracked => { + const d = await loadDecide(settings, tracked, { now }); + const calls = settings.autoCalls + ? await callsOrNone(() => callsFrom(settings, d, Math.floor(now))) + : []; + return { ...d, rows: withoutAnswered(d.rows, calls), calls, tracked }; + }); respondOrError( res, - load.then(([projects, allItems, work, linked, tracked]) => { - const now = Date.now() / 1000; - const pulls = pullManager.getPulls(); - const fullToday = todayFromBoard(pulls, projects, settings.prefix, now, linked); - const { today, items } = trackedOnly(tracked.untracked, { - today: fullToday, - items: allItems, - }); + load.then(({ rows, projects, fullToday, tracked, calls }) => { const bySlug = new Map(projects.map(p => [p.slug, p])); // from the full set: a grown project's row says its open PRs too const open = new Map(fullToday.live.map(g => [g.slug, g.open.length])); const byId = new Map(tracked.initiatives.map(i => [i.id, i])); - const rows = decideQueue({ - live: decideProjects(today), - items, - initiatives: tracked.initiatives, - grown: [...tracked.grown].map(([slug, g]) => ({ slug, ...g })), - closed: closedIssues(projects), - planCounts: new Map( - work.plans.map(p => [ - p.planId, - { - openPulls: p.openPulls, - afterEnd: p.afterEnd.length, - afterDone: p.afterDone.length, - }, - ]) - ), - issues: new Map(Object.entries(work.projects)), - ongoing: ongoingSlugs(settings, projects), - // the board's Decide asks by the browser's day: the team's - today: teamDay(now), - now, - }); const day = teamDay(now); return { server_time: Math.floor(now), stall_days: STALL_DAYS, + // Pulldasher makes the calls that only say what happened + // (config.js projects.autoCalls); what this pass's calls + // answer isn't asked + auto_calls: settings.autoCalls, + pending_calls: calls, // who takes their turn running this list, this week and next runs_this_week: decideTurn(settings.decideRotation, day), runs_next_week: decideTurn(settings.decideRotation, addWeeks(day, 1)), diff --git a/controllers/roadmap.js b/controllers/roadmap.js index 5cf869b1..5ead5a6b 100644 --- a/controllers/roadmap.js +++ b/controllers/roadmap.js @@ -17,6 +17,7 @@ import { } from '../lib/roadmap.js'; import { serially, tracking, trackedOnly } from '../lib/initiatives.js'; import { sharedValue } from '../lib/ttl-cache.js'; +import { forgetPendingCalls } from '../lib/auto-calls.js'; import { loadPullLinks, loadWork } from '../lib/work.js'; import { checkRoadmapFields, @@ -165,6 +166,7 @@ const standingNow = sharedValue(STANDING_MS, async () => { /** Forget what's kept of the projects: a project write, or a sync of the * issues and their links, changed it (app.js). */ export function forgetProjectCaches() { + forgetPendingCalls(); latelyNow.forget(); trackingNow.forget(); standingNow.forget(); diff --git a/frontend-v2/src/backend/dummyProjects.ts b/frontend-v2/src/backend/dummyProjects.ts index 91333fe2..c50f6116 100644 --- a/frontend-v2/src/backend/dummyProjects.ts +++ b/frontend-v2/src/backend/dummyProjects.ts @@ -353,11 +353,16 @@ export const { DUMMY_ROADMAP, DUMMY_ROADMAP_UPDATES } = (() => { end_kind: 'hard', done_when: 'Leads can see and approve leave balances', }), - // planned to start in two weeks: the list's "Starts" cell + // planned to start in two weeks: the list's "Starts" cell; its lead + // filled in by Pulldasher two days ago, Decide's "Changed on its own" item(11, 'Newsletter promo page', 2, 4, { project: 'newsletter-promo', team: 'Store', lead: 'zdmitchell', + auto_note: 'Lead: zdmitchell, with the most PRs here in the last 60 days (3)', + auto_at: now - 2 * 86400, + // nobody changed it since, or the line would be theirs, not ours + updated_at: now - 3 * 86400, }), // parked twelve days ago while its PRs kept moving: "Parked, still // worked on", which Decide asks about diff --git a/frontend-v2/src/model/autoCalls.test.ts b/frontend-v2/src/model/autoCalls.test.ts new file mode 100644 index 00000000..6f269135 --- /dev/null +++ b/frontend-v2/src/model/autoCalls.test.ts @@ -0,0 +1,608 @@ +import { describe, expect, it } from 'vitest'; +import { dayStart } from '../../../shared/model/projects'; +import { labeledBy } from '../../../shared/model/projectLabel'; +import { + autoCalls, + leadOf, + withoutAnswered, + IDLE_DAYS, + type AutoPull, +} from '../../../shared/model/autoCalls'; +import { + answeredOnItsOwn, + decideQueue, + type DecideProject, + type DecideRow, +} from '../../../shared/model/decide'; +import type { RoadmapItem } from '../../../shared/model/roadmap'; + +const today = '2026-09-30'; +const NOW = dayStart(today) as number; +const DAY = 86400; +const ago = (days: number) => NOW - days * DAY; + +const item = (id: number, over: Partial): RoadmapItem => ({ + id, + name: `Plan ${id}`, + project: `p${id}`, + team: null, + lead: 'dana', + status: 'active', + origin: null, + start: '2026-09-07', + weeks: 8, + end_kind: 'soft', + done_when: '', + priority: id, + notes: '', + waits_on: [], + updated_by: 'dana', + updated_at: ago(30), + status_at: ago(30), + created_at: ago(60), + update: null, + ...over, +}); +const pull = (number: number, over: Partial = {}): AutoPull => ({ + repo: 'iFixit/ifixit', + number, + author: 'erin', + opened: ago(5), + merged: null, + pushed: null, + open: true, + labeledAt: ago(5), + by: 'person', + ...over, +}); +const project = (slug: string, over: Partial = {}): DecideProject => ({ + slug, + firstOpened: '2026-09-21', + lastActivity: ago(1), + open: 0, + prs: 3, + due: null, + ...over, +}); + +/** The calls for these plans and PRs, with Decide's rows read as the server does. */ +function calls( + items: RoadmapItem[], + pulls: Record, + { rows = [] as DecideRow[], planned = new Set(), ongoing = new Set() } = {} +) { + return autoCalls({ + rows, + items, + pulls: new Map(Object.entries(pulls)), + planned: new Set([...planned, ...items.map(i => i.project ?? '')]), + ongoing, + now: NOW, + }); +} + +describe('picked up again', () => { + it('puts a parked plan back in progress on one PR opened after', () => { + { + const got = calls([item(1, { status: 'parked', status_at: ago(10) })], { + p1: [pull(7, { opened: ago(3), labeledAt: ago(3) })], + }); + expect(got).toEqual([ + { + kind: 'status', + id: 1, + from: 'parked', + fromAt: ago(10), + status: 'active', + note: 'Picked up again by iFixit/ifixit#7 on 2026-09-27', + }, + ]); + } + }); + + it('takes two PRs after a done or dropped plan: one fix is not a return', () => { + for (const status of ['done', 'dropped'] as const) { + const plan = item(1, { status, status_at: ago(10) }); + const one = [pull(7, { opened: ago(3) })]; + expect(calls([plan], { p1: one })).toEqual([]); + // the second: an older open PR newly labeled into it + const two = [...one, pull(8, { opened: ago(40), labeledAt: ago(2) })]; + expect(calls([plan], { p1: two })).toEqual([ + { + kind: 'status', + id: 1, + from: status, + fromAt: ago(10), + status: 'active', + note: `Picked up again: 2 PRs since it was marked ${status}, the latest iFixit/ifixit#8 on 2026-09-28`, + }, + ]); + } + }); + + it('leaves a plan marked done while its old PRs are still open', () => { + const got = calls([item(1, { status: 'done', status_at: ago(2) })], { + p1: [pull(7, { opened: ago(20), labeledAt: ago(20) })], + }); + expect(got).toEqual([]); + }); + + it('reads only the latest plan, and none when another is under way', () => { + const pulls = { p1: [pull(7, { opened: ago(1) }), pull(8, { opened: ago(1) })] }; + const old = item(1, { status: 'done', status_at: ago(40), updated_at: ago(40) }); + const later = item(2, { + project: 'p1', + status: 'done', + status_at: ago(10), + updated_at: ago(10), + }); + expect(calls([old, later], pulls).map(c => 'id' in c && c.id)).toEqual([2]); + const running = item(2, { project: 'p1', status: 'planned' }); + expect(calls([old, running], pulls)).toEqual([]); + }); + + it('waits for the job’s label to settle; a person’s counts at once', () => { + const plan = item(1, { status: 'parked', status_at: ago(10) }); + // opened after it was parked, filed by the job: counts once settled + const job = (labeledAt: number) => pull(7, { opened: ago(5), labeledAt, by: 'job' }); + expect(calls([plan], { p1: [job(ago(2))] })).toEqual([]); + expect(calls([plan], { p1: [job(ago(3))] })[0]).toMatchObject({ status: 'active' }); + // an old PR the job filed after it was parked (a seed run): never new + // work, however long ago it was labeled + const seeded = pull(9, { opened: ago(40), labeledAt: ago(9), by: 'job' }); + expect(calls([plan], { p1: [seeded] })).toEqual([]); + // the same, labeled in by a person: they say it's this project's work now + const handed = pull(9, { opened: ago(40), labeledAt: ago(9), by: 'person' }); + expect(calls([plan], { p1: [handed] })[0]).toMatchObject({ status: 'active' }); + const person = pull(8, { opened: ago(1), labeledAt: ago(1), by: 'person' }); + expect(calls([plan], { p1: [person] })[0]).toMatchObject({ status: 'active' }); + // joined by a link, no label: at once + const linked = pull(8, { opened: ago(1), labeledAt: null, by: null }); + expect(calls([plan], { p1: [linked] })[0]).toMatchObject({ status: 'active' }); + }); + + it('never picks up a plan whose issue closed after its last change: Decide asks that', () => { + const plan = item(1, { status: 'parked', status_at: ago(10), updated_at: ago(10) }); + const got = autoCalls({ + rows: [], + items: [plan], + pulls: new Map([['p1', [pull(7, { opened: ago(3) })]]]), + planned: new Set(['p1']), + closed: new Map([['p1', { as: 'done' as const, at: ago(5) }]]), + now: NOW, + }); + expect(got).toEqual([]); + // closed before it was parked: the PR still picks it up + const before = autoCalls({ + rows: [], + items: [plan], + pulls: new Map([['p1', [pull(7, { opened: ago(3) })]]]), + planned: new Set(['p1']), + closed: new Map([['p1', { as: 'done' as const, at: ago(20) }]]), + now: NOW, + }); + expect(before[0]).toMatchObject({ status: 'active' }); + }); + + it('picks up the plan Decide reads: the last one changed', () => { + const pulls = { p1: [pull(7, { opened: ago(1) })] }; + // the earlier status, but changed since: Decide reads this one + const a = item(1, { status: 'parked', status_at: ago(40), updated_at: ago(2) }); + const b = item(2, { + project: 'p1', + status: 'parked', + status_at: ago(10), + updated_at: ago(10), + }); + expect(calls([a, b], pulls).map(c => 'id' in c && c.id)).toEqual([1]); + }); + + it('lets work go on after a finished plan of an ongoing project', () => { + const got = calls( + [item(1, { status: 'done', status_at: ago(10) })], + { p1: [pull(7, { opened: ago(1) }), pull(8, { opened: ago(1) })] }, + { ongoing: new Set(['p1']) } + ); + expect(got).toEqual([]); + }); +}); + +describe('put down', () => { + it(`parks a plan in progress after ${IDLE_DAYS} days with no PR opened, merged or pushed`, () => { + const got = calls([item(1, { status_at: ago(90), updated_at: ago(90) })], { + p1: [pull(7, { opened: ago(80), merged: ago(40), open: false, pushed: ago(30) })], + }); + expect(got).toEqual([ + { + kind: 'status', + id: 1, + from: 'active', + fromAt: ago(90), + status: 'parked', + note: 'Put down: no PR opened, merged or pushed to since 2026-08-31', + }, + ]); + }); + + it('keeps one with a push inside the window', () => { + const got = calls([item(1, { status_at: ago(90) })], { + p1: [pull(7, { opened: ago(80), pushed: ago(3) })], + }); + expect(got).toEqual([]); + }); + + it('keeps one with an update, or a person’s edit, inside the window', () => { + const old = { status_at: ago(90), updated_at: ago(90) }; + const prs = { p1: [pull(7, { opened: ago(80), open: false, merged: ago(80) })] }; + const update = { + id: 1, + item_id: 1, + health: 'on_track' as const, + body: '', + plan_start: '2026-09-07', + plan_weeks: 8, + author: 'dana', + at: ago(5), + }; + expect(calls([item(1, { ...old, update })], prs)).toEqual([]); + expect(calls([item(1, { ...old, updated_at: ago(5) })], prs)).toEqual([]); + // Pulldasher's own last write isn't a person's + const ours = item(1, { ...old, updated_at: ago(5), updated_by: 'pulldasher' }); + expect(calls([ours], prs)[0]).toMatchObject({ status: 'parked' }); + }); + + it('keeps one whose status someone set inside the window', () => { + const got = calls([item(1, { status_at: ago(5) })], { + p1: [pull(7, { opened: ago(80) })], + }); + expect(got).toEqual([]); + }); + + it('says so for a plan whose project never had a PR', () => { + const got = calls([item(1, { status_at: ago(40), updated_at: ago(40) })], {}); + expect(got[0]).toMatchObject({ + status: 'parked', + note: `Put down: no PRs in ${IDLE_DAYS} days`, + }); + }); + + it('leaves planned plans alone', () => { + expect(calls([item(1, { status: 'planned', status_at: ago(90) })], {})).toEqual([]); + }); +}); + +describe('finished', () => { + /** Decide's rows for one project's plan, asked without `auto`. */ + const rowsFor = (plan: RoadmapItem, live: DecideProject[], over = {}) => + decideQueue({ live, items: [plan], today, now: NOW, ...over }); + const closedOn = (as: 'done' | 'dropped', daysAgo: number) => + new Map([['p1', { as, at: ago(daysAgo) }]]); + + it('finishes a plan whose issue closed with no PRs open, as the issue says', () => { + const plan = item(1, {}); + for (const [as, word] of [ + ['done', 'completed'], + ['dropped', 'not planned'], + ] as const) { + const rows = rowsFor(plan, [project('p1')], { closed: closedOn(as, 3) }); + expect(calls([plan], { p1: [] }, { rows })).toEqual([ + { + kind: 'status', + id: 1, + from: 'active', + fromAt: ago(30), + status: as, + note: `Finished: its issue was closed as ${word} on 2026-09-27`, + }, + ]); + } + }); + + it('leaves it, and Decide asks, while PRs are still open', () => { + const plan = item(1, {}); + const rows = rowsFor(plan, [project('p1', { open: 2 })], { closed: closedOn('done', 3) }); + expect(calls([plan], { p1: [pull(7, { opened: ago(1) })] }, { rows })).toEqual([]); + expect(answeredOnItsOwn(rows[0].reasons[0], plan)).toBe(false); + }); + + it('finishes a plan all of whose issues closed, dropped when none was done', () => { + const plan = item(1, { start: '2026-09-07' }); + const counts = (done: number, dropped: number) => + new Map([['p1', { total: done + dropped, open: 0, done, dropped, lastClosedAt: ago(2) }]]); + const done = rowsFor(plan, [project('p1')], { issues: counts(2, 1) }); + expect(calls([plan], { p1: [] }, { rows: done })[0]).toMatchObject({ + status: 'done', + note: 'Finished: all 3 of its issues are closed, no PRs open', + }); + const dropped = rowsFor(plan, [project('p1')], { issues: counts(0, 2) }); + expect(calls([plan], { p1: [] }, { rows: dropped })[0]).toMatchObject({ status: 'dropped' }); + }); + + it('finishes a plan past its hard end with none open, until someone reopens it', () => { + const plan = item(1, { end_kind: 'hard', start: '2026-08-03', weeks: 4, status_at: ago(60) }); + const rows = rowsFor(plan, [project('p1')]); + expect(calls([plan], { p1: [] }, { rows })).toEqual([ + { + kind: 'status', + id: 1, + from: 'active', + fromAt: ago(60), + status: 'done', + note: 'Finished: its end, 2026-08-30, passed with no PRs open', + }, + ]); + // changed at 8pm on its end day where the team is, already the next + // day in UTC: after its end by UTC, not by the team's day + const lateThatDay = { ...plan, status_at: (dayStart('2026-08-31') as number) + 3 * 3600 }; + const late = rowsFor(lateThatDay, [project('p1')]); + const utc = calls([lateThatDay], { p1: [] }, { rows: late }); + expect(utc.some(c => c.kind === 'status' && c.status === 'done')).toBe(false); + expect( + autoCalls({ + rows: late, + items: [lateThatDay], + pulls: new Map([['p1', []]]), + planned: new Set(['p1']), + timeZone: 'America/Los_Angeles', + now: NOW, + }).map(c => c.kind === 'status' && c.status) + ).toEqual(['done']); + // a person put it back in progress after its end: theirs stands + const reopened = { ...plan, status_at: ago(2), updated_at: ago(2) }; + const again = rowsFor(reopened, [project('p1')]); + expect(calls([reopened], { p1: [] }, { rows: again })).toEqual([]); + }); +}); + +describe('new project', () => { + const rows: DecideRow[] = [ + { slug: 'ga4-events', item: null, reasons: [{ kind: 'new', since: '2026-09-21' }] }, + ]; + const prs = { + 'ga4-events': [ + pull(1, { opened: ago(20), author: 'erin', open: false, merged: ago(10) }), + pull(2, { opened: ago(9), author: 'kim' }), + pull(3, { opened: ago(8), author: 'kim' }), + ], + }; + + it('plans it in progress with no end, from its first open PR’s week, named from its slug', () => { + expect(calls([], prs, { rows })).toEqual([ + { + kind: 'new', + fields: { + name: 'GA4 events', + project: 'ga4-events', + status: 'active', + // the week of its first open PR, not of #1, merged + start: '2026-09-21', + end_kind: 'ongoing', + lead: 'kim', + }, + note: + 'Planned on its own: 3 PRs open or merged in the last 14 days and no plan. ' + + 'Lead: kim, with the most PRs here in the last 60 days (2)', + }, + ]); + }); + + it('takes the name its issue gives it', () => { + const got = autoCalls({ + rows, + items: [], + pulls: new Map(Object.entries(prs)), + planned: new Set(), + names: new Map([['ga4-events', 'GA4 event tracking']]), + now: NOW, + }); + expect(got[0]).toMatchObject({ fields: { name: 'GA4 event tracking' } }); + }); + + it('makes it only once 3 PRs count: the job’s labels after they settle', () => { + const job = (n: number, labeledAt: number) => + pull(n, { opened: ago(9), labeledAt, by: 'job', author: 'kim' }); + const fresh = { 'ga4-events': [pull(1), job(2, ago(1)), job(3, ago(2))] }; + expect(calls([], fresh, { rows })).toEqual([]); + const settledPrs = { 'ga4-events': [pull(1), job(2, ago(3)), job(3, ago(4))] }; + expect(calls([], settledPrs, { rows }).map(c => c.kind)).toEqual(['new']); + }); + + it('plans a burst that already merged, from its first PR’s week, with no Decide row', () => { + const merged = (n: number, days: number) => + pull(n, { opened: ago(days + 1), open: false, merged: ago(days) }); + const burst = { 'template-checker': [merged(1, 6), merged(2, 4), merged(3, 2)] }; + expect(calls([], burst)).toMatchObject([ + { kind: 'new', fields: { project: 'template-checker', start: '2026-09-21' } }, + ]); + // merged longer ago than 14 days: not lately + expect(calls([], { old: [merged(1, 20), merged(2, 18), merged(3, 15)] })).toEqual([]); + }); + + it('leaves out Not tracked, ongoing and closed projects', () => { + const got = autoCalls({ + rows: [], + items: [], + pulls: new Map([ + ['ga4-events', prs['ga4-events']], + ['ops', prs['ga4-events']], + ['upkeep', prs['ga4-events']], + ['shipped', prs['ga4-events']], + ]), + planned: new Set(), + untracked: new Set(['ops']), + ongoing: new Set(['upkeep']), + closed: new Map([['shipped', { as: 'done' as const, at: ago(1) }]]), + now: NOW, + }); + expect(got.map(c => c.kind === 'new' && c.fields.project)).toEqual(['ga4-events']); + }); + + it('never makes again a plan someone removed', () => { + expect(calls([], prs, { rows, planned: new Set(['ga4-events']) })).toEqual([]); + }); +}); + +describe('lead', () => { + it('fills an empty lead with the author of the most recent PRs', () => { + const got = calls([item(1, { lead: null, status_at: ago(3) })], { + p1: [ + pull(1, { author: 'erin', opened: ago(200), open: false, merged: ago(190) }), + pull(2, { author: 'erin', opened: ago(190), open: false, merged: ago(180) }), + pull(3, { author: 'kim', opened: ago(10) }), + ], + }); + expect(got).toEqual([ + { + kind: 'lead', + id: 1, + lead: 'kim', + note: 'Lead: kim, with the most PRs here in the last 60 days (1)', + }, + ]); + }); + + it('takes its issue’s assignee first', () => { + const got = autoCalls({ + rows: [], + items: [item(1, { lead: null, status_at: ago(3) })], + pulls: new Map([['p1', [pull(3, { author: 'kim' })]]]), + planned: new Set(['p1']), + leads: new Map([['p1', 'dana']]), + now: NOW, + }); + expect(got).toEqual([ + { kind: 'lead', id: 1, lead: 'dana', note: "Lead: dana, its issue's assignee" }, + ]); + }); + + it('fills only plans under way, and none whose status moves this pass', () => { + const prs = { p1: [pull(3, { author: 'kim', opened: ago(80) })] }; + for (const status of ['done', 'dropped', 'parked'] as const) { + expect(calls([item(1, { lead: null, status, status_at: ago(3) })], prs)).toEqual([]); + } + // put down this pass: the status call's note stands + const idle = item(1, { lead: null, status_at: ago(90), updated_at: ago(90) }); + expect(calls([idle], prs).map(c => c.kind)).toEqual(['status']); + }); + + it('leaves an empty lead a person may have cleared since its PRs last moved', () => { + const prs = { p1: [pull(3, { author: 'kim', opened: ago(10) })] }; + const cleared = item(1, { + lead: null, + status_at: ago(30), + updated_by: 'dana', + updated_at: ago(2), + }); + expect(calls([cleared], prs)).toEqual([]); + // a PR moved after their change: it fills + const moved = { p1: [pull(3, { author: 'kim', opened: ago(1) })] }; + expect(calls([cleared], moved).map(c => c.kind)).toEqual(['lead']); + // Pulldasher's own last change doesn't hold it back + const ours = { ...cleared, updated_by: 'pulldasher' }; + expect(calls([ours], prs).map(c => c.kind)).toEqual(['lead']); + }); + + it('never replaces a lead someone set', () => { + expect(calls([item(1, { status_at: ago(3) })], { p1: [pull(3)] })).toEqual([]); + }); + + it('breaks a tie by login', () => { + expect(leadOf([pull(1, { author: 'zed' }), pull(2, { author: 'amy' })], ago(60))).toEqual({ + login: 'amy', + prs: 1, + }); + }); +}); + +describe('labeledBy', () => { + const runs = [{ actor: 'jarstelfox', from: '2026-10-10T07:55:00Z', to: '2026-10-10T08:05:00Z' }]; + const isBot = (login: string) => login.endsWith('[bot]') || login === 'fixture-bot'; + const at = (iso: string) => Date.parse(iso) / 1000; + + it('says the job added a label its run added with a person’s token', () => { + expect(labeledBy('jarstelfox', at('2026-10-10T08:00:00Z'), { isBot, runs })).toBe('job'); + expect(labeledBy('JarStelfox', at('2026-10-10T08:05:00Z'), { isBot, runs })).toBe('job'); + }); + + it('says the person added one outside the run, or another person inside it', () => { + expect(labeledBy('jarstelfox', at('2026-10-10T08:05:01Z'), { isBot, runs })).toBe('person'); + expect(labeledBy('dana', at('2026-10-10T08:00:00Z'), { isBot, runs })).toBe('person'); + }); + + it('says the job added one a bot added, or one whose adder is not known', () => { + expect(labeledBy('github-actions[bot]', at('2026-10-01T00:00:00Z'), { isBot })).toBe('job'); + expect(labeledBy('fixture-bot', null, { isBot })).toBe('job'); + expect(labeledBy(null, null, { isBot })).toBe('job'); + }); +}); + +describe('withoutAnswered', () => { + const plan = (id: number, status: RoadmapItem['status']) => item(id, { status }); + const rows: DecideRow[] = [ + { + slug: 'p1', + item: plan(1, 'done'), + reasons: [{ kind: 'reopened', open: 2, late: 0, as: 'done', by: 'roadmap' }], + }, + { slug: 'p2', item: plan(2, 'parked'), reasons: [{ kind: 'moving' }] }, + { + slug: 'p3', + item: plan(3, 'active'), + reasons: [ + { kind: 'issue_closed', as: 'done', on: '2026-09-27', open: 0 }, + { kind: 'at_risk' }, + ], + }, + { slug: 'p4', item: null, reasons: [{ kind: 'new', since: '2026-09-21' }] }, + { slug: 'p5', item: null, reasons: [{ kind: 'new', since: '2026-09-21' }] }, + { + slug: 'p6', + item: null, + reasons: [{ kind: 'reopened', open: 1, late: 0, as: 'done', by: 'issue' }], + }, + ]; + const status = (id: number, from: RoadmapItem['status'], to: RoadmapItem['status']) => ({ + kind: 'status' as const, + id, + from, + fromAt: null, + status: to, + note: '', + }); + + it('asks a finished plan nothing more, its judgment calls included', () => { + const kept = withoutAnswered(rows, [status(3, 'active', 'done')]); + expect(kept.some(r => r.slug === 'p3')).toBe(false); + }); + + it('leaves out only what a call this pass answers', () => { + const kept = withoutAnswered(rows, [ + status(1, 'done', 'active'), + status(3, 'active', 'done'), + { + kind: 'new', + fields: { + name: 'P4', + project: 'p4', + status: 'active', + start: '2026-09-21', + end_kind: 'ongoing', + lead: null, + }, + note: '', + }, + ]); + expect(kept.map(r => [r.slug, ...r.reasons.map(x => x.kind)])).toEqual([ + // moved with no new PR: no call, still asked + ['p2', 'moving'], + // a removed plan's project got no new plan: still asked + ['p5', 'new'], + // reopened with no plan: nothing to pick up, still asked + ['p6', 'reopened'], + ]); + }); + + it('asks everything when no call was made', () => { + expect(withoutAnswered(rows, [])).toEqual(rows); + }); +}); diff --git a/frontend-v2/src/model/decide.test.ts b/frontend-v2/src/model/decide.test.ts index e8f1443a..9839742f 100644 --- a/frontend-v2/src/model/decide.test.ts +++ b/frontend-v2/src/model/decide.test.ts @@ -220,7 +220,12 @@ describe('decideQueue', () => { as: 'dropped', by: 'issue', }); - expect(rows[1].reasons[0]).toEqual({ kind: 'issue_closed', as: 'done', on: '2026-09-27' }); + expect(rows[1].reasons[0]).toEqual({ + kind: 'issue_closed', + as: 'done', + on: '2026-09-27', + open: 2, + }); }); it('flags a missed target until someone replans', () => { @@ -858,6 +863,37 @@ describe('filledIn', () => { ]); expect(found.started.map(p => p.id)).toEqual([1]); expect(found.vouched.map(v => [v.plan.id, v.vouch.merged])).toEqual([[3, 2]]); + expect(found.moved).toEqual([]); + }); + + it('lists the plans Pulldasher changed on its own this week, newest first', () => { + const plans = [ + item(1, { + status: 'parked', + auto_note: 'Put down: no PRs in 28 days', + auto_at: ago(2), + updated_by: 'pulldasher', + updated_at: ago(2), + }), + // a lead filled after a person's last change: still Pulldasher's + item(2, { auto_note: 'Lead: kim', auto_at: ago(1), updated_by: 'dana', updated_at: ago(5) }), + // last week's + item(3, { status: 'done', auto_note: 'Finished', auto_at: ago(8) }), + // put down, then a person changed it: theirs now + item(4, { + status: 'active', + auto_note: 'Put down: no PRs in 28 days', + auto_at: ago(3), + updated_by: 'dana', + updated_at: ago(1), + }), + ]; + const today: Pick = { live: [], quiet: [] }; + const found = filledIn({ today, prefix: 'project:', plans, pages: new Map(), now: NOW }); + expect(found.moved.map(p => [p.id, p.auto_note])).toEqual([ + [2, 'Lead: kim'], + [1, 'Put down: no PRs in 28 days'], + ]); }); }); diff --git a/frontend-v2/src/model/portfolio.ts b/frontend-v2/src/model/portfolio.ts index 9d1750e7..468a8067 100644 --- a/frontend-v2/src/model/portfolio.ts +++ b/frontend-v2/src/model/portfolio.ts @@ -1,3 +1,4 @@ +import { slugName } from '../../../shared/model/projectLabel'; import { epoch, n } from '../../../shared/format'; import { compareRows, @@ -62,38 +63,9 @@ export const upperFirst = (words: string) => words.charAt(0).toUpperCase() + wor /** Words written to start a line ("No PR activity") in the middle of one. */ export const lowerFirst = (words: string) => words.charAt(0).toLowerCase() + words.slice(1); -/** Words a slug can't case on its own, from the live P- labels: acronyms, - * and brands spelled their own way. */ -const SLUG_WORDS: Record = Object.fromEntries( - [ - 'GA4', - 'BI', - 'UPS', - 'CI', - 'PLP', - 'SSR', - 'API', - 'CAD', - 'npm', - 'idx', - 'ChatGPT', - 'NetSuite', - 'FixHub', - 'FixBot', - 'RepairDesk', - 'v2', - 'v5', - ].map(w => [w.toLowerCase(), w]) -); - -/** A label's slug read as words, the name of a project no issue or plan - * names yet: "webdriver-deflake" is "Webdriver deflake", "ga4-events" is - * "GA4 events", and "npm-token" stays "npm token", as the brand writes it. */ -export const slugName = (slug: string) => { - const words = slug.split(/[-_]+/).map(w => SLUG_WORDS[w.toLowerCase()] ?? w); - const name = words.join(' '); - return SLUG_WORDS[words[0].toLowerCase()] ? name : upperFirst(name); -}; +// a slug read as words lives with the label rule, so the server names a +// project the same way (shared/model/projectLabel.ts) +export { slugName }; /** Where a project is: being worked on (an open PR or a merge in the last * 14 days, and not parked, done or dropped on the roadmap), parked, quiet diff --git a/frontend-v2/src/model/projectData.ts b/frontend-v2/src/model/projectData.ts index 3beb7be2..7fdc8862 100644 --- a/frontend-v2/src/model/projectData.ts +++ b/frontend-v2/src/model/projectData.ts @@ -1,3 +1,4 @@ +import type { AutoCall } from '../../../shared/model/autoCalls'; import { useEffect, useMemo, useState } from 'react'; import type { Initiative } from '../../../shared/model/initiatives'; import { dummyTracking, isNotTrackedNow, useInitiatives } from './initiativesData'; @@ -55,6 +56,10 @@ export interface ProjectsData { untracked?: string[]; grew?: string[]; grew_counts?: Record; + /** the roadmap calls the server makes this pass (shared/model/autoCalls.ts): + * Decide leaves out what they answer. Empty while they're off; older + * servers omit it. */ + pending_calls?: AutoCall[]; window: WindowStats; } diff --git a/frontend-v2/src/views/MyWork.tsx b/frontend-v2/src/views/MyWork.tsx index d8861726..55451433 100644 --- a/frontend-v2/src/views/MyWork.tsx +++ b/frontend-v2/src/views/MyWork.tsx @@ -4,6 +4,7 @@ import { rowWord } from '../model/actions'; import type { PullData } from '../../../shared/types'; import { n, pullKey } from '../../../shared/format'; import { closedIssues, decideProjects, decideQueue } from '../../../shared/model/decide'; +import { withoutAnswered } from '../../../shared/model/autoCalls'; import { buildToday } from '../../../shared/model/projects'; import { dayOf } from '../model/days'; import { @@ -186,27 +187,30 @@ function useYourProjects( const ongoing = ongoingSlugs(data); // Decide's questions, asked as decideRows asks them // (views/projects/Decide.tsx), so each word names the call Decide puts first - const asked = decideQueue({ - live: decideProjects(trackedToday(today, untracked)), - items: trackedPlans(plans, initiatives), - initiatives: initiatives ?? [], - grown: grownProjects(data, initiatives), - closed: closedIssues(data.projects), - planCounts: new Map( - [...(work?.plans ?? [])].map(([id, w]) => [ - id, - { - openPulls: w.openPulls, - afterEnd: w.afterEnd.length, - afterDone: w.afterDone.length, - }, - ]) - ), - issues: work?.projects, - ongoing, - today: dayOf(new Date(now)), - now: now / 1000, - }); + const asked = withoutAnswered( + decideQueue({ + live: decideProjects(trackedToday(today, untracked)), + items: trackedPlans(plans, initiatives), + initiatives: initiatives ?? [], + grown: grownProjects(data, initiatives), + closed: closedIssues(data.projects), + planCounts: new Map( + [...(work?.plans ?? [])].map(([id, w]) => [ + id, + { + openPulls: w.openPulls, + afterEnd: w.afterEnd.length, + afterDone: w.afterDone.length, + }, + ]) + ), + issues: work?.projects, + ongoing, + today: dayOf(new Date(now)), + now: now / 1000, + }), + data.pending_calls ?? [] + ); const items = withCalls( portfolioItems( data.projects, diff --git a/frontend-v2/src/views/Projects.tsx b/frontend-v2/src/views/Projects.tsx index fdad01b2..7acba4db 100644 --- a/frontend-v2/src/views/Projects.tsx +++ b/frontend-v2/src/views/Projects.tsx @@ -182,9 +182,10 @@ export function Projects({ ? decideRows(today, plans, closedProjects, work, ongoing, { initiatives: initiatives ?? [], grown, + calls: data?.pending_calls ?? [], }) : null, - [today, plans, closedProjects, work, ongoing, initiatives, grown] + [today, plans, closedProjects, work, ongoing, initiatives, grown, data?.pending_calls] ); // a call made this visit stays with its row (a receipt and Undo) on // Decide and on the project's page; the badge counts what's still owed @@ -331,6 +332,7 @@ export function Projects({ navigate={navigate} onPerson={onPersonHere} projectsRepo={data?.projects_repo} + pendingCalls={data?.pending_calls ?? []} /> ) : nav.view === 'retro' ? ( + p.updated_by === AUTO_LOGIN || (p.updated_at ?? 0) <= (p.auto_at ?? 0); + const moved = plans + .filter(p => (p.auto_at ?? 0) >= since && stillOurs(p)) + .sort((a, b) => (b.auto_at ?? 0) - (a.auto_at ?? 0)); + return { joined, linked, started, vouched, moved }; } /** A PR's own fields, open (as the board derives it) or closed. */ @@ -2061,7 +2081,12 @@ function FilledInSection({ for (const j of held.values()) if (!joined.some(o => keyOf(o) === keyOf(j))) joined.push(j); const nameOf = (slug: string) => items.find(i => i.slug === slug)?.name ?? slug; const line = 'border-t border-secondary px-3.5 py-2 first:border-t-0'; - const total = joined.length + found.linked.length + found.started.length + found.vouched.length; + const total = + joined.length + + found.linked.length + + found.started.length + + found.vouched.length + + found.moved.length; return ( <> {withIssues.map(slug => ( @@ -2081,7 +2106,8 @@ function FilledInSection({ {WEEK_DAYS} days rather than asking anyone: issues that joined a project because its PRs link them, PRs that count in a project because they link one of its issues, plans that read In progress because their PRs moved, - and updates nobody owes because the numbers vouch. + updates nobody owes because the numbers vouch, and plans Pulldasher + changed itself because their PRs or issues said so.

Each line opens where it happened. A joined issue that doesn’t belong @@ -2156,6 +2182,26 @@ function FilledInSection({ ); })} + +

    + {found.moved.map(plan => ( +
  • + + navigate(openPlan(nav, plan.id))}> + {plan.name} + + {' · '} + {plan.auto_note} + +
  • + ))} +
+ { entry, at }. entry is { login } | { invalid: true } | // { forbidden: true }. A short-lived identity cache so a burst of API calls @@ -54,6 +56,21 @@ async function isOrgAllowed(login) { } } +/** + * Whether a Bearer value is one of config.js's `apiReadKeys`: SHA-256 hex + * digests of keys made for a job (the labeling job sends one), so the raw + * key is never in config. Each digest is compared in constant time. + */ +export function isReadKey(token, digests = config.apiReadKeys || []) { + const mine = Buffer.from(keyFor(token), 'hex'); + let found = false; + for (const digest of digests) { + const theirs = Buffer.from(String(digest).toLowerCase(), 'hex'); + if (theirs.length === mine.length && crypto.timingSafeEqual(theirs, mine)) found = true; + } + return found; +} + async function resolveUser(token) { const key = keyFor(token); const cached = await getOrSet(cache, key, TTL_MS, async () => { @@ -72,6 +89,10 @@ async function resolveUser(token) { * new credential is minted or stored -- the token is the caller's, scoped and * revocable by them, and the same one the downstream review work already * needs. MOCK_AUTH_AS_USER bypasses for local dev, mirroring lib/authentication. + * A read key (config.js apiReadKeys, isReadKey) reads only: it has no login + * to record a write as, so anything but a GET is refused before anything + * else is checked, and GitHub is never asked about it (req.apiUser then has + * a null login). */ export default function apiAuth(req, res, next) { if (FAKE_USER) { @@ -81,8 +102,18 @@ export default function apiAuth(req, res, next) { const header = req.get('authorization') || ''; const match = header.match(/^Bearer\s+(.+)$/i); if (!match) return res.status(401).json({ error: 'missing bearer token' }); + const token = match[1].trim(); + if (isReadKey(token)) { + if (!READS.has(req.method)) { + return res.status(403).json({ + error: "a read key can only read; send a person's GitHub token to make changes", + }); + } + req.apiUser = { login: null, readKey: true }; + return next(); + } - resolveUser(match[1].trim()) + resolveUser(token) .then(entry => { if (entry.invalid) return res.status(401).json({ error: 'invalid github token' }); if (entry.forbidden) { diff --git a/lib/auto-calls.js b/lib/auto-calls.js new file mode 100644 index 00000000..1dc496a4 --- /dev/null +++ b/lib/auto-calls.js @@ -0,0 +1,310 @@ +import db from './db.js'; +import pullManager from './pull-manager.js'; +import { isBot } from './review-model.js'; +import { + loadProjects, + projectSettings, + pullProjects, + teamDay, + todayFromBoard, +} from './projects.js'; +import { sharedValue } from './ttl-cache.js'; +import { autoWrite, createItem, listItems, noteItem, plannedSlugs } from './roadmap.js'; +import { serially, tracking, trackedOnly } from './initiatives.js'; +import { loadPullLinks, loadWork } from './work.js'; +import { + autoCalls, + closedIssues, + decideProjects, + dayStart, + decideQueue, + issueKey, + labeledBy, + AUTO_LOGIN, + MISC_SLUG, +} from '../shared/dist/index.js'; + +/** + * The roadmap calls Pulldasher makes itself (shared/model/autoCalls.ts): + * read what Decide reads, plus every project's PRs from the tables, and + * write each call as AUTO_LOGIN with why (lib/roadmap.js autoWrite). Runs + * after each sync of the projects' work (app.js), which a webhook on a + * project's issue or the hourly pass starts, when config.js + * projects.autoCalls is true (it's off by default). + */ + +/** Projects that run with no end: marked on the board, or by the `ongoing` + * label on their issue. */ +export function ongoingSlugs(settings, projects) { + return new Set([...settings.ongoing, ...projects.filter(p => p.ongoing).map(p => p.slug)]); +} + +/** + * Decide's queue now, and what it was read from: the projects, the plans + * (tracked ones, `items`), today's projects (`fullToday`, not-tracked ones + * too) and which projects each PR links: what the calls read + * (callsFrom). `tracked` is lib/initiatives.js tracking's answer. + */ +export async function loadDecide(settings, tracked, { now = Date.now() / 1000 } = {}) { + const [projects, allItems, linked] = await Promise.all([ + loadProjects(settings), + listItems(), + loadPullLinks(settings), + ]); + const work = await loadWork(settings, { plans: allItems, projects }); + const fullToday = todayFromBoard(pullManager.getPulls(), projects, settings.prefix, now, linked); + const { today, items } = trackedOnly(tracked.untracked, { today: fullToday, items: allItems }); + const ongoing = ongoingSlugs(settings, projects); + const closed = closedIssues(projects); + const rows = decideQueue({ + live: decideProjects(today), + items, + initiatives: tracked.initiatives, + grown: [...tracked.grown].map(([slug, g]) => ({ slug, ...g })), + closed, + planCounts: new Map( + work.plans.map(p => [ + p.planId, + { + openPulls: p.openPulls, + afterEnd: p.afterEnd.length, + afterDone: p.afterDone.length, + }, + ]) + ), + issues: new Map(Object.entries(work.projects)), + ongoing, + // the board's Decide asks by the browser's day: the team's + today: teamDay(now), + now, + }); + return { + rows, + projects, + items, + fullToday, + linked, + ongoing, + closed, + untracked: tracked.untracked, + }; +} + +const num = v => (v == null ? null : Number(v)); + +/** + * Every project's PRs, by slug, people's and no drafts, each in its project + * by the one rule (projectOf: its label, else the issue it links), with who + * put its project label on and when (null for one that joins by a link). + * Reads every PR carrying a project label and every PR a project's issue + * links, whenever they opened. + * ponytail: a few thousand rows a sync; bound it by date if the labels grow + * past tens of thousands. + */ +export async function loadProjectPulls(settings, linked) { + const like = settings.prefix.replace(/[\\%_]/g, '\\$&') + '%'; + const columns = + 'p.repo, p.number, p.title AS pr_title, p.owner, p.date, p.ready_at, p.date_merged, p.date_pushed, ' + + 'p.state, p.draft'; + const pairs = Object.keys(linked).map(k => { + const at = k.lastIndexOf('#'); + return [k.slice(0, at), Number(k.slice(at + 1))]; + }); + const [labelRows, linkedRows] = await Promise.all([ + db.query( + `SELECT ${columns}, l.title, l.user AS label_user, l.date AS label_date FROM pulls p ` + + 'JOIN pull_labels l ON l.repo = p.repo AND l.number = p.number WHERE l.title LIKE ?', + [like] + ), + pairs.length + ? db.query(`SELECT ${columns} FROM pulls p WHERE (p.repo, p.number) IN (?)`, [pairs]) + : [], + ]); + return projectPullsFrom( + labelRows, + linkedRows, + settings.prefix, + linked, + settings.machineLabelRuns + ); +} + +/** loadProjectPulls's rows by slug (pure, for the tests). `by` says who put + * the project label on, a person or the job (shared labeledBy, with the + * machine label `runs`), null for a PR that joins by a link. */ +export function projectPullsFrom(labelRows, linkedRows, prefix, linked, runs = []) { + const slugOf = pullProjects(labelRows, prefix, linked); + const label = new Map(labelRows.map(r => [`${issueKey(r)} ${r.title}`, r])); + const rows = new Map([...linkedRows, ...labelRows].map(r => [issueKey(r), r])); + const out = new Map(); + for (const r of rows.values()) { + if (r.date == null || Number(r.draft) || isBot(r.owner)) continue; + const slug = slugOf(r.repo, r.number); + if (slug == null || slug === MISC_SLUG) continue; + const put = label.get(`${issueKey(r)} ${prefix}${slug}`); + const pull = { + repo: r.repo, + number: Number(r.number), + title: r.pr_title, + author: r.owner, + opened: Number(r.ready_at ?? r.date), + merged: num(r.date_merged), + pushed: num(r.date_pushed), + open: r.state === 'open', + labeledAt: put ? num(put.label_date) : null, + labeledBy: put ? put.label_user ?? null : null, + by: put ? labeledBy(put.label_user ?? null, num(put.label_date), { isBot, runs }) : null, + }; + out.set(slug, [...(out.get(slug) ?? []), pull]); + } + return out; +} + +/** how many of a project's PRs /api/v1/projects names as its examples */ +const EXAMPLES = 5; + +/** + * What /api/v1/projects says of a project's PRs for the labeling job: + * `last_activity`, the team's day (projects.timeZone) its newest PR + * opened, merged or was pushed to, the activity a plan is put down by, and + * `examples`, up to EXAMPLES of its labeled PRs that are open or merged + * (one closed unmerged was no part of it), the ones a person labeled + * first, then the job's (projectPullsFrom `by`), newest first within each. + */ +export function projectActivity(pulls) { + const last = Math.max(0, ...pulls.flatMap(p => [p.opened, p.merged ?? 0, p.pushed ?? 0])); + const recent = p => p.merged ?? p.opened; + const examples = pulls + .filter(p => p.by != null && (p.open || p.merged != null)) + .map(p => ({ p, by: p.by })) + .sort((a, b) => (a.by === b.by ? recent(b.p) - recent(a.p) : a.by === 'person' ? -1 : 1)) + .slice(0, EXAMPLES) + .map(({ p, by }) => ({ + repo: p.repo, + number: p.number, + title: p.title, + merged_at: p.merged == null ? null : new Date(p.merged * 1000).toISOString(), + by, + })); + return { last_activity: last ? teamDay(last) : null, examples }; +} + +/** The calls the roadmap owes now, from what loadDecide read (`decide`). */ +export async function callsFrom(settings, decide, now = Math.floor(Date.now() / 1000)) { + const { rows, items, projects, linked, ongoing, closed, untracked } = decide; + const [pulls, planned] = await Promise.all([loadProjectPulls(settings, linked), plannedSlugs()]); + return autoCalls({ + rows, + items, + pulls, + planned, + names: new Map(projects.map(p => [p.slug, p.name])), + leads: new Map(projects.filter(p => p.lead).map(p => [p.slug, p.lead])), + ongoing, + closed, + untracked, + idleDays: settings.idleDays, + settleDays: settings.settleDays, + timeZone: settings.timeZone, + now, + }); +} + +// whether the last read of the calls failed, so a failing read is logged +// once, not on every Decide and board fetch +let failing = false; + +/** + * The calls, or none when reading them fails: Decide and the board then ask + * every reason, as with the calls off, rather than failing to load. The + * failure is logged once until a read works again. + */ +export function callsOrNone(reading) { + return Promise.resolve() + .then(reading) + .then(calls => { + failing = false; + return calls; + }) + .catch(err => { + if (!failing) + console.error('Reading the roadmap calls failed: %s', (err && err.message) || err); + failing = true; + return []; + }); +} + +/** + * The calls Pulldasher would make now, for the board's Decide to leave out + * what they answer; none while projects.autoCalls is off. Kept a minute and + * forgotten on every project write (controllers/roadmap.js + * forgetProjectCaches), like the other reads open boards share. + */ +const pending = sharedValue(60 * 1000, async () => { + const settings = projectSettings(); + if (!settings?.autoCalls) return []; + return callsFrom(settings, await loadDecide(settings, await tracking(settings))); +}); +export const pendingCalls = () => callsOrNone(() => pending.get()); +export const forgetPendingCalls = () => pending.forget(); + +/** a run in progress, so a sync finishing during one doesn't start a second */ +let running = null; + +/** + * Make the calls the roadmap owes now. With `apply` false it only says + * what it would do (bin/auto-calls). Resolves the calls, each with + * `landed`: false when a person's change got there first, or when not + * applied. + */ +export function runAutoCalls(settings, { apply = true, now = Math.floor(Date.now() / 1000) } = {}) { + // a dry run says what it would do whether or not the calls are on + if (!settings || (apply && !settings.autoCalls)) return Promise.resolve([]); + if (running) return running; + running = (async () => { + const decide = await loadDecide(settings, await tracking(settings), { now }); + const calls = await callsFrom(settings, decide, now); + const out = []; + for (const call of calls) { + const landed = apply && (await serially(() => write(call, now))); + if (landed) console.log('Roadmap call: %s', describe(call)); + out.push({ ...call, landed }); + } + return out; + })().finally(() => { + running = null; + }); + return running; +} + +async function write(call, now) { + if (call.kind !== 'new') return autoWrite(call, AUTO_LOGIN, now); + // checked again inside the write chain: a person may have planned it since + if ((await plannedSlugs()).has(call.fields.project)) return false; + const item = await createItem(call.fields, AUTO_LOGIN, now); + await noteItem(item.id, call.note, now); + return true; +} + +/** One call in a line, for the log and the dry run. */ +export function describe(call) { + if (call.kind === 'new') + return `new plan "${call.fields.name}" (${call.fields.project}): ${call.note}`; + if (call.kind === 'lead') return `#${call.id} lead ${call.lead}: ${call.note}`; + return `#${call.id} ${call.from} -> ${call.status}: ${call.note}`; +} + +/** + * The moment bin/auto-calls runs the rules as of: now, or with + * `--at YYYY-MM-DD` noon UTC on that day, to see the calls a later day + * brings (the put-downs idleDays after a burst of labels). Throws on a + * day it can't read. + */ +export function asOf(argv, now = Math.floor(Date.now() / 1000)) { + const i = argv.indexOf('--at'); + if (i < 0) return now; + const day = argv[i + 1]; + const start = typeof day === 'string' ? dayStart(day) : null; + if (start == null) throw new Error('--at takes a day, YYYY-MM-DD'); + return start + 12 * 3600; +} diff --git a/lib/projects.js b/lib/projects.js index bde3aa1b..a8f48eee 100644 --- a/lib/projects.js +++ b/lib/projects.js @@ -11,7 +11,9 @@ import { timeSpent, utcDay, DEFAULT_PROJECT_PREFIX, + IDLE_DAYS, ONGOING_LABEL, + SETTLE_DAYS, PARENT_PREFIX, } from '../shared/dist/index.js'; @@ -70,6 +72,17 @@ export function projectSettings() { // repos whose projects are Not tracked until someone files them // (lib/initiatives.js untrackedSlugs) untrackedRepos: config.projects.untrackedRepos || [], + // Pulldasher makes the roadmap calls that only say what happened + // (lib/auto-calls.js) only when told to, and parks a plan in + // progress after this many days with no PR activity + autoCalls: config.projects.autoCalls === true, + idleDays: config.projects.idleDays || IDLE_DAYS, + // days the labeling job's label waits before it wakes or starts a plan + settleDays: config.projects.settleDays ?? SETTLE_DAYS, + timeZone: config.projects.timeZone || DEFAULT_TIME_ZONE, + // when a script added project labels with a person's token, so those + // read as the job's (shared/model/projectLabel.ts labeledBy) + machineLabelRuns: config.projects.machineLabelRuns || [], teamOf: login => byLogin.get(login.toLowerCase()) ?? null, }; } @@ -193,7 +206,7 @@ const pullKey = (repo, number) => `${String(repo).toLowerCase()}#${number}`; * loadPullLinks), by projectOf. One lookup shared by spansFromRows, the * review narrowing in loadWindow and Look back, so a PR's project can never * disagree between them. */ -function pullProjects(labelRows, prefix, linked = {}) { +export function pullProjects(labelRows, prefix, linked = {}) { const titles = new Map(); for (const { repo, number, title } of labelRows) { const k = pullKey(repo, number); diff --git a/lib/roadmap.js b/lib/roadmap.js index 17d841b1..cba8af64 100644 --- a/lib/roadmap.js +++ b/lib/roadmap.js @@ -23,7 +23,7 @@ import { const COLUMNS = "`id`, `name`, `project`, `team`, `lead_login`, `status`, `origin`, DATE_FORMAT(`start`, '%Y-%m-%d') AS `start`, " + '`weeks`, `end_kind`, `done_when`, `priority`, `notes`, `waits_on`, `updated_by`, `updated_at`, ' + - '`created_at`, `status_at`'; + '`created_at`, `status_at`, `auto_note`, `auto_at`'; const UPDATE_COLUMNS = "`id`, `item_id`, `health`, `body`, DATE_FORMAT(`plan_start`, '%Y-%m-%d') AS `plan_start`, " + @@ -66,6 +66,8 @@ export function itemFromRow(row, update = null) { updated_at: row.updated_at == null ? null : Number(row.updated_at), created_at: row.created_at == null ? null : Number(row.created_at), status_at: row.status_at == null ? null : Number(row.status_at), + auto_note: row.auto_note || null, + auto_at: row.auto_at == null ? null : Number(row.auto_at), update, }; } @@ -344,3 +346,54 @@ export async function reorderItems(ids) { } return { items: await listItems() }; } + +/** Every project that has, or ever had, a plan: removed ones too, so + * Pulldasher never plans again a project whose plan someone removed. */ +export async function plannedSlugs() { + const rows = await db.query( + 'SELECT DISTINCT `project` FROM `roadmap_items` WHERE `project` IS NOT NULL' + ); + return new Set(rows.map(r => r.project)); +} + +// auto_note's width +const NOTE_MAX = 200; + +/** + * One of Pulldasher's own calls on a plan (shared/model/autoCalls.ts), with + * why. A status change is a call like a person's: it says who (`login`) and + * when, and moves status_at. It only lands while the plan's status is still + * `from`, set at `fromAt`, so a person's change since the calls were read + * wins, a call someone made again included. A lead is + * filled only while the plan has none, and isn't a call: updated_at stays, + * so Decide's questions about the plan still stand. False when it didn't + * land. + */ +export async function autoWrite(call, login, now = Math.floor(Date.now() / 1000)) { + const note = { auto_note: call.note.slice(0, NOTE_MAX), auto_at: now }; + const result = + call.kind === 'lead' + ? await db.query( + `UPDATE \`roadmap_items\` SET ? WHERE \`id\` = ? AND \`lead_login\` IS NULL AND ${LIVE}`, + [{ lead_login: call.lead, ...note }, call.id] + ) + : await db.query( + `UPDATE \`roadmap_items\` SET ? WHERE \`id\` = ? AND \`status\` = ? ` + + `AND \`status_at\` <=> ? AND ${LIVE}`, + [ + { status: call.status, status_at: now, updated_by: login, updated_at: now, ...note }, + call.id, + call.from, + call.fromAt, + ] + ); + return result.affectedRows > 0; +} + +/** Say why Pulldasher made a plan it just added (autoWrite's note). */ +export async function noteItem(id, note, now = Math.floor(Date.now() / 1000)) { + await db.query('UPDATE `roadmap_items` SET ? WHERE `id` = ?', [ + { auto_note: note.slice(0, NOTE_MAX), auto_at: now }, + id, + ]); +} diff --git a/lib/schema-check.js b/lib/schema-check.js index e286e870..55aa95f9 100644 --- a/lib/schema-check.js +++ b/lib/schema-check.js @@ -60,6 +60,12 @@ export const MIGRATIONS = [ { file: '0039-initiatives--add-tables.sql', table: 'initiative_projects' }, { file: '0040-pulls--add-ready-at.sql', table: 'pulls', column: 'ready_at' }, { file: '0041-pulls--add-ready-at-index.sql', table: 'pulls', index: 'pulls_ready_at' }, + // one statement adds both columns, so one stands for the pair + { + file: '0042-roadmap-items--add-auto-note.sql', + table: 'roadmap_items', + column: 'auto_note', + }, ]; /** The migration files this database hasn't had yet, oldest first. */ diff --git a/migrations/0042-roadmap-items--add-auto-note.sql b/migrations/0042-roadmap-items--add-auto-note.sql new file mode 100644 index 00000000..9d59df62 --- /dev/null +++ b/migrations/0042-roadmap-items--add-auto-note.sql @@ -0,0 +1,5 @@ +-- Why Pulldasher last changed a plan on its own (shared/model/autoCalls.ts): +-- picked up again, put down, finished, planned or given a lead, and when. +-- NULL on a plan it never changed. A person's write leaves them, so the +-- last automatic call stays readable after someone edits the plan. +ALTER TABLE `roadmap_items` ADD COLUMN `auto_note` varchar(200) COLLATE utf8mb4_general_ci DEFAULT NULL AFTER `removed_at`, ADD COLUMN `auto_at` int unsigned DEFAULT NULL AFTER `auto_note`; diff --git a/migrations/schema.sql b/migrations/schema.sql index fae4ec5f..97cf3415 100644 --- a/migrations/schema.sql +++ b/migrations/schema.sql @@ -222,6 +222,8 @@ CREATE TABLE IF NOT EXISTS `roadmap_items` ( `updated_at` int unsigned DEFAULT NULL, `status_at` int unsigned DEFAULT NULL, `removed_at` int unsigned DEFAULT NULL, + `auto_note` varchar(200) COLLATE utf8mb4_general_ci DEFAULT NULL, + `auto_at` int unsigned DEFAULT NULL, PRIMARY KEY (`id`), KEY `roadmap_items_priority` (`priority`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci; diff --git a/shared/index.ts b/shared/index.ts index dc9e1300..175e3f10 100644 --- a/shared/index.ts +++ b/shared/index.ts @@ -107,6 +107,7 @@ export { export type { DecideRotation, DeveloperTeams } from './model/settings'; export { + answeredOnItsOwn, closedIssues, decideProjects, decideQueue, @@ -143,6 +144,17 @@ export type { InitiativeSpan, InitiativeStatus, } from './model/initiatives'; +export { + autoCalls, + withoutAnswered, + AUTO_LOGIN, + IDLE_DAYS, + LEAD_DAYS, + SETTLE_DAYS, +} from './model/autoCalls'; +export type { AutoCall, AutoPull } from './model/autoCalls'; +export { labeledBy, slugName } from './model/projectLabel'; +export type { MachineLabelRun } from './model/projectLabel'; export { loadByWeek, mondaysBetween, peakFrom, spansFrom } from './model/load'; export type { InFlightSpan, LoadWeek, OriginCounts } from './model/load'; export { timeSpent } from './model/retro'; diff --git a/shared/model/autoCalls.ts b/shared/model/autoCalls.ts new file mode 100644 index 00000000..923ca88f --- /dev/null +++ b/shared/model/autoCalls.ts @@ -0,0 +1,383 @@ +import { + answeredOnItsOwn, + lastPlan, + DECIDE_MIN_PRS, + type ClosedIssue, + type DecideReason, + type DecideRow, +} from './decide'; +import { slugName } from './projectLabel'; +import { utcDay, LIVE_DAYS } from './projects'; +import { isUnderWay, mondayOf, planEnd, type RoadmapItem } from './roadmap'; + +/** + * The calls Pulldasher makes on the roadmap itself, from what already + * happened, so a plan follows the work without anyone answering Decide: + * - picked up again: a parked plan whose project gets a PR opened, or an + * open PR a person labeled into it, after that status was set, is in + * progress again; a done or dropped one takes RETURN_PRS of them, since + * one fix after a finish isn't a return (only the project's latest plan, + * only when none of its plans is under way, and never once its issue + * closed since). The job's labels never wake a plan by when they went + * on, only by when their PR opened + * - put down: a plan in progress whose project had no PR opened, merged or + * pushed to for `idleDays`, with no update posted and no change by a + * person in that time, is parked + * - finished: a plan under way whose issue closed, or all of whose issues + * closed, with none of its PRs open, is done (dropped when they were + * closed as not planned); one past its hard end with none open, its + * status untouched since, is done (model/decide.ts answeredOnItsOwn) + * - new: a project with DECIDE_MIN_PRS or more PRs open or merged in the + * last LIVE_DAYS, that never had a plan (not even one since removed), + * gets one in progress, with no end; put down parks it once it's idle. + * Unlike Decide's `new`, a burst that already merged counts too + * - lead: a plan under way with no lead, whose status this pass leaves + * alone, and that no person changed since its project's latest PR + * activity (they may have cleared it), gets its project issue's + * assignee, else the author with the most PRs in its project over the + * last LEAD_DAYS; a lead someone set is never replaced + * The job's labels count toward a pick-up or a new plan only once they + * are `settleDays` old. + * Pure: the server reads the inputs and writes the calls (lib/auto-calls.js). + * A person's write always wins, since each rule only reads changes made + * after the last status change. Decide leaves out only what a call made + * this pass answers (withoutAnswered). + */ + +/** days with no PR activity before a plan in progress is put down: the + * default for config.js projects.idleDays */ +export const IDLE_DAYS = 28; +/** how far back a plan's lead is read from its project's PRs */ +export const LEAD_DAYS = 60; +/** the login Pulldasher's own calls are recorded as */ +export const AUTO_LOGIN = 'pulldasher'; +/** PRs a done or dropped plan takes after its finish to be picked up again */ +export const RETURN_PRS = 2; +/** days the labeling job's label on a PR waits before it counts toward a + * pick-up or a new plan: the default for config.js projects.settleDays. + * The job re-sorts its own filings at each sweep, so a fresh one is still + * a guess; a person's label counts at once */ +export const SETTLE_DAYS = 3; +const NAME_MAX = 120; +const DAY = 86400; + +/** A project's PR as the calls read it: people's only, drafts left out. */ +export interface AutoPull { + repo: string; + number: number; + author: string; + /** epoch secs it opened, or was marked ready for a former draft */ + opened: number; + merged: number | null; + /** epoch secs of its last push; null when not known */ + pushed: number | null; + open: boolean; + /** epoch secs its project label was put on; null when it joins by a link */ + labeledAt: number | null; + /** who put its project label on (projectLabel.ts labeledBy); null when + * it joins by a link */ + by: 'person' | 'job' | null; +} + +/** A plan's fields a new plan is written with. */ +export interface NewPlan { + name: string; + project: string; + status: 'active'; + start: string; + end_kind: 'ongoing'; + lead: string | null; +} + +export type AutoCall = + /** change a plan's status from `from`, set at `fromAt` (its status_at + * as read, null when not stored): a call, recorded as AUTO_LOGIN's */ + | { + kind: 'status'; + id: number; + from: RoadmapItem['status']; + fromAt: number | null; + status: RoadmapItem['status']; + note: string; + } + /** fill a plan's empty lead; not a call, so the plan's other questions stand */ + | { kind: 'lead'; id: number; lead: string; note: string } + | { kind: 'new'; fields: NewPlan; note: string }; + +/** When a plan's status was last set, as near as the row knows. */ +const statusAt = (item: RoadmapItem) => item.status_at ?? item.created_at ?? item.updated_at ?? 0; + +/** The author with the most of these PRs open, or opened or merged since + * `since`, and how many; ties go to the first login alphabetically. */ +export function leadOf( + pulls: readonly AutoPull[], + since: number +): { login: string; prs: number } | null { + const counts = new Map(); + for (const p of pulls) { + if (p.open || p.opened >= since || (p.merged ?? 0) >= since) { + counts.set(p.author, (counts.get(p.author) ?? 0) + 1); + } + } + const [best] = [...counts].sort(([a, x], [b, y]) => y - x || a.localeCompare(b)); + return best ? { login: best[0], prs: best[1] } : null; +} + +/** A project's lead and why: its issue's assignee, someone's choice, else + * its busiest author lately (leadOf). */ +function leadFor( + assignee: string | undefined, + pulls: readonly AutoPull[], + now: number +): { login: string; note: string } | null { + if (assignee) return { login: assignee, note: `Lead: ${assignee}, its issue's assignee` }; + const lead = leadOf(pulls, now - LEAD_DAYS * DAY); + return ( + lead && { + login: lead.login, + note: `Lead: ${lead.login}, with the most PRs here in the last ${LEAD_DAYS} days (${lead.prs})`, + } + ); +} + +/** Whether a PR counts toward a call yet: a person's label, or one that + * joins by a link, at once; the job's once it's `settleDays` old. */ +const settled = (p: AutoPull, now: number, settleDays: number) => + p.by !== 'job' || (p.labeledAt != null && now - p.labeledAt >= settleDays * DAY); + +/** The PRs that came to a project after `since`, each opened, or open and + * labeled into it by a person, after then, with when, first first. The + * job labeling an old PR (a seed run's filing) isn't new work. */ +function cameAfter(pulls: readonly AutoPull[], since: number) { + const after = pulls.flatMap(p => { + if (p.opened > since) return [{ pull: p, at: p.opened }]; + if (p.by !== 'job' && p.open && p.labeledAt != null && p.labeledAt > since) + return [{ pull: p, at: p.labeledAt }]; + return []; + }); + return after.sort((a, b) => a.at - b.at); +} + +/** When a plan was last touched by a person or an update, as far as the + * row says: its latest update, and its last change unless Pulldasher made + * it. */ +const lastTouched = (item: RoadmapItem) => + Math.max( + item.update?.at ?? 0, + item.updated_by && item.updated_by !== AUTO_LOGIN ? item.updated_at ?? 0 : 0 + ); + +/** The newest PR opened, merged or pushed to; null with none. */ +const lastActivity = (pulls: readonly AutoPull[]) => + pulls.reduce( + (last, p) => Math.max(last ?? 0, p.opened, p.merged ?? 0, p.pushed ?? 0), + null + ); + +export function autoCalls({ + rows, + items, + pulls, + planned, + names = new Map(), + leads = new Map(), + ongoing = new Set(), + closed = new Map(), + untracked = new Set(), + idleDays = IDLE_DAYS, + settleDays = SETTLE_DAYS, + timeZone, + now, +}: { + /** Decide's queue: what happened, as it reads it */ + rows: readonly DecideRow[]; + /** the plans, tracked projects' only, as Decide was given them */ + items: readonly RoadmapItem[]; + /** each project's PRs, by slug */ + pulls: ReadonlyMap; + /** every project that has, or ever had, a plan (removed ones too), so a + * plan someone removed isn't made again */ + planned: ReadonlySet; + /** the name its issue gives a project, by slug */ + names?: ReadonlyMap; + /** its issue's assignee, by slug */ + leads?: ReadonlyMap; + /** projects that run with no end: a finished plan of theirs isn't + * picked up again by the work that follows (as Decide's `reopened`) */ + ongoing?: ReadonlySet; + /** closed project issues, by slug (decide.ts closedIssues): one closed + * after a plan stopped is the news, so the plan isn't picked up */ + closed?: ReadonlyMap; + /** Not tracked projects (initiatives.ts), which `pulls` holds too: never + * given a plan */ + untracked?: ReadonlySet; + idleDays?: number; + settleDays?: number; + /** the team's (projects.timeZone), for the day a plan ends */ + timeZone?: string; + /** epoch secs */ + now: number; +}): AutoCall[] { + const counted = (prs: readonly AutoPull[]) => prs.filter(p => settled(p, now, settleDays)); + const calls: AutoCall[] = []; + const moved = new Set(); + const setStatus = (item: RoadmapItem, status: RoadmapItem['status'], note: string) => { + if (moved.has(item.id) || item.status === status) return; + moved.add(item.id); + calls.push({ + kind: 'status', + id: item.id, + from: item.status, + fromAt: item.status_at ?? null, + status, + note, + }); + }; + + // finished, and new: what Decide reads off the issues, the end and the PRs + for (const row of rows) { + for (const r of row.reasons) { + const item = row.item; + if (!item || !isUnderWay(item.status) || !answeredOnItsOwn(r, item, timeZone)) continue; + if (r.kind === 'issue_closed') { + const as = r.as === 'dropped' ? 'not planned' : 'completed'; + setStatus(item, r.as, `Finished: its issue was closed as ${as} on ${r.on}`); + } else if (r.kind === 'issues_done') { + const all = r.done + r.dropped; + const status = r.done === 0 ? 'dropped' : 'done'; + setStatus(item, status, `Finished: all ${all} of its issues are closed, no PRs open`); + } else if (r.kind === 'ended') { + setStatus(item, 'done', `Finished: its end, ${planEnd(item)}, passed with no PRs open`); + } + } + } + + // new: enough PRs open or merged lately, and no plan ever; Decide asks + // its own `new` with every PR, but only while one is open + for (const [slug, prs] of pulls) { + if (planned.has(slug) || untracked.has(slug) || ongoing.has(slug) || closed.has(slug)) + continue; + const live = counted(prs).filter(p => p.open || (p.merged ?? 0) >= now - LIVE_DAYS * DAY); + if (live.length >= DECIDE_MIN_PRS) calls.push(newPlan(slug, live)); + } + + // in Decide's order, so lastPlan breaks a tie as it does + const plansOf = new Map(); + for (const item of [...items].sort((a, b) => a.priority - b.priority || a.id - b.id)) { + if (item.project) plansOf.set(item.project, [...(plansOf.get(item.project) ?? []), item]); + } + for (const [slug, plans] of plansOf) { + const prs = pulls.get(slug) ?? []; + // picked up again: only the plan Decide reads for the project, with + // none under way, and not when its issue closed since: that's asked + const last = plans.some(p => isUnderWay(p.status)) ? null : lastPlan(plans); + const issue = closed.get(slug); + // the issue closed after the plan's last change, as Decide reads it + if (last && !(issue && issue.at > (last.updated_at ?? 0))) { + const finished = last.status === 'done' || last.status === 'dropped'; + const after = finished && ongoing.has(slug) ? [] : cameAfter(counted(prs), statusAt(last)); + if (after.length >= (finished ? RETURN_PRS : 1)) { + const at = (p: { pull: AutoPull; at: number }) => + `${p.pull.repo}#${p.pull.number} on ${utcDay(p.at)}`; + setStatus( + last, + 'active', + finished + ? `Picked up again: ${after.length} PRs since it was marked ${last.status}, ` + + `the latest ${at(after[after.length - 1])}` + : `Picked up again by ${at(after[0])}` + ); + } + } + // put down: in progress, and nothing moved, its status included + for (const item of plans) { + if (item.status !== 'active') continue; + const pr = lastActivity(prs); + const last = Math.max(pr ?? 0, statusAt(item), lastTouched(item)); + if (now - last < idleDays * DAY) continue; + setStatus( + item, + 'parked', + pr == null + ? `Put down: no PRs in ${idleDays} days` + : `Put down: no PR opened, merged or pushed to since ${utcDay(pr)}` + ); + } + } + + // a lead for each plan under way with none, read off its project's PRs; + // a plan whose status moved this pass keeps that call's note + for (const item of items) { + if (item.lead || !item.project || !isUnderWay(item.status) || moved.has(item.id)) continue; + const prs = pulls.get(item.project) ?? []; + // a person changed it since its PRs last moved: an empty lead may be theirs + const byPerson = item.updated_by != null && item.updated_by !== AUTO_LOGIN; + if (byPerson && (item.updated_at ?? 0) > (lastActivity(prs) ?? 0)) continue; + const lead = leadFor(leads.get(item.project), prs, now); + if (lead) calls.push({ kind: 'lead', id: item.id, lead: lead.login, note: lead.note }); + } + return calls; + + // starts the week of its first open PR, else of the first that counted + function newPlan(slug: string, live: readonly AutoPull[]): AutoCall { + const prs = pulls.get(slug) ?? []; + const open = live.filter(p => p.open); + const first = Math.min(...(open.length ? open : live).map(p => p.opened)); + const lead = leadFor(leads.get(slug), prs, now); + const name = (names.get(slug) ?? slugName(slug)).slice(0, NAME_MAX).trim(); + return { + kind: 'new', + fields: { + name, + project: slug, + status: 'active', + start: mondayOf(utcDay(first)), + end_kind: 'ongoing', + lead: lead?.login ?? null, + }, + note: + `Planned on its own: ${live.length} PRs open or merged in the last ${LIVE_DAYS} days ` + + 'and no plan' + + (lead ? `. ${lead.note}` : ''), + }; + } +} + +/** The reasons each kind of status call answers, by the status it sets: + * a finished plan is asked nothing more ('all'). */ +const ANSWERS: Record = { + active: ['reopened', 'moving'], + done: 'all', + dropped: 'all', + parked: [], + planned: [], +}; + +/** + * Decide's rows without what this pass's calls answer: a plan's reasons its + * status call settles, and `new` on a project a new plan is made for. A + * row with nothing left goes. Everything no call answers is still asked: + * work that moved with no new PR, a finish whose PRs are open, a plan + * someone removed. + */ +export function withoutAnswered( + rows: readonly DecideRow[], + calls: readonly AutoCall[] +): DecideRow[] { + const byItem = new Map(); + const planned = new Set(); + for (const c of calls) { + if (c.kind === 'status') byItem.set(c.id, ANSWERS[c.status]); + else if (c.kind === 'new') planned.add(c.fields.project); + } + return rows.flatMap(row => { + const answered = row.item + ? byItem.get(row.item.id) ?? [] + : row.slug && planned.has(row.slug) + ? ['new'] + : []; + const reasons = row.reasons.filter(r => answered !== 'all' && !answered.includes(r.kind)); + if (reasons.length === row.reasons.length) return [row]; + return reasons.length ? [{ ...row, reasons }] : []; + }); +} diff --git a/shared/model/decide.ts b/shared/model/decide.ts index 68a656d2..d483dc46 100644 --- a/shared/model/decide.ts +++ b/shared/model/decide.ts @@ -1,4 +1,12 @@ -import { dayStart, firstOpenDay, targetOf, utcDay, type Project, type Today } from './projects'; +import { + dayIn, + dayStart, + firstOpenDay, + targetOf, + utcDay, + type Project, + type Today, +} from './projects'; import { goingProjects, homesOf, type Initiative } from './initiatives'; import { healthStanding, isUnderWay, planEnd, weeksPast, type RoadmapItem } from './roadmap'; import type { IssueCounts } from './work'; @@ -41,6 +49,9 @@ import type { IssueCounts } from './work'; * Every plan still under way is judged on its own, so a project's finished * first phase can't hide its second running late. A call is a roadmap write * (commit, park, finish, drop); any change to a plan counts as one. + * With config.js projects.autoCalls on, Pulldasher makes the calls that + * only say what already happened itself (model/autoCalls.ts), and Decide + * leaves out what a call this pass answers (autoCalls.ts withoutAnswered). */ export const STALL_DAYS = 21; @@ -64,7 +75,8 @@ export type DecideReason = | { kind: 'missed'; due: string; open: number } | { kind: 'off_track' } | { kind: 'at_risk' } - | { kind: 'issue_closed'; as: 'done' | 'dropped'; on: string } + /** `open`: the plan's PRs still open */ + | { kind: 'issue_closed'; as: 'done' | 'dropped'; on: string; open: number } /** `late`: PRs opened more than a week after it was marked so */ | { kind: 'reopened'; @@ -183,6 +195,53 @@ export const RANK: Record = { untracked_grew: 10, }; +/** Whether nobody changed a plan's status since its end passed: an `ended` + * plan Pulldasher finishes on its own. One a person put back under way + * after its end stays as they left it. */ +export function endedUnanswered(item: RoadmapItem, timeZone = 'UTC'): boolean { + const changed = item.status_at ?? item.updated_at ?? 0; + // its end is a day where the team is (projects.ts dayIn), as `today` is + return dayIn(changed, timeZone) <= planEnd(item); +} + +/** The plan that stands for a project with none under way: the one changed + * last (the first in `plans`' order on a tie). Decide asks about it, and + * it's the one Pulldasher picks up again (model/autoCalls.ts). */ +export function lastPlan(plans: readonly RoadmapItem[]): RoadmapItem | null { + return plans.reduce( + (a, i) => (!a || (i.updated_at ?? 0) > (a.updated_at ?? 0) ? i : a), + null + ); +} + +/** + * Whether a reason is one Pulldasher can answer itself (model/autoCalls.ts + * autoCalls reads it), when its other rules agree: a first plan (`new`), + * work picked up again (`reopened`, `moving`), and a plan finished by its + * issues or its end with nothing left open. A closed issue with PRs still + * open, or an end a person answered, never is. + */ +export function answeredOnItsOwn( + reason: DecideReason, + item: RoadmapItem | null, + /** the team's (projects.timeZone), for the `ended` day */ + timeZone?: string +): boolean { + switch (reason.kind) { + case 'new': + case 'reopened': + case 'moving': + return true; + case 'ended': + return item != null && endedUnanswered(item, timeZone); + case 'issue_closed': + case 'issues_done': + return reason.open === 0; + default: + return false; + } +} + /** A plan's PRs, by the dates (model/work.ts PlanWork), as Decide counts them. */ export interface PlanCounts { /** its PRs still open */ @@ -277,12 +336,6 @@ export function decideQueue({ // what a plan under way owes; `project` is its live project, if any const planReasons = (item: RoadmapItem, project: DecideProject | null) => { - const issue = item.project ? closed.get(item.project) : undefined; - if (issue && issue.at > decidedAt(item)) { - // the closed issue is the news; the plan's other troubles follow from it - add(item.project, item, { kind: 'issue_closed', as: issue.as, on: utcDay(issue.at) }); - return; - } const counts = planCounts.get(item.id); // a project with several plans counts each plan's own PRs (by the // dates), so a later plan's open PRs don't keep a finished one over; @@ -290,6 +343,17 @@ export function decideQueue({ // them now const several = (item.project ? plansOf.get(item.project)?.length ?? 0 : 0) > 1; const open = counts && several ? counts.openPulls : project?.open ?? 0; + const issue = item.project ? closed.get(item.project) : undefined; + if (issue && issue.at > decidedAt(item)) { + // the closed issue is the news; the plan's other troubles follow from it + add(item.project, item, { + kind: 'issue_closed', + as: issue.as, + on: utcDay(issue.at), + open, + }); + return; + } // only a hard end is a commitment to ask about once it passes if (item.end_kind === 'hard' && planEnd(item) < today) { add(item.project, item, { @@ -332,12 +396,7 @@ export function decideQueue({ for (const item of going) planReasons(item, project); // with nothing under way, its latest plan is the decision, unless its // issue was closed after that - const last = going.length - ? null - : plans.reduce( - (a, i) => (!a || decidedAt(i) > decidedAt(a) ? i : a), - null - ); + const last = going.length ? null : lastPlan(plans); const issue = closed.get(project.slug); // open PRs gone quiet, and no call in that time const idle = diff --git a/shared/model/projectLabel.ts b/shared/model/projectLabel.ts index 178826dc..50a6aa78 100644 --- a/shared/model/projectLabel.ts +++ b/shared/model/projectLabel.ts @@ -33,3 +33,68 @@ export function projectOf( const slugs = projectSlugs(labels, prefix); return slugs.find(s => s !== MISC_SLUG) ?? linked[0] ?? slugs[0] ?? null; } + +/** Words a slug can't case on its own, from the live P- labels: acronyms, + * and brands spelled their own way. */ +const SLUG_WORDS: Record = Object.fromEntries( + [ + 'GA4', + 'BI', + 'UPS', + 'CI', + 'PLP', + 'SSR', + 'API', + 'CAD', + 'npm', + 'idx', + 'ChatGPT', + 'NetSuite', + 'FixHub', + 'FixBot', + 'RepairDesk', + 'v2', + 'v5', + ].map(w => [w.toLowerCase(), w]) +); + +/** A label's slug read as words, the name of a project no issue or plan + * names yet: "webdriver-deflake" is "Webdriver deflake", "ga4-events" is + * "GA4 events", and "npm-token" stays "npm token", as the brand writes it. */ +export const slugName = (slug: string) => { + const words = slug.split(/[-_]+/).map(w => SLUG_WORDS[w.toLowerCase()] ?? w); + const name = words.join(' '); + return SLUG_WORDS[words[0].toLowerCase()] ? name : name.charAt(0).toUpperCase() + name.slice(1); +}; + +/** A stretch when a script added labels with a person's token + * (config.js projects.machineLabelRuns): a label `actor` added from `from` + * to `to` (ISO times, both counted) is the job's, not theirs. */ +export interface MachineLabelRun { + actor: string; + from: string; + to: string; +} + +/** + * Who put a project label on a PR: a `person`, or the `job` that files PRs + * (a bot, or a person's token inside one of `runs`). A label whose adder + * isn't known (a renamed label loses it) is the job's, since no person can + * be said to have chosen it. `at` is epoch secs, null when not known. + */ +export function labeledBy( + login: string | null, + at: number | null, + { isBot, runs = [] }: { isBot: (login: string) => boolean; runs?: readonly MachineLabelRun[] } +): 'person' | 'job' { + if (!login || isBot(login)) return 'job'; + const ms = at == null ? null : at * 1000; + const scripted = runs.some( + r => + r.actor.toLowerCase() === login.toLowerCase() && + ms != null && + ms >= Date.parse(r.from) && + ms <= Date.parse(r.to) + ); + return scripted ? 'job' : 'person'; +} diff --git a/shared/model/roadmap.ts b/shared/model/roadmap.ts index ff60ebdc..5b16536b 100644 --- a/shared/model/roadmap.ts +++ b/shared/model/roadmap.ts @@ -98,6 +98,10 @@ export interface RoadmapItem { updated_at: number | null; /** when it was added to the roadmap (epoch secs); null when not known */ created_at: number | null; + /** why Pulldasher last changed it on its own (model/autoCalls.ts), and + * when (epoch secs); null, or absent from older servers, when it never has */ + auto_note?: string | null; + auto_at?: number | null; /** the latest update on how it's going; null before the first */ update: RoadmapUpdate | null; /** what its project did lately, as the server read it when it sent the diff --git a/test/api-auth.test.js b/test/api-auth.test.js new file mode 100644 index 00000000..f9c3e49f --- /dev/null +++ b/test/api-auth.test.js @@ -0,0 +1,78 @@ +import { test, beforeEach, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import crypto from 'node:crypto'; +import config from '../lib/config-loader.js'; +import apiAuth, { _clearAuthCache, isReadKey } from '../lib/api-auth.js'; + +// apiAuth with a stubbed GitHub: a read key (config.js apiReadKeys holds its +// SHA-256) reads without asking GitHub, and never writes. + +const KEY = 'pd-read-0123456789abcdef'; +const digest = key => crypto.createHash('sha256').update(key).digest('hex'); +const realFetch = globalThis.fetch; +let calls; +let answer; + +beforeEach(() => { + _clearAuthCache(); + config.apiReadKeys = [digest('another key'), digest(KEY).toUpperCase()]; + calls = []; + globalThis.fetch = async url => { + calls.push(String(url)); + return answer(); + }; +}); +afterEach(() => { + globalThis.fetch = realFetch; + delete config.apiReadKeys; +}); + +const json = (status, body) => () => ({ status, ok: status < 300, json: async () => body }); + +/** Run apiAuth on one request; resolves {status, body} or {next: true, user}. */ +function run(method, token) { + return new Promise(resolve => { + const req = { method, get: () => `Bearer ${token}` }; + const res = { + code: 200, + status(code) { + this.code = code; + return this; + }, + json(body) { + resolve({ status: this.code, body }); + }, + }; + apiAuth(req, res, () => resolve({ next: true, user: req.apiUser })); + }); +} + +test('a read key reads, and GitHub is never asked', async () => { + const got = await run('GET', KEY); + assert.deepEqual(got, { next: true, user: { login: null, readKey: true } }); + assert.equal(calls.length, 0); +}); + +test('a read key never writes, refused before anything else', async () => { + for (const method of ['POST', 'PUT', 'PATCH', 'DELETE']) { + const got = await run(method, KEY); + assert.equal(got.status, 403); + assert.match(got.body.error, /can only read/); + } + assert.equal(calls.length, 0); +}); + +test('a value matching no read key is checked as a GitHub token', async () => { + answer = json(401, { message: 'Bad credentials' }); + assert.equal((await run('GET', 'pd-read-wrong')).status, 401); + assert.match(calls[0], /\/user$/); + assert.equal(isReadKey(KEY, []), false); + // a digest of the wrong length or not hex matches nothing + assert.equal(isReadKey(KEY, ['abc', 'not hex']), false); +}); + +test("a person's token still resolves to its login", async () => { + answer = json(200, { login: 'dana' }); + const got = await run('PATCH', 'gho_xyz'); + assert.deepEqual(got, { next: true, user: { login: 'dana' } }); +}); diff --git a/test/auto-calls.test.js b/test/auto-calls.test.js new file mode 100644 index 00000000..a2b676c3 --- /dev/null +++ b/test/auto-calls.test.js @@ -0,0 +1,167 @@ +import { test, mock } from 'node:test'; +import assert from 'node:assert/strict'; +import db from '../lib/db.js'; +import { asOf, describe, projectActivity, projectPullsFrom } from '../lib/auto-calls.js'; +import { autoWrite } from '../lib/roadmap.js'; + +const row = (number, over = {}) => ({ + repo: 'test/repo-a', + number, + pr_title: `PR ${number}`, + owner: 'alice', + date: 1000, + ready_at: null, + date_merged: null, + date_pushed: null, + state: 'open', + draft: 0, + ...over, +}); + +test('each PR lands in its project by the one rule, with who labeled it', () => { + const labeled = [ + { ...row(1), title: 'project:alpha', label_user: 'dana', label_date: 1100 }, + // a real project before misc + { ...row(2), title: 'project:misc', label_user: 'dana', label_date: 1100 }, + { ...row(2), title: 'project:beta', label_user: 'erin', label_date: 1200 }, + // left out: a bot's PR, a draft, a one-off + { ...row(3, { owner: 'fixture-bot' }), title: 'project:alpha' }, + { ...row(4, { draft: 1 }), title: 'project:alpha' }, + { ...row(5), title: 'project:misc' }, + ]; + // no label: it joins alpha by the issue it links, labeled by nobody + const linked = [row(6, { ready_at: 1500 })]; + const out = projectPullsFrom(labeled, linked, 'project:', { 'test/repo-a#6': ['alpha'] }); + assert.deepEqual( + [...out].map(([slug, prs]) => [slug, prs.map(p => [p.number, p.labeledBy])]), + [ + [ + 'alpha', + [ + [6, null], + [1, 'dana'], + ], + ], + ['beta', [[2, 'erin']]], + ] + ); + // a former draft counts from when it was marked ready + assert.equal(out.get('alpha')[0].opened, 1500); + assert.equal(out.get('alpha')[1].title, 'PR 1'); +}); + +test('the examples put a person’s labels first, then the job’s, newest first, five at most', () => { + const day = 86400; + const inRun = Date.parse('2026-10-10T08:00:00Z') / 1000; + // every PR in alpha, labeled by whom and when + const labeled = (number, at, user, over = {}) => ({ + ...row(number, { date: at, ...over }), + title: 'project:alpha', + label_user: user, + label_date: at, + }); + const runs = [{ actor: 'jarstelfox', from: '2026-10-10T07:55:00Z', to: '2026-10-10T08:05:00Z' }]; + const pulls = projectPullsFrom( + [ + labeled(1, 10 * day, 'dana'), + labeled(2, 20 * day, 'job[bot]'), + // a person's token inside a script run: the job's + { ...labeled(3, inRun - day, 'jarstelfox'), label_date: inRun }, + labeled(4, 5 * day, 'erin', { state: 'closed', date_merged: 30 * day }), + // closed without merging: no part of the project's work + labeled(5, 50 * day, 'erin', { state: 'closed' }), + labeled(6, 1 * day, 'fixture-bot', { owner: 'erin' }), + labeled(7, 2 * day, 'job[bot]'), + ], + // no label: joins by a link, never an example + [row(8, { date: 40 * day })], + 'project:', + { 'test/repo-a#8': ['alpha'] }, + runs + ).get('alpha'); + const { examples } = projectActivity(pulls); + assert.deepEqual( + examples.map(e => [e.number, e.by]), + [ + [4, 'person'], + [1, 'person'], + [3, 'job'], + [2, 'job'], + [7, 'job'], + ] + ); + assert.equal(examples[0].merged_at, new Date(30 * day * 1000).toISOString()); + assert.deepEqual(projectActivity([]), { last_activity: null, examples: [] }); +}); + +test('last activity is the team’s day of the newest PR opened, merged or pushed to', () => { + const pull = over => ({ opened: 0, merged: null, pushed: null, open: true, by: null, ...over }); + // 3am UTC on Oct 11 is still Oct 10 in Los Angeles, the default team zone + const lateSunday = Date.parse('2026-10-11T03:00:00Z') / 1000; + assert.equal( + projectActivity([pull({ opened: 1000 }), pull({ opened: 2000, pushed: lateSunday })]) + .last_activity, + '2026-10-10' + ); +}); + +test("an automatic status change only lands while the plan's status is still as it was read", async () => { + const seen = []; + mock.method(db, 'query', async (sql, params) => { + seen.push({ sql, params }); + return { affectedRows: 0 }; + }); + const call = { + kind: 'status', + id: 4, + from: 'done', + fromAt: 300, + status: 'active', + note: 'Picked up again', + }; + assert.equal(await autoWrite(call, 'pulldasher', 500), false); + // the status and when it was set, so a call made again since wins too + assert.match(seen[0].sql, /`status` = \? AND `status_at` <=> \? AND `removed_at` IS NULL/); + assert.deepEqual(seen[0].params, [ + { + status: 'active', + status_at: 500, + updated_by: 'pulldasher', + updated_at: 500, + auto_note: 'Picked up again', + auto_at: 500, + }, + 4, + 'done', + 300, + ]); + // a lead only fills an empty one, and isn't a call: updated_at stays + await autoWrite({ kind: 'lead', id: 4, lead: 'kim', note: 'Lead: kim' }, 'pulldasher', 500); + assert.match(seen[1].sql, /`lead_login` IS NULL/); + assert.deepEqual(seen[1].params[0], { lead_login: 'kim', auto_note: 'Lead: kim', auto_at: 500 }); + mock.restoreAll(); +}); + +test('a call reads in one line', () => { + assert.equal( + describe({ + kind: 'status', + id: 4, + from: 'active', + fromAt: 1, + status: 'parked', + note: 'Put down: x', + }), + '#4 active -> parked: Put down: x' + ); +}); + +test('bin/auto-calls runs as of now, or noon UTC on the day --at names', () => { + assert.equal(asOf(['node', 'bin/auto-calls'], 500), 500); + assert.equal( + asOf(['node', 'bin/auto-calls', '--at', '2026-11-20'], 500), + Date.parse('2026-11-20T12:00:00Z') / 1000 + ); + assert.throws(() => asOf(['node', 'bin/auto-calls', '--at', 'next week']), /YYYY-MM-DD/); + assert.throws(() => asOf(['node', 'bin/auto-calls', '--at']), /YYYY-MM-DD/); +}); diff --git a/test/projects.test.js b/test/projects.test.js index 64c03b2f..971c976d 100644 --- a/test/projects.test.js +++ b/test/projects.test.js @@ -342,6 +342,10 @@ function pullRow(number, owner, over = {}) { // the roadmap_items rows the stubbed db answers with; a test sets its own let roadmapRows = []; +// the projects that ever had a plan (lib/roadmap.js plannedSlugs) +let everPlanned = []; +// when set, reading every project's PRs for the calls fails +let pullsFail = false; // the initiatives tables, and what the not-tracked reads find in // pull_labels joined to pulls: each label's repos, and each PR's open time let initiativeRows = []; @@ -375,7 +379,15 @@ before(() => { // her own) aren't, so the window's dev/non-dev and review splits have // something real to split. 'Alice' is capitalized on purpose: teamOf must // still match the lowercase 'alice' pulls are seeded with. - config.projects = { repo: 'test/projects', developerTeams: { Store: ['Alice'] } }; + // Decide asks every call here; one test turns on the calls Pulldasher makes + config.projects = { + repo: 'test/projects', + developerTeams: { Store: ['Alice'] }, + autoCalls: false, + machineLabelRuns: [ + { actor: 'jarstelfox', from: '2026-10-10T07:55:00Z', to: '2026-10-10T08:05:00Z' }, + ], + }; const now = Math.floor(Date.now() / 1000); // two open PRs in alpha (one person), one unsorted, one bot PR in alpha const seed = (row, labels) => @@ -457,6 +469,40 @@ before(() => { .filter(r => params[0].includes(r.title)) .map((r, number) => ({ repo: 'test/repo-a', number, owner: 'alice', ...r })); } + // every project's PRs (lib/auto-calls.js loadProjectPulls): who put + // each label on, a person, a bot, or a person's token in a script run + if (sql.startsWith('SELECT p.repo, p.number, p.title AS pr_title')) { + if (pullsFail) throw new Error('lost the database'); + if (!sql.includes('JOIN pull_labels')) return []; + const pr = (number, over) => ({ + repo: 'test/repo-a', + number, + pr_title: `PR ${number}`, + owner: 'alice', + date: now - 3 * DAY, + state: 'open', + draft: 0, + title: 'project:alpha', + ...over, + }); + return [ + pr(11, { + date: now - 4 * DAY, + label_user: 'jarstelfox', + label_date: Date.parse('2026-10-10T08:00:00Z') / 1000, + }), + pr(12, { label_user: 'job-bot[bot]', label_date: now - 3 * DAY }), + pr(15, { + state: 'closed', + date: now - 5 * DAY, + date_merged: now - DAY, + label_user: 'alice', + label_date: now - 5 * DAY, + }), + ]; + } + // every project that ever had a plan, removed ones too + if (sql.includes('SELECT DISTINCT `project` FROM `roadmap_items`')) return everPlanned; if (sql.includes('FROM `roadmap_items`')) return roadmapRows; if (sql.includes('FROM `roadmap_updates`')) return []; // alpha's own issue carries its label; its side names #11, on the @@ -527,6 +573,22 @@ test('/api/v1/projects lists each project with today and the window', async () = const beta = body.projects.find(p => p.slug === 'beta'); assert.equal(beta.standing, 'dropped'); assert.deepEqual(body.unsorted.open, ['test/repo-a#13']); + // what the labeling job reads: no plan, its newest PR merged yesterday, + // and the PR a person labeled before the job's, a script run's included + assert.equal(alpha.status, null); + const yesterday = teamDay(Date.now() / 1000 - DAY); + assert.equal(alpha.last_activity, yesterday); + assert.deepEqual( + alpha.examples.map(e => [e.number, e.by]), + [ + [15, 'person'], + [12, 'job'], + [11, 'job'], + ] + ); + assert.equal(alpha.examples[0].title, 'PR 15'); + assert.ok(alpha.examples[0].merged_at); + assert.equal(alpha.examples[1].merged_at, null); assert.equal(body.totals.backlog_end, 2); assert.equal(body.days.length, 30); }); @@ -700,6 +762,67 @@ test('/api/v1/decide lists a live project with no plan as new, then a slipped pl roadmapRows = []; }); +test('/api/v1/decide leaves out only what a call Pulldasher makes this pass answers', async () => { + roadmapRows = []; + config.projects.autoCalls = true; + const asksNew = body => body.decisions.some(d => d.reasons.some(r => r.kind === 'new')); + try { + const { body } = await get('/api/v1/decide'); + assert.equal(body.auto_calls, true); + // a first plan for alpha is made, not asked + assert.deepEqual( + body.pending_calls.map(c => [c.kind, c.fields?.project]), + [['new', 'alpha']] + ); + assert.ok(!asksNew(body)); + // a plan someone removed isn't made again, so Decide still asks + everPlanned = [{ project: 'alpha' }]; + const removed = await get('/api/v1/decide'); + assert.deepEqual(removed.body.pending_calls, []); + assert.ok(asksNew(removed.body)); + } finally { + config.projects.autoCalls = false; + everPlanned = []; + } +}); + +test('Decide and the board still load when the calls can’t be read, asking everything', async () => { + roadmapRows = []; + config.projects.autoCalls = true; + pullsFail = true; + const logged = mock.method(console, 'error', () => {}); + try { + const decide = await get('/api/v1/decide'); + assert.equal(decide.status, 200); + assert.deepEqual(decide.body.pending_calls, []); + assert.ok(decide.body.decisions.some(d => d.reasons.some(r => r.kind === 'new'))); + const board = await get('/projects-data'); + assert.equal(board.status, 200); + assert.deepEqual(board.body.pending_calls, []); + // said once, not on every fetch + assert.equal( + logged.mock.calls.filter(c => String(c.arguments[0]).includes('roadmap calls')).length, + 1 + ); + } finally { + logged.mock.restore(); + pullsFail = false; + config.projects.autoCalls = false; + } +}); + +test('the calls Pulldasher makes are off unless config.js says true', async () => { + delete config.projects.autoCalls; + try { + assert.equal(projectSettings().autoCalls, false); + const { body } = await get('/api/v1/decide'); + assert.equal(body.auto_calls, false); + assert.deepEqual(body.pending_calls, []); + } finally { + config.projects.autoCalls = false; + } +}); + test('/api/v1/load counts this week and runs undecided projects ahead', async () => { roadmapRows = []; const { status, body } = await get('/api/v1/load'); diff --git a/test/roadmap.test.js b/test/roadmap.test.js index 0f8bc1f9..a2a0e534 100644 --- a/test/roadmap.test.js +++ b/test/roadmap.test.js @@ -194,6 +194,8 @@ test('itemFromRow renames the lead and fills the blanks', () => { updated_at: 1790000000, created_at: 1789000000, status_at: 1789500000, + auto_note: null, + auto_at: null, update: null, } ); diff --git a/test/schema-check.test.js b/test/schema-check.test.js index 6bbd438f..30467e61 100644 --- a/test/schema-check.test.js +++ b/test/schema-check.test.js @@ -125,6 +125,7 @@ test('names the roadmap and settings tables a database from before them lacks, i '0032-roadmap-items--add-end-kind-done-when.sql', // 0022 makes the column 24 wide; 0038 widens it '0038-roadmap-items--widen-project.sql', + '0042-roadmap-items--add-auto-note.sql', ]); mock.restoreAll(); }); @@ -146,3 +147,9 @@ test('names the ready_at index until pulls has it', async () => { assert.deepEqual(await missingMigrations(), ['0041-pulls--add-ready-at-index.sql']); mock.restoreAll(); }); + +test('names the auto note migration until roadmap_items has it', async () => { + mock.method(db, 'query', async () => all.filter(r => r.c !== 'auto_note')); + assert.deepEqual(await missingMigrations(), ['0042-roadmap-items--add-auto-note.sql']); + mock.restoreAll(); +});