Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ This file is the canonical source for unreleased changes and published release n
- `Runner.NoModelError`, reported as `no-model-error`: a run that has no model, because settings name none and no provider is configured. The CLI prints it with a hint.
- Queued steers and follow-ups stack above the desktop composer as cards: the newest in front, up to three edges behind, opening into one list on hover or focus without moving the conversation. Parked prompts show a pulsing dot, the lane (Steer or Follow Up) trails each row, and a row is edited in place (Enter saves, Esc cancels).
- A notice about the session (No Model, Something Went Wrong, Not Sent, Not Forked, a broken settings file) is the front card of that stack, tinted by its tone, with queued prompts behind it.
- Space daemons: builtin `git` and `skills` daemons keep each space's state current from inside CodeWork. A space's daemons start with the first run there and stop after an hour without one; they read a sandbox only while a run, or a short handover after it, already holds it, so they never wake one.
- `plugin.git.status` events: a space's branch, head and changed files with line counts, a wholly untracked folder as one entry. `plugin.skills.list` events: the skills sessions in a space see. Both carry `spaceId` and `observedAt`.
- `daemon.call({ plugin, spaceId, method, input })`. The git daemon's `diff` returns per-file patches; while the space's sandbox is not running it answers from the last read, marked `cached`.
- `event.subscribe` takes `types`: only those types, and first the last event of each type per space.
- `space.mount.attached` and `space.mount.detached` events: a run's drain mounting and releasing its space.
- `@codeworksh/plugin/plugin/daemon`: the daemon plugin contract (space scope).
- The desktop's project settings list each space's Skills as its skills daemon last published them, kept in `app.db`: they update as runs change them, a stopped sandbox shows its last list and its age, and a space never read says "not synced yet".

### Changed

Expand Down
10 changes: 10 additions & 0 deletions apps/contract/src/contract.ts
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,8 @@ export const syncTopics = {
delta: (threadId: string) => `sync/delta/${threadId}`,
/** `SyncTracking` whenever the user edits a desktop column other than status; stored, in the transaction that wrote it. */
tracking: "sync/tracking",
/** `SyncState` whenever a space daemon's mirrored state changed (git status, skills); stored. */
state: "sync/state",
};

/**
Expand All @@ -403,6 +405,14 @@ export const SyncChanged = Schema.Struct({
});
export type SyncChanged = typeof SyncChanged.Type;

/** A space daemon's last event of `type` for a space was mirrored anew. */
export const SyncState = Schema.Struct({
machineId: Schema.String,
spaceId: Schema.String,
type: Schema.String,
});
export type SyncState = typeof SyncState.Type;

/** A thread's status moved; `at` is the app server's clock. */
export const SyncStatus = Schema.Struct({
threadId: Schema.String,
Expand Down
12 changes: 7 additions & 5 deletions apps/server/src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,14 +134,16 @@ const migrations = {
)
`;
yield* sql`CREATE INDEX space_project_idx ON space (machine_id, project_id)`;
// The last skills read from each space: a stopped sandbox is shown from here, never woken.
// What each space's daemons last published (git status, skills), mirrored from CodeWork's event
// stream: a stopped sandbox is shown from here, never woken.
yield* sql`
CREATE TABLE skill (
CREATE TABLE space_state (
machine_id TEXT NOT NULL,
space_id TEXT NOT NULL,
skills TEXT NOT NULL,
synced_at INTEGER NOT NULL,
PRIMARY KEY (machine_id, space_id),
type TEXT NOT NULL,
data TEXT NOT NULL,
observed_at INTEGER NOT NULL,
PRIMARY KEY (machine_id, space_id, type),
FOREIGN KEY (machine_id, space_id) REFERENCES space(machine_id, id) ON DELETE CASCADE
)
`;
Expand Down
3 changes: 2 additions & 1 deletion apps/server/src/handlers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,8 @@ export const layer = Contract.Api.toLayer(

"sidebar.list": (input) => Reads.sidebar(input).pipe(Effect.provideService(SqlClient.SqlClient, sql)),
"projects.spaces": (input) => Reads.spaces(input).pipe(Effect.provideService(SqlClient.SqlClient, sql)),
"projects.skills": (input) => Skills.list(input).pipe(Effect.provideService(SqlClient.SqlClient, sql)),
"projects.skills": (input) =>
Skills.list(input, sync.client(input.machineId)).pipe(Effect.provideService(SqlClient.SqlClient, sql)),
"threads.list": (input) => Reads.threads(input).pipe(Effect.provideService(SqlClient.SqlClient, sql)),
"threads.line": (input) => Reads.line(input).pipe(Effect.provideService(SqlClient.SqlClient, sql)),

Expand Down
83 changes: 41 additions & 42 deletions apps/server/src/skills.ts
Original file line number Diff line number Diff line change
@@ -1,56 +1,55 @@
import { SpaceSkill, type SpaceSkills } from "@codeworksh/contract";
import { type CodeworkUnavailable, SpaceSkill, type SpaceSkills } from "@codeworksh/contract";
import { Effect, Option, Schema } from "effect";
import { SqlClient } from "effect/sql";
import { Codework } from "./codework.ts";
import type { Codework } from "@codeworksh/widget/backend";
import { Reads } from "./reads.ts";

const Stored = Schema.fromJsonString(Schema.Array(SpaceSkill));
const encode = Schema.encodeSync(Stored);
const Stored = Schema.fromJsonString(Schema.Struct({ skills: Schema.Array(SpaceSkill) }));
const decode = Schema.decodeUnknownOption(Stored);

/**
* Each active space's skills, read live where that costs nothing: the host every time, a sandbox
* only while CodeWork reports it running. Every read is kept, so a stopped sandbox, or a server that
* is down, shows the last one with when it was taken.
* Each active space's skills as its skills daemon last published them, mirrored from CodeWork's
* event stream: nothing here reads a space. `syncedAt` is when the daemon read them, so a stopped
* sandbox shows its last list and how old it is.
*/
export const list = Effect.fn("Skills.list")(function* (input: {
readonly machineId: string;
readonly projectId: string;
}) {
export const list = Effect.fn("Skills.list")(function* (
input: { readonly machineId: string; readonly projectId: string },
/** The machine's own connection: its sandboxes are not the default machine's. */
machine: Effect.Effect<typeof Codework.Service, CodeworkUnavailable>,
) {
const sql = yield* SqlClient.SqlClient;
const spaces = (yield* Reads.spaces(input)).filter((space) => space.status === "active");
return yield* Effect.forEach(
spaces,
Effect.fnUntraced(function* (space) {
// Mounting a running sandbox can take a while; the host answers at once.
const live = yield* Codework.call((client) => client("skill.list", { spaceId: space.id }), {
timeout: "30 seconds",
}).pipe(Effect.option);
const sandbox = Option.isSome(live) ? live.value.sandbox : null;
if (Option.isSome(live) && live.value.skills !== null) {
const skills = live.value.skills.map(({ name, description, hidden }) => ({ name, description, hidden }));
const syncedAt = Date.now();
yield* sql`
INSERT INTO skill (machine_id, space_id, skills, synced_at)
VALUES (${input.machineId}, ${space.id}, ${encode(skills)}, ${syncedAt})
ON CONFLICT (machine_id, space_id) DO UPDATE SET skills = excluded.skills, synced_at = excluded.synced_at
`.pipe(Effect.orDie);
return { space, sandbox, skills, syncedAt } satisfies SpaceSkills;
}
const [kept] = yield* sql<{ skills: string; syncedAt: number }>`
SELECT skills, synced_at AS "syncedAt" FROM skill
WHERE machine_id = ${input.machineId} AND space_id = ${space.id}
`.pipe(Effect.orDie);
const skills = kept === undefined ? Option.none() : decode(kept.skills);
return {
space,
sandbox,
skills: Option.getOrNull(skills),
syncedAt: Option.isSome(skills) && kept !== undefined ? kept.syncedAt : null,
} satisfies SpaceSkills;
}),
{ concurrency: "unbounded" },
// Sandbox rows from CodeWork's store, never from a sandbox: listing them wakes nothing.
const sandboxes = yield* machine.pipe(
Effect.flatMap((client) => client("sandbox.list", {})),
Effect.timeout("10 seconds"),
Effect.orElseSucceed(() => []),
);
const kept = yield* sql<{ spaceId: string; data: string; observedAt: number }>`
SELECT space_id, data, observed_at FROM space_state
WHERE machine_id = ${input.machineId} AND type = 'plugin.skills.list'
`.pipe(Effect.orDie);
return spaces.map((space): SpaceSkills => {
const row = kept.find((state) => state.spaceId === space.id);
const stored = row === undefined ? Option.none() : decode(row.data);
const sandbox = space.env === null ? undefined : sandboxes.find((info) => info.id === space.env);
return {
space,
sandbox:
sandbox === undefined
? null
: {
driver: sandbox.driver,
providerResourceId: Option.getOrNull(sandbox.providerResourceId),
status: sandbox.status,
},
skills: Option.match(stored, {
onNone: () => null,
onSome: (value) => value.skills.map(({ name, description, hidden }) => ({ name, description, hidden })),
}),
syncedAt: Option.isSome(stored) && row !== undefined ? row.observedAt : null,
};
});
});

export * as Skills from "./skills.ts";
74 changes: 73 additions & 1 deletion apps/server/src/sync.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
import { Codework as Client } from "@codeworksh/widget/backend";
import type { Codework as Wire } from "@codeworksh/widget/codework";
import { Clock, Context, Deferred, Effect, FiberMap, FileSystem, Layer, Option, Stream, SubscriptionRef } from "effect";
import {
Cause,
Clock,
Context,
Deferred,
Effect,
FiberHandle,
FiberMap,
FileSystem,
Layer,
Option,
Schema,
Stream,
SubscriptionRef,
} from "effect";
import { SqlClient } from "effect/sql";
import type { Publish } from "@codeworksh/widget/event";
import { CodeworkUnavailable, SyncError, type SyncThread, syncTopics } from "@codeworksh/contract";
Expand All @@ -10,6 +24,12 @@ import { Mirror } from "./mirror.ts";
import { Servers } from "./servers.ts";

type Connection = typeof Client.Service;

/** The space daemon events this server mirrors: the builtin ones the desktop shows. */
const daemonTypes = ["plugin.git.status", "plugin.skills.list"];
/** Where a daemon event belongs and when it was read; the rest is stored as it came. */
const decodePoint = Schema.decodeUnknownOption(Schema.Struct({ spaceId: Schema.String, observedAt: Schema.Number }));
const DataText = Schema.fromJsonString(Schema.Unknown);
type IndexSynced = Extract<Wire.IndexItem, { _tag: "Synced" }>;
type IndexRow = Exclude<Wire.IndexItem, IndexSynced>;
type ThreadSynced = Extract<Wire.ThreadItem, { _tag: "Synced" }>;
Expand Down Expand Up @@ -239,6 +259,50 @@ export const layer = Layer.effect(
}
});

/**
* The builtin space daemons' events, into `space_state`. A subscription opens with each space's
* last event: it is (re)started after an index batch mirrored spaces, so that replay finds their
* rows, and again whenever it fails or ends. Only a newer event replaces a row.
*/
const followDaemons = (machineId: string, client: Connection) =>
client("event.subscribe", { types: daemonTypes }).pipe(
Stream.filter((frame) => daemonTypes.includes(frame.type)),
Stream.runForEach((frame) =>
Effect.gen(function* () {
const point = decodePoint(frame.data);
if (Option.isNone(point)) return;
const { spaceId, observedAt } = point.value;
const text = yield* Schema.encodeEffect(DataText)(frame.data).pipe(Effect.orDie);
yield* transact((append) =>
Effect.gen(function* () {
const written = yield* sql<{ spaceId: string }>`
INSERT INTO space_state (machine_id, space_id, type, data, observed_at)
SELECT ${machineId}, ${spaceId}, ${frame.type}, ${text}, ${observedAt}
WHERE EXISTS (SELECT 1 FROM space WHERE machine_id = ${machineId} AND id = ${spaceId})
ON CONFLICT (machine_id, space_id, type) DO UPDATE
SET data = excluded.data, observed_at = excluded.observed_at
WHERE space_state.observed_at < excluded.observed_at
RETURNING space_id
`;
if (written.length === 0) return;
yield* append({
topic: syncTopics.state,
source: "sync",
payload: { machineId, spaceId, type: frame.type },
});
}),
);
}),
),
Effect.catchCause((cause) =>
Cause.hasInterruptsOnly(cause)
? Effect.interrupt
: Effect.logWarning(`Space daemon events of machine ${machineId} stopped`, cause),
),
Effect.andThen(Effect.sleep("1 second")),
Effect.forever,
);

/** One connection's sync: decides between resume and full sync, then follows the index. */
const connected = (machineId: string, client: Connection) =>
Effect.gen(function* () {
Expand All @@ -263,6 +327,8 @@ export const layer = Layer.effect(
if (lives.get(machineId) === live) lives.delete(machineId);
}),
);
const daemons = yield* FiberHandle.make<never>();
let following = false;
yield* batches<IndexRow, IndexSynced>(
client("index.sync", { after: full ? 0 : machine.cursor }),
(item) => item._tag === "Synced",
Expand All @@ -271,6 +337,7 @@ export const layer = Layer.effect(
Effect.gen(function* () {
const now = yield* Clock.currentTimeMillis;
const wasFull = full;
let spaces = false;
yield* transact((append) =>
Effect.gen(function* () {
const changes = yield* Mirror.index(
Expand All @@ -280,6 +347,7 @@ export const layer = Layer.effect(
append,
);
if (wasFull) yield* Mirror.unseen(machineId, changes, now);
spaces = changes.space.size > 0;
yield* sql`
UPDATE machine SET cursor = ${synced.rev}, store_id = ${synced.storeId} WHERE id = ${machineId}
`;
Expand All @@ -292,6 +360,10 @@ export const layer = Layer.effect(
}),
);
full = false;
if (wasFull || spaces || !following) {
following = true;
yield* FiberHandle.run(daemons, followDaemons(machineId, client));
}
lives.set(machineId, live);
yield* SubscriptionRef.update(applied, (cursors) => new Map(cursors).set(machineId, synced.rev));
yield* startOpened(machineId, live);
Expand Down
79 changes: 79 additions & 0 deletions apps/server/test/__artifacts__/daemon.machines.daytona.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
{
"the sandbox's own folder is not a repository": {
"available": false,
"files": [],
"reason": "not-a-repository"
},
"pushed after an edit in the sandbox": {
"head": "<pinned>",
"available": true,
"branch": "main",
"files": [
{
"file": "README.md",
"status": "modified",
"additions": 1,
"deletions": 0
},
{
"file": "sandbox.txt",
"status": "added",
"additions": 1,
"deletions": 0
}
]
},
"diff right after the run": {
"cached": true,
"files": [
{
"file": "README.md",
"patch": "diff --git a/README.md b/README.md\nindex <blobs>\n--- a/README.md\n+++ b/README.md\n@@ -1,2 +1,3 @@\n # 69th\n 69th Testing Repository for Agents\n+from daytona\n",
"status": "modified",
"additions": 1,
"deletions": 0
}
]
},
"the machine's row after the owner stopped it": "online",
"diff once it is stopped": {
"cached": true,
"files": [
{
"file": "README.md",
"patch": "diff --git a/README.md b/README.md\nindex <blobs>\n--- a/README.md\n+++ b/README.md\n@@ -1,2 +1,3 @@\n # 69th\n 69th Testing Repository for Agents\n+from daytona\n",
"status": "modified",
"additions": 1,
"deletions": 0
}
]
},
"provider status after that call": "offline",
"replayed while it is stopped": [
"server.connected",
{
"available": false,
"files": [],
"reason": "not-a-repository"
},
{
"head": "<pinned>",
"available": true,
"branch": "main",
"files": [
{
"file": "README.md",
"status": "modified",
"additions": 1,
"deletions": 0
},
{
"file": "sandbox.txt",
"status": "added",
"additions": 1,
"deletions": 0
}
]
}
]
}
Loading