Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugins/fxtr/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "fxtr",
"version": "0.1.2",
"version": "0.1.3",
"description": "Write, run, and view fxtr experiments, and call models from them with behaviors.",
"author": {
"name": "Transluce",
Expand Down
2 changes: 1 addition & 1 deletion plugins/fxtr/skills/behaviors/provenance.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"fxtr": "0.0.1a4",
"behaviors": "0.0.1a3"
},
"revision": "051a5d3ab1f6df872092af2365b34c5d0f6671d3",
"revision": "c3782b939f40b8fbffa00c43c984c0ff83087cb6",
"modified": false,
"content_sha256": "9ca8c278e92e77f29f281313b17717930e6ecc67e13027fedde3152fc37ea5b9"
}
2 changes: 1 addition & 1 deletion plugins/fxtr/skills/first-experiment/provenance.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"fxtr": "0.0.1a4",
"behaviors": "0.0.1a3"
},
"revision": "051a5d3ab1f6df872092af2365b34c5d0f6671d3",
"revision": "c3782b939f40b8fbffa00c43c984c0ff83087cb6",
"modified": false,
"content_sha256": "1b39e083738e6d10b515568bf7912241a1874cda150eb40908b92e7927d38b5f"
}
12 changes: 7 additions & 5 deletions plugins/fxtr/skills/fxtr/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ Do these by default, unless the user asks otherwise.
* **Write each docstring's first sentence for the viewer.** The operation's card shows it, so make
it a short imperative sentence saying what the operation does
([Watch jobs in the viewer](references/guides/running-and-viewing-jobs.mdx#watch-jobs-in-the-viewer)).
* **Describe the experiment in `EXPERIMENT.md`** at the project root, which the viewer's rail
links to: the question, what is compared or measured and why, and how scores are defined, in
plain English for a reader new to the experiment. Keep it current as the experiment changes
([Watch jobs in the viewer](references/guides/running-and-viewing-jobs.mdx#watch-jobs-in-the-viewer)).
* **Return a report from the root workflow**: one entity that references every result worth
presenting, for the launcher and the results overview to read
([Read results](references/guides/running-and-viewing-jobs.mdx#read-results)).
Expand All @@ -101,13 +105,11 @@ before [adding one](references/reference/renderers.mdx#adding-a-package-to-a-pro
* **Give each step an `InvocationPanel`**, registered
under the step's name, whenever there is something useful to show beyond the output array itself. The step's pane already shows its arrays as tables, so don't redraw them
([Where a view appears](references/guides/designing-views.mdx#where-a-view-appears)).
The panel becomes the step's default view, in its pane and in its card's preview, and it is
The panel becomes the step's default view, in the pane its card opens, and it is
handed every invocation the viewer's selection picks out, up to all of a mapped step's. Show
one in full and several together (a chart or a summary), loading their outputs with `LoadEach`
([Step panels](references/reference/renderers.mdx#step-panels)).
Wrapping a view of one invocation in `singleInvocation` asks the reader to select one when
there are several, so keep it for the root workflow and for steps that are neither mapped nor
replicated.
([Step panels](references/reference/renderers.mdx#step-panels)). Never show a note
asking the reader to select one; `singleInvocation`, which does that, is deprecated.
* **Give the root workflow a results overview**: an `InvocationPanel` that reads the workflow's
report and presents the experiment's findings
([Results overviews](references/concepts/visualization-and-custom-renderers.mdx#results-overviews)).
Expand Down
4 changes: 2 additions & 2 deletions plugins/fxtr/skills/fxtr/provenance.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"fxtr": "0.0.1a4",
"behaviors": "0.0.1a3"
},
"revision": "051a5d3ab1f6df872092af2365b34c5d0f6671d3",
"revision": "c3782b939f40b8fbffa00c43c984c0ff83087cb6",
"modified": false,
"content_sha256": "51ba3d88cfa95b80fee856d1d9a8672f7df95e54d5ae269b9ceee5fcee63bbe9"
"content_sha256": "c4930667ed6ac4665628fab04edcd9a755516598d49cdd08199f1cb544779c7d"
}
2 changes: 1 addition & 1 deletion plugins/fxtr/skills/fxtr/references/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ in the order of the site's navigation.
- [Designing views](guides/designing-views.mdx): How a renderer's views should look and behave in the viewer: surfaces, type, color, charts, links, controls, and how to check them.
- [Array queries](reference/array-queries.mdx): Filter, reshape, aggregate, and join arrays with the query builder or SQL, in a workflow or in a step.
- [Project configuration](reference/project-configuration.mdx): What fxtr new writes, and the settings in pyproject.toml and fxtr.local.toml.
- [Renderers](reference/renderers.mdx): The renderer API: slots and registration, loading entities and invocations, step panels, links, panel state, the kit, and building the renderer package.
- [Renderers](reference/renderers.mdx): The renderer API: slots and registration, loading entities and invocations, step panels, links and detail views, panel state, the kit, and building the renderer package.
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,11 @@ an entity.
<Note>
fxtr currently converts dataclasses to and from their stored form with the
[cattrs](https://catt.rs/) library, with strict settings: unknown fields, missing fields, and
values of the wrong type are rejected rather than coerced. This implementation detail may change
in a future version; the dataclass surface described here is what to rely on.
values of the wrong type are rejected rather than coerced. Storing an entity checks that it will
load back under these rules, so a value they reject, such as an `int` in a `float` field or a
subclass in a field annotated with its base class, fails when it is stored rather than when it is
next read. This implementation detail may change in a future version; the dataclass surface
described here is what to rely on.
</Note>

### Advanced use: low-level entity classes
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,13 @@ ID, and the component:

```ts views/src/bundle.ts
import { EntityPanel, InvocationPanel, renderer, type ViewBundle } from "fxtr-view";
import { singleInvocation } from "fxtr-view-kit";
import { AnswerPanel } from "./answer";
import { Overview } from "./overview";

const bundle: ViewBundle = {
renderers: [
renderer(EntityPanel, "org.example.qa.Answer.v1", "my-project.AnswerPanel", AnswerPanel),
renderer(InvocationPanel, "qa.evaluate", "my-project.Overview", singleInvocation(Overview)),
renderer(InvocationPanel, "qa.evaluate", "my-project.Overview", Overview),
],
};

Expand All @@ -81,7 +80,7 @@ export default bundle;

Renderers are built with two libraries. **`fxtr-view`** is the small, stable contract between a
renderer and the viewer: the slots, `renderer(...)`, and hooks for loading data and linking to
entities. **`fxtr-view-kit`** has building blocks that match the viewer: layout classes, chart
entities and to other views. **`fxtr-view-kit`** has building blocks that match the viewer: layout classes, tables, chart
parts and controls, number formats, and adapters for showing invocations and loading many entities
at once. The [renderer reference](../reference/renderers.mdx) describes both, with the rules for
registering renderers.
Expand Down
66 changes: 55 additions & 11 deletions plugins/fxtr/skills/fxtr/references/guides/designing-views.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,19 +15,31 @@ covers registering renderers, loading data, and the kit's API.
A project written with `fxtr new --example` has a complete small example in
`views/src/overview.tsx`: a results overview with a chart that follows everything below.

## What a view is for

A view gives its reader an accurate and detailed picture of what happened in the experiment and
what data it produced, even when the reader knows the experiment only at a high level. A view
that says only "A scored 40 and B scored 50" leaves them asking what the scores are for and how
they were computed. Answer that with the data itself: a good visualization shows the important
facts in its marks, and links let the reader dig into the details behind each one
([Linking to detail views](../reference/renderers.mdx#linking-to-detail-views)). Leave out
explanatory prose the data can show instead. The project's `EXPERIMENT.md`
([Watch jobs in the viewer](running-and-viewing-jobs.mdx#watch-jobs-in-the-viewer)) is
where the experiment itself is explained.

## Where a view appears

| Surface | Renderer | Room | Base type | Surface under it |
| ---------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------- | --------- | ---------------- |
| Results overview | `InvocationPanel` of the root workflow | the page, up to 1400px wide, 12px side margins | 13px | `--bg-alt` |
| A pane | `EntityPanel`, or a step's `InvocationPanel` | resizable; can be 300px or less | 12px | `--bg-alt` |
| A card's preview | the node's default view | beside each card while no pane is open, about half the page wide, as tall as the card, scrolling inside | 12px | `--bg-alt` |
| Hover card | `EntityHoverPreview` | 360px wide, at most 300px tall, the full cid below it | 11px | `--bg` |
| Inline link | `EntityInlineLink` | one line inside a pill, cut off with an ellipsis | 11px | the pill |
| Surface | Renderer | Room | Base type | Surface under it |
| ---------------- | -------------------------------------------- | ----------------------------------------------------- | --------- | ---------------- |
| Results overview | `InvocationPanel` of the root workflow | the page, up to 1400px wide, 12px side margins | 13px | `--bg-alt` |
| A pane | `EntityPanel`, or a step's `InvocationPanel` | 600px wide when it opens; resizable | 12px | `--bg-alt` |
| Hover card | `EntityHoverPreview` | 360px wide, at most 300px tall, the full cid below it | 11px | `--bg` |
| Inline link | `EntityInlineLink` | one line inside a pill, cut off with an ellipsis | 11px | the pill |

A project's own renderer becomes the default view of its node, so the same component may draw at
full page width, in a narrow pane, and in a card's preview. Design for every width from about 320px
A project's own renderer becomes the default view of its node's pane, so the same component may
draw at full page width and in a narrow pane. Design for every width from about 320px
to 1400px, and put the finding that matters most at the top left, where every surface shows it.
Where you can, fit a pane's opening width of 600px without overflowing or scrolling sideways.

The viewer draws the frame. Leave out what it already gives:

Expand Down Expand Up @@ -124,6 +136,31 @@ Where marks need separating, leave real space between them, not a stroke or bord
* Shadows only on what floats (the tooltip). Rules and borders are 1px.
* No animation or transition of your own; the viewer animates only what is running.

## Tables

A list of records a reader may want to reorder or search (every prompt, every reading, a run's
items) goes in the kit's `DataTable`, not a table written by hand: its columns sort, filter, and
resize, and its header stays in view as the rows scroll.

```tsx
<DataTable
rows={readings}
rowKey={(r) => r.id}
columns={[
{ key: "judge", label: "judge", value: (r) => r.judge },
{ key: "score", label: "score", numeric: true, value: (r) => r.score, cell: (r) => formatNumber(r.score, 2) },
]}
/>
```

A column's `value` is what it sorts and filters by, and its `cell` draws it (a bar, a link);
`numeric` right-aligns it in tabular figures. `renderRow(row, tr)` draws a row through `tr(props,
selected)`, to make rows pressable or show one as selected. To open a detail view from a row,
return a component that calls `useLink` and draws the row with `tr(link.props, link.open)`
([Linking to detail views](../reference/renderers.mdx#linking-to-detail-views)). Write a table by
hand, with `layout.table`, only for a matrix or a chart laid out as a table, such as the heatmap
[below](#example-a-heatmap).

## Charts

### Choose the form
Expand Down Expand Up @@ -175,7 +212,10 @@ draws follows this guide: its own colors, fonts, tooltips, and fixed sizes do no
3 to 4px radius; lines 1.5 to 2px. Direct labels (`layout.chartLabel`) on the few values that
matter, not a number on every mark.
* **Legends.** A `Legend` for two or more series, placed above the chart; none for one series
(its heading names it). A `ScaleLegend` for a sequential or diverging fill.
(its heading names it). A `ScaleLegend` for a sequential or diverging fill. Explain any other
encoding with a legend drawn from the chart's own marks, not a sentence: a filled square
labeled "won" beside an open one labeled "lost", rather than "a square is filled when the poet
won".
* **Text in SVG** inherits the viewer's sans; size and color it with the `layout` classes or the
type tokens. Align with SVG attributes (`textAnchor`, `dominantBaseline`).

Expand All @@ -199,6 +239,10 @@ draws follows this guide: its own colors, fonts, tooltips, and fixed sizes do no
focus, cleared on leave and blur: the mark's name in bold, its value, and a line of context.
Nothing in a tooltip can be pressed, so links go in the chart or under it. Never use SVG
`<title>` or the `title` attribute for this.
* **Details beside the view.** A pick whose detail needs room of its own (a point's readings, a
row's transcript) opens a detail view in a pane beside this one, through `useLink`, rather than
growing the view ([Linking to detail views](../reference/renderers.mdx#linking-to-detail-views)).
Spread the link's props onto the mark and draw it with `data-open={link?.open}`.
* **Picks within the view.** A reader's pick that stays inside the view (a series focused, a
point whose detail shows under the chart) is drawn with `data-picked="true"` (the same ink
ring), the rest `layout.faded`; its detail goes in a `DetailBox`.
Expand Down Expand Up @@ -320,4 +364,4 @@ grep -rnE 'font-family: *[^v ]|font-size: *[0-9]|color-scheme|!important|<title>

The rest is a look at the view in the viewer. The usual things to catch are a theme it reads badly
in (the toggle is at the foot of the viewer's left rail) and a width it breaks at: drag a pane
narrow, or close every pane to see the card previews.
narrow.
Original file line number Diff line number Diff line change
Expand Up @@ -121,14 +121,23 @@ For one job, the viewer shows:
by dimension. A value that references an entity is a link.
* **Entity details**, opened from any link, in a pane you can keep alongside the trace. Hovering a
link shows a preview.
* **Your project's README**, linked from the side rail.
* **Your project's `EXPERIMENT.md`**, linked from the side rail.

The first sentence of each step's and workflow's docstring is what appears on its card, so write
those as short imperative sentences that tell a reader what the operation does, such as "Compute
mean scores across judges." Leave out edge cases and what the step is mapped over, which the card
already shows. For progress worth reading while a step runs, write log lines with
`context.log(...)`.

`EXPERIMENT.md`, at the project root, describes the experiment for someone reading its results:
the question it asks, what it compares or measures and why, and how each score or judgment is
defined. Keep it current as the experiment changes.

* Write it in plain English for a reader who knows the field but not this experiment. Define any
term specific to the experiment, or better, avoid it.
* Leave out the steps of the computation, which the viewer's graph already shows, and leave out
code-level details and how to run it (the project's `README.md` is for those).

All of this is drawn by the viewer's default renderers until the project adds its own, which can
also give the root workflow a results overview: a view of the run that presents its results. See
[Visualization and custom renderers](../concepts/visualization-and-custom-renderers.mdx).
Expand Down
Loading
Loading