Skip to content

docs: agent evaluation framework (stack 5/5) - #481

Open
MarioCadenas wants to merge 2 commits into
pr/agent-evals-4-suitefrom
pr/agent-evals-5-docs
Open

docs: agent evaluation framework (stack 5/5)#481
MarioCadenas wants to merge 2 commits into
pr/agent-evals-4-suitefrom
pr/agent-evals-5-docs

Conversation

@MarioCadenas

Copy link
Copy Markdown
Collaborator

Stack 5/5 · targets pr/agent-evals-4-suite (review after #4).

Documentation for the eval framework: a new "Evaluating agents" section in docs/docs/plugins/agents.md covering defineEval, assertions, judges, conversations, datasets, the full CLI flag reference, per-directory config, and MLflow reporting.

(Regenerated typedoc API docs will be added here too.)

@MarioCadenas
MarioCadenas requested a review from a team as a code owner July 16, 2026 14:26
@MarioCadenas
MarioCadenas requested review from calvarjorge and removed request for a team July 16, 2026 14:26
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

This pull request has had no activity for 23 days and has been marked as stale. It will be closed in 7 days if there is no further activity. Add a comment, push a commit, or apply the no-stale label to keep it open.

@github-actions github-actions Bot added the stale label Aug 9, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed because it had no activity for a month. Feel free to reopen it if you would like to continue the work.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit ⚠️ over budget

npm tarball (packed): 1.1 MB (+61 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 1.1 MB (+50 KB) 403 KB (+20 KB)
Type declarations 417 KB (+31 KB) 150 KB (+14 KB)
Source maps 2.2 MB (+102 KB) 755 KB (+40 KB)
Other 11 KB 3.7 KB
Total 3.8 MB (+183 KB) 1.3 MB (+74 KB)
Per-entry composition (own code — deps external (as shipped))
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
. 95 KB (-1 B) 2.5 KB 98 KB (-1 B) external 313 KB
./beta 89 KB (+12 KB) 455 B (-1 B) 89 KB (+12 KB) external 267 KB (+37 KB)
./testing 17 KB 0 B 17 KB external 51 KB
./tsdown 520 B 0 B 520 B external 813 B
./type-generator 23 KB 0 B 23 KB external 65 KB

Chunks:

Entry Chunk Load Size (gz)
. index.js initial 91 KB
. utils.js initial 4.0 KB
. remote-tunnel-manager.js lazy 2.5 KB
./beta beta.js initial 72 KB
./beta stream-manager.js initial 5.8 KB
./beta wide-event-emitter.js initial 3.2 KB
./beta databricks.js initial 3.2 KB
./beta configuration.js initial 2.2 KB
./beta service-context.js initial 1.3 KB
./beta client.js initial 434 B
./beta client-options.js initial 219 B
./beta supervisor-api.js lazy 192 B
./beta databricks.js lazy 141 B
./beta index.js lazy 122 B
./testing index.js initial 17 KB
./tsdown index.js initial 520 B
./type-generator index.js initial 23 KB

@databricks/appkit-ui

npm tarball (packed): 350 KB — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist raw gzip
JS (runtime) 395 KB 132 KB
Type declarations 229 KB 84 KB
Source maps 766 KB 253 KB
CSS 16 KB 3.2 KB
Total 1.4 MB 473 KB
Per-entry composition (consumer bundle — deps bundled, peerDeps external)
Entry Initial (gz) Lazy (gz) Total (gz) node_modules (min) Own code (min)
./js 5.3 KB 49 KB 55 KB 208 KB 14 KB
./js/beta 20 B 0 B 20 B 0 B 0 B
./react 432 KB 49 KB 481 KB 1.3 MB 177 KB
./react/beta 1.0 KB 0 B 1.0 KB 0 B 1.9 KB

Chunks:

Entry Chunk Load Size (gz)
./js index.js initial 5.2 KB
./js chunk initial 120 B
./js apache-arrow lazy 49 KB
./js/beta beta.js initial 20 B
./react index.js initial 430 KB
./react tslib initial 2.1 KB
./react apache-arrow lazy 49 KB
./react/beta beta.js initial 1.0 KB

⚠️ Over budget: a package's shipped tarball, or a browser entry's consumer bundle (deps included), grew by more than 5% (and >10 KB). This check will fail — reduce the size, or acknowledge the increase by updating bundle-size-baseline.json.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 33882127851 -R databricks/appkit -n appkit-template-0.71.0-pr.793552a-pr-agent-evals-5-docs-481 -D appkit-pr-481 \
  && unzip -o "appkit-pr-481/appkit-template-0.71.0-pr.793552a-pr-agent-evals-5-docs-481.zip" -d "appkit-pr-481" \
  && databricks apps init --template "appkit-pr-481"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.

@MarioCadenas
MarioCadenas force-pushed the pr/agent-evals-5-docs branch 2 times, most recently from 7ed45cc to e45529e Compare September 3, 2026 15:37
Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
…int parsing

Signed-off-by: MarioCadenas <MarioCadenas@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant