Skip to content

03_Python_API.md

Latest commit

 

History

History
120 lines (86 loc) · 2.75 KB

03_Python_API.md

File metadata and controls

120 lines (86 loc) · 2.75 KB
Raw

Python API Tutorial

This tutorial shows how to integrate RESON into Python workflows and scripts.

For a full class and method reference (Manifest, RESON, RESONRun, providers, reports), see docs/api.md.

Imports

from reson import RESON, RESONRun, Manifest

1) Load and inspect manifest

manifest = Manifest.load_from("tutorials/data/candidate.json")
print(manifest.summary)

2) Normalize selected fields

normalized = manifest.normalize(
    fields=["text", "prediction"],
    remove_punct=True,
    lowercase=True,
    replace_yo=True,
)

To apply a substitution dictionary, pass an explicit CSV path:

normalized = manifest.normalize(
    fields=["text", "prediction"],
    remove_punct=True,
    lowercase=True,
    replace_yo=True,
    subst_table="tutorials/data/subst_table/example.csv",
)

3) Analyze a run

reson = RESON(
    name="candidate_model",
    metrics=("WER", "CER"),
    meta={"model_version": "candidate_v1"},
)

run = reson.analyze(normalized, artifacts_dir="artifacts/python_api/candidate")
run.to_json("artifacts/python_api/candidate.json")

Inspect generated structures:

print(run.metrics)
print(run.summary)
print(run.vocab.data.head())
print(run.alphabet.data.head())

4) Build a single report from Python

Ready-made reports you can open in a browser before running this step: single, compare, leaderboard.

html_path = run.report(output_html="artifacts/python_api/candidate_report.html")
print(html_path)

5) Compare two runs

baseline_manifest = Manifest.load_from("tutorials/data/baseline.json")
baseline_run = RESON(name="baseline_model", metrics=("WER", "CER")).analyze(
    baseline_manifest.normalize(["text", "prediction"]),
    artifacts_dir="artifacts/python_api/baseline",
)

comparison = reson.compare(baseline=baseline_run, candidate=run)
print(comparison.metrics_delta_records())

6) Generate compare HTML report

reson.compare_report(
    runs=[run, baseline_run],
    model_names=["candidate_model", "baseline_model"],
    output_html="artifacts/python_api/compare_report.html",
    base_model="baseline_model",
    target_model="candidate_model",
    num_bootstraps=2000,
    alpha=0.05,
)

7) Restore run from JSON

restored = RESONRun.from_json("artifacts/python_api/candidate.json")
print(restored.name)
print(restored.metrics.head())

Practical patterns

  • Keep one RESON instance per model configuration.
  • Store run metadata in meta to make reports self-descriptive.
  • Persist every important run as JSON (RESONRun.to_json) for reproducible comparisons.