All pull requests

Pull requests

• feat(cli): add --case selector to run command

  area-clienhancement
  #492·liramon2 opened Oct 28, 2026Contributor·10 tasks done·

  ·
  8

• feat(skills): add locate-sessions step to session-experiment skill

  area-generatorsenhancement
  #491·liramon2 opened Oct 28, 2026Contributor·9 tasks done·

  ·
  14

• fix: prevent evaluator judges from invoking tools

  area-evaluatorsbug
  #490·mikemikimike opened Oct 28, 2026·

  ·
  1

• fix(types): discriminate user/assistant messages on role

  area-devxarea-tracingbug
  #488·ruserxd opened Oct 28, 2026·9 of 14 tasks·

  ·
  1

• fix(redteam): handle StructuredOutputException in strategy judges and attackers

  area-redteambug
  #479·kevinlupera opened Oct 26, 2026Contributor·9 tasks done·

  ·
  1

• fix(mappers): detect OpenInference spans by span.kind attribute

  area-tracingbug
  #478·nsknojj opened Oct 25, 2026·

  ·
  1

• feat(optimization): add the skeleton for optimization

  area-coreenhancement
  #475·nhungbi opened Oct 23, 2026Contributor·8 of 9 tasks·

  ·
  19

  

• fix(providers)!: support Langfuse v4 in LangfuseProvider

  area-tracingbug
  #469·fbalicchia opened Oct 21, 2026·9 tasks done·

  ·
  1

• fix(mappers): parse string-encoded timeUnixNano in CloudWatchSessionMapper

  area-tracingbug
  #461·kevinlupera opened Oct 20, 2026Contributor·9 tasks done·

  ·
  1

• feat: add versioned calibration profiles

  area-evaluatorsdesignenhancement
  #437·leonhuang29 opened Oct 16, 2026·

  ·
  2

  1

• feat: add experimental evaluator calibration

  area-evaluatorsdesignenhancement
  #429·leonhuang29 opened Oct 13, 2026·11 of 13 tasks·

  ·
  1

  40

• feat(providers): add HoneyHiveProvider

  area-tracingenhancement
  #419·sanjeed5 opened Oct 9, 2026·9 tasks done·

  ·
  1

• test: add integration tests for output-level judge evaluators

  area-evaluatorschore
  #407·AmitKulkarni23 opened Sep 29, 2026·9 tasks done·

  ·
  1

• feat: report judge context overflow as could-not-evaluate, not score-0

  area-corearea-evaluatorsenhancement
  #399·pdebjyot opened Sep 14, 2026Contributor·

  ·

• feat(output_evaluator): add cache_config support for prompt caching

  area-evaluatorsenhancement
  #397·shssrd opened Sep 11, 2026·

  ·
  8

• feat(tools): add KnowledgeIndex for progressive disclosure over reference knowledge

  area-evaluatorsenhancement
  #396·pdebjyot opened Sep 9, 2026Contributor·

  ·
  1

  1

• fix: expose and serialize user tools on TrajectoryEvaluator

  area-evaluatorsbug
  #394·AmirF194 opened Sep 7, 2026·8 tasks done·

  ·
  1

  1

• fix: make EvaluationReport.to_file() reject non-strict JSON

  area-corearea-devxbug
  #393·serenearyal opened Sep 4, 2026·9 tasks done·

  ·
  1

  2

• feat(providers): add session discovery + evaluate_sessions batch helper (#143)

  area-tracingenhancement
  #391·lifelmy opened Sep 1, 2026·4 tasks done·

  ·
  1

  3

• feat(store): add per-run keying and config-drift guard to EvaluationDataStore (#347)

  area-coreenhancement
  #390·lifelmy opened Sep 1, 2026·6 of 7 tasks·

  ·
  4

• feat(experiment): add ExperimentTrendAnalyzer for cross-run regression detection (#186)

  area-coreenhancement
  #389·lifelmy opened Sep 1, 2026·6 of 7 tasks·

  ·
  1

  3

• feat(classification): add classify_risk utility for multi-run results (#349)

  area-coreenhancement
  #388·lifelmy opened Aug 31, 2026·

  ·
  5

• feat(evaluators): add FuzzyEquals deterministic evaluator

  area-evaluatorsenhancement
  #387·lifelmy opened Aug 31, 2026·

  ·
  3

• ci: update langfuse requirement from <4,>=2.0.0 to >=2.0.0,<5

  area-communitychoredependenciespython
  #367·dependabot[bot] opened Aug 13, 2026Bot·

  ·
  2

• feat: add ToolEfficiencyEvaluator for session-level tool usage analysis

  area-evaluatorsenhancement
  #362·max-rattray-aws opened Aug 10, 2026Contributor·

  ·
  8