All pull requests

Pull requests

• docs: fix relative links with wrong directory depth

  #1600·pratikgx opened Oct 26, 2026·1 of 8 tasks·

  ·

• fix(babyvision): don't count judge failures as wrong answers

  #1597·FazeelUsmani opened Oct 23, 2026·2 of 8 tasks·

  ·

• sparc_visual: state the negative polyshape rule as the puzzles use it

  #1596·lkaesberg opened Oct 22, 2026Contributor·2 of 8 tasks·

  ·
  1

• fix(mathvision): restore math names when evaluating latex2sympy output

  #1595·mottopanikeiku opened Oct 22, 2026Contributor·2 of 8 tasks·

  ·
  1

• fix(openai): disable SDK retries to avoid nested retry loops

  #1593·sangk-kim opened Oct 19, 2026·2 of 8 tasks·

  ·

• feat(tasks): add AI2D task with per-question images

  #1590·jiahaozheng406 opened Oct 17, 2026·3 of 8 tasks·

  ·
  1

• feat: add Qwen3.6 model support

  #1585·devledas opened Oct 16, 2026Contributor·2 of 8 tasks·

  ·

• fix(whisper_vllm): recover LibriSpeech early-EOT

  #1581·harshaladhav-amd opened Oct 14, 2026·2 tasks done·

  ·
  1

• feat(openai): add video_target_frames and video_max_frames for server-side video decoding

  #1571·sahirema opened Oct 9, 2026·2 of 8 tasks·

  ·

• Fix typos in model comments and example scripts

  #1569·Jacky1128 opened Oct 7, 2026Contributor·

  ·

• Massive fixing of registed → registered

  #1568·Jacky1128 opened Oct 7, 2026Contributor·

  ·

• feat(mathvision): add mathvision_relaxed_eval with the four extraction fixes

  #1552·feiiiiii5 opened Oct 1, 2026Contributor·2 tasks done·

  ·

• fix(vllm_generate): pass video frames to vLLM as uint8 (T, H, W, C)

  #1544·jbyczkow opened Sep 22, 2026Contributor·1 task done·

  ·
  1

• feat: add group-level postprocess hook for custom result aggregation

  #1528·lorenzozanee opened Sep 15, 2026Contributor·7 tasks·

  ·
  1

• fix(qwen2_5_vl): propagate sampled fps to processor for correct second_per_grid_ts

  #1523·lorenzozanee opened Sep 15, 2026Contributor·1 task done·

  ·
  1

• Add DIVE-Bench and isolated GRT reproduction profiles

  #1521·Hai-chao-Zhang opened Sep 14, 2026·3 of 4 tasks·

  ·

• perf: (PR 4/4) add reproducible evaluator benchmark harness

  #1460·Luodian opened Aug 23, 2026Contributor·4/4·1 of 7 tasks·

  ·

• perf: (PR 3/4) instrument evaluation phases

  #1459·Luodian opened Aug 23, 2026Contributor·3/4·1 of 7 tasks·

  ·

• perf: (PR 2/4) atomically write baseline records

  #1457·Luodian opened Aug 23, 2026Contributor·2/4·1 of 7 tasks·

  ·

• perf: (PR 1/4) add baseline phase recorder

  #1456·Luodian opened Aug 23, 2026Contributor·1/4·1 of 7 tasks·

  ·

• perf: (PR 4/4) capture baseline provenance

  #1455·Luodian opened Aug 23, 2026Contributor·4/4·1 of 7 tasks·

  ·

• perf: (PR 3/4) add secret-safe baseline invocation

  #1454·Luodian opened Aug 23, 2026Contributor·3/4·1 of 7 tasks·

  ·

• perf: (PR 2/4) add fail-closed legacy redaction

  #1453·Luodian opened Aug 23, 2026Contributor·2/4·1 of 7 tasks·

  ·

• perf: (PR 1/4) add RFC 8785 JSON primitives

  #1452·Luodian opened Aug 23, 2026Contributor·1/4·1 of 7 tasks·

  ·

• refactor(models): (collector) make ModelRegistryV2 authoritative

  #1450·Luodian opened Aug 23, 2026Contributor·3 of 7 tasks·

  ·