Skip to content

Pull requests: open-compass/VLMEvalKit

New pull request New
79 Open 980 Closed
79 Open 980 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

[Fix] Molmo AI2D: map no-letter answers to the option letter, not its…
#1655 opened Aug 24, 2026 by aarjunsrinivasan Loading…
[Benchmark] Add SIS-Bench video benchmark
#1654 opened Aug 24, 2026 by Saad1926Q Loading…
fix(dataset): preserve literal 'None' options in TSV loading and MCQ prompt building
#1651 opened Aug 23, 2026 by Xayar145 Loading…
fix(distributed): default LOCAL_RANK to 0 instead of 1 in run.py
#1650 opened Aug 23, 2026 by Xayar145 Loading…
feat(api): add OrcaRouter provider
#1648 opened Aug 21, 2026 by XiaoHuo888-hue Loading…
[Fix]Fix CCOCR evaluation for long text inputs
#1647 opened Aug 20, 2026 by XiaoBao-175 Loading…
[Benchmark] Add ActiveVision benchmark
#1646 opened Aug 20, 2026 by lzpp2598 Contributor Loading…
[Fix] Keep Thyme sandbox images in user observation turns
#1645 opened Aug 18, 2026 by reacher-z Loading…
1
[Feature] Add multi-turn chat for Llama 4, Llama Vision, and Qwen3-VL
#1644 opened Aug 18, 2026 by reacher-z Loading…
[Fix] Detect MIME types for extensionless images
#1643 opened Aug 18, 2026 by reacher-z Loading…
1
[Fix] Evaluate MMMU open-ended answers by content
#1642 opened Aug 18, 2026 by reacher-z Loading…
[Fix] Accept numeric LogicVista judge answers
#1641 opened Aug 18, 2026 by reacher-z Loading…
[Benchmark] Upgrade OmniDocBench to v1.5
#1640 opened Aug 18, 2026 by reacher-z Loading…
1
[Dataset] Add downloadable OmniMat data URLs
#1639 opened Aug 18, 2026 by yansumm Loading…
[Feature] Add multi-turn chat for Gemma 3, Mantis, and Pixtral
#1637 opened Aug 16, 2026 by reacher-z Loading…
Add Japanese Heron-Bench support
#1636 opened Aug 16, 2026 by reacher-z Loading…
1
1
[Feature] Add multi-turn chat for Kimi-VL, Aria, and Phi-4
#1635 opened Aug 16, 2026 by reacher-z Loading…
[Feature] Support evaluating selected dataset indices
#1634 opened Aug 16, 2026 by reacher-z Loading…
1
[Benchmark] Add support for CAPEval
#1633 opened Aug 14, 2026 by liuzhipenggg Loading…
2
[Model] Add VQToken for LLaVA-OneVision
#1625 opened Aug 11, 2026 by Hai-chao-Zhang Loading…
1
Add C4 Bench evaluation
#1624 opened Aug 10, 2026 by sci-m-wang Loading…
2
[Refactor] Refactor dataset config aliases and video presets
#1622 opened Aug 6, 2026 by ssiq Collaborator Loading…
[Fix] add missing case for answer regex in spatial457
#1620 opened Aug 4, 2026 by JackLangerman Loading…
1
[Fix] fix tqdm import in spatial457
#1619 opened Aug 4, 2026 by JackLangerman Loading…
1
[Model] Support VisionPsy-Nano-460M / -Flash (hub-packaged nanoVLM) with an optional in-process vLLM path
#1613 opened Jul 29, 2026 by cdpark92 Loading…
1
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.