Repository navigation
Add standalone OpenVINO EPContext compilation support - #2732
Open
Xiaoyu Z (xiaoyu-work) wants to merge 3 commits into
Open
Xiaoyu Z (xiaoyu-work) wants to merge 3 commits into
Xiaoyu Z (xiaoyu-work) wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Workflow initialization can reject OpenVINO before compilation, and device matching and failed-output cleanup need correction.
3 open findings
What changed in this PR
Adds standalone OpenVINO EPContext compilation to Olive’s existing context-generation pass, without model packaging.
Changes:
- Adds OpenVINO AOT/JIT compilation, device filtering, and virtual targets.
- Validates generated context artifacts while preserving QNN behavior.
- Documents configuration and adds focused tests.
| File | Description |
|---|---|
| test/passes/onnx/test_context_binary.py | Tests OpenVINO options, registration ownership, and output validation. |
| olive/passes/onnx/context_binary.py | Adds OpenVINO compilation and explicit provider dispatch. |
| olive/olive_config.json | Declares OpenVINO and CPU/GPU/NPU support. |
| docs/source/reference/pass.rst | Documents standalone compilation and limitations. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Describe your changes
Extract the OpenVINO plugin EP context-compilation support from #2717 into a standalone change to
EPContextBinaryGenerator. This PR intentionally excludesOpenVINOModelPackage, package manifests, shared assets, and multi-device package assembly._run_qnnand_run_openvino; preserve QNN's existing session-based compilation, composite/weight-sharing behavior, metadata, and optional GenAI config updates.ModelCompiler, and JIT viaInferenceSession.embed_context,disable_cpu_fallback, and typed logging options. Parse JSON boolean/compiler enum values correctly and avoid redundant AOT session entries already controlled by the compile API.Review fixes
0xprefix, leading zeros, and integer IDs.Validation
& .\.venv\Scripts\python.exe -m pytest test\passes\onnx\test_context_binary.py test\common\test_ort_inference.py test\systems\test_local.py -q— passed: 72 tests; 4 QNN hardware tests skipped because QNNExecutionProvider is unavailable.lintrunner olive\common\ort_inference.py olive\passes\onnx\context_binary.py test\common\test_ort_inference.py test\passes\onnx\test_context_binary.py docs\source\reference\pass.rst— passed, including PYLINT.device_id: 0x00B03E.Release note:
EPContextBinaryGeneratornow supports standalone OpenVINO plugin EP compilation through ORT's AOT/JIT APIs, including automatic Windows LocalSystem plugin discovery, virtual target selection, embedded/external context data, and clean retries after failed compilation.Checklist before requesting a review
lintrunner -a(Optional) Issue link
Extracted compilation support from #2717; the original packaging PR remains separate.