Skip to content

vpage: fix region size underflow, free tracking, and enable vpages on NVL-S - #11303

Open
lgirdwood wants to merge 2 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow
Open

lgirdwood wants to merge 2 commits into
thesofproject:mainfrom
lgirdwood:fix/vpage-region-underflow

Conversation

@lgirdwood

Copy link
Copy Markdown
Member

Fixes the NVL-S vpage_alloc failed -22 ... total 1048294 free 1048294 seen in the daily-nvl-s CI run.

Root cause

vpage_init() computed the region size as an unsigned subtraction L2_end - (unused_start + virtual_heap + library). On NVL-S (2MB L2, default 0x140000 heap + 0x100000 library) that underflows by 0x11a000 into a 0xFFEE6000 byte (1048294 page) region whose end wraps the 32-bit address space. sys_mm_drv_map_region_safe() then fails virt + size <= region_end for every address, so every vpage_alloc() returns -EINVAL and vregion falls back to the heap. It is not a leak: the first page (0xa023a000) is always chosen because nothing is ever allocated.

Changes

  1. Bounds-check the layout instead of underflowing; if nothing fits, leave vpages disabled and fail cleanly with -ENODEV.
  2. Enable vpages on small-L2 platforms: the virtual heap and library regions are only virtual address space, so when they do not fit in L2 place the vpage region after the virtual heap, end it at CONFIG_LIBRARY_BASE_ADDRESS (or end of virtual memory), capped by the physical L2 left after the image. Layouts that fit in L2 are unchanged.
  3. Fix vpages_free_and_unmap() moving velems[num_elems_in_use] (one past the last element) over the released slot. Freeing a non-last allocation orphaned the last element; the next alloc overwrote its tracking entry and it could never be freed (leaked virtual + physical pages).
  4. Release pages already mapped when a multi-page map fails; the mm driver leaves them mapped, leaking physical L2.
  5. Add vpage_free_out_of_order boot test.

Testing

  • Builds for intel_adsp/ace40/nvls (xt-clang, full link) and the vpage object for nvl and ptl.
  • For the NVL-S layout this gives a non-empty region (232 pages with the xt-clang image, ~294 with the CI image); previously it underflowed.
  • NOT yet run on hardware: needs a boot on ba-nvls-rvp to confirm the vpage: region ... log line, the boot test, and check-playback-all-formats.
  • Note: the vpage errors may not be the cause of the 25-test CI failure cascade (vregion has a heap fallback); that needs the mtrace checked for heap exhaustion.

Draft until tested on NVL-S.

🤖 Generated with Claude Code

@lgirdwood

Copy link
Copy Markdown
Member Author

@harajend fyi - pls let me know if this helps.

@lgirdwood
lgirdwood marked this pull request as ready for review October 13, 2026 17:10
Copilot AI balanced review requested due to automatic review settings October 13, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Region boundary handling still has edge-case bugs, and the new regression test does not detect the original leak.

3 open findings
What changed in this PR

Fixes vpage sizing and allocation tracking, including NVL-S support.

Changes:

  • Prevents virtual-region size underflow.
  • Cleans up partially mapped pages and fixes out-of-order frees.
  • Adds an out-of-order free boot test.
File Description
zephyr/​lib/​vpage.c Updates region sizing, cleanup, and allocation tracking.
zephyr/​test/​vpage.c Adds an out-of-order free regression test.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread zephyr/lib/vpage.c Outdated
Comment thread zephyr/lib/vpage.c
Comment thread zephyr/test/vpage.c Outdated
lrgirdwo and others added 2 commits October 13, 2026 19:17
vpage_init() computed the vpage region size as an unsigned subtraction
without checking that the image, virtual heap and library regions leave
any L2 for vpages. On platforms with a small L2 (NVL-S: 2MB, with the
default 0x140000 virtual heap and 0x100000 library region) this
underflows into a ~4GB region whose end wraps the 32-bit address space.
sys_mm_drv_map_region_safe() then rejects every mapping with -EINVAL, so
every vpage_alloc() fails and vregion creation falls back to the heap.

Check that the reserved areas fit before adding the region. If not, log
an error and leave vpages disabled; vpage_alloc()/vpage_free() now fail
cleanly with -ENODEV instead of dereferencing a missing region.

Also fix vpages_free_and_unmap() moving velems[num_elems_in_use], which
is one past the last in-use element, over the released slot. Freeing a
non-last allocation left the last element outside the in-use count, so
the next allocation overwrote its tracking entry and that allocation
could never be freed, leaking its virtual and physical pages. Decrement
the count first so the real last element is moved.

Add a boot test that frees allocations out of order.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The vpage region assumed that the virtual heap and library regions are
carved out of physical L2 in addition to the image. On NVL-S (2MB L2)
they do not fit, so vpages were unavailable. Those regions are only
virtual address space backed on demand, so they don't need to fit in L2.

When they do not fit, place the vpage region right after the virtual
heap and end it at the library base address (or the end of virtual
memory), capped by the physical L2 left after the image. For the NVL-S
CI layout this gives a 294 page region at 0xa023a000. Layouts where
everything fits in L2 are unchanged.

Also release pages already mapped by a failed multi-page map. The mm
driver maps page by page and leaves earlier pages and their physical
pages mapped on failure, which leaked physical L2 when it ran out.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
@lgirdwood
lgirdwood force-pushed the fix/vpage-region-underflow branch from 6115723 to b6f127a Compare October 13, 2026 18:17
@intel-sofci

intel-sofci commented Oct 13, 2026 •

Copy link
Copy Markdown

PR 11303: test results

Run date: 2026-10-14 07:36 UTC

Tested commit: b6f127a467cf69bff8316170c8910770bd21d50c

mtl pass rate lnl pass rate ptl pass rate wcl pass rate nvl pass rate

@harajend

harajend commented Oct 15, 2026 •

Copy link
Copy Markdown

@harajend fyi - pls let me know if this helps.

@lgirdwood

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01, ba-nvls-rvp-02).

This issue is not fixed. All playback/capture tests still fail on both
boards with a DSP firmware crash (EXCCAUSE 29 store prohibited) triggered
on the first pipeline-create IPC, halting DSP core 0 with no recovery.

Detailed analysis:

CI failure: DSP firmware crash on NVL-S (regression vs. previously-reported bug)

Tested commit b6f127a46 via intel-innersource CI run
38693960124
on both NVL-S DUTs (ba-nvls-rvp-01 SoundWire, ba-nvls-rvp-02 HDA).

Result: All playback/capture-dependent test cases fail on both boards with an
identical, reproducible DSP crash triggered by the very first pipeline-create IPC:

<inf> [core 0] vregion.vregion_create: new at base 0xa023a000 size 0x12000 pages 18 metadata at 0x400fdd80
<inf> [core 0] ipc.ipc_cmd: rx: 0x40000004|0x15
<err> [core 0] os.print_fatal_exception:  ** FATAL EXCEPTION
<err> [core 0] os.print_fatal_exception:  ** CPU 0 EXCCAUSE 29 (store prohibited)
<err> [core 0] os.print_fatal_exception:  **  PC 0xa007ef38 VADDR 0xa0031050
...
<err> [core 0] os.z_fatal_error: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
<err> [core 0] zephyr.k_sys_fatal_error_handler: Halting system

Identical PC/backtrace/address on both boards → deterministic firmware bug, not
hardware/board-specific noise. DSP core 0 halts and never recovers (unlike a
suspend/resume cycle, which does not bring it back here).

This looks like a regression against a previously-filed bug, not a fix

We have an existing internal bug report (BUG_REPORT_NVL-S_vpage_alloc.md,
CI run 38194969857, firmware beeac4752) describing a failure at the
exact same virtual region 0xa023a000:

<err> [core 0] vpage.vpages_alloc_and_map: error: failed to map virtual region 0xa023a000 ..., error -22
<err> [core 0] vpage.vpage_alloc: vpage_alloc failed -22 for N pages, total 1048294 free 1048294
<err> [core 0] pipe.pipeline_new: Failed to create pipeline vregion of X bytes, using heap

In that prior report, the failure was graceful: vpage_alloc() returned
-22 (EINVAL), the firmware fell back to heap allocation, and the DSP stayed
alive (recoverable via a suspend/resume D0→D3→D0 cycle).

With this PR, the same 0xa023a000 region now produces a hard CPU
exception (store prohibited) and a full core halt
, with no recovery path
short of a host reboot.

Hypothesis: this PR enables actually mapping/using the vpage region
instead of returning EINVAL, but something writes into 0xa023a000 /
VADDR 0xa0031050 without the mapping having succeeded (or before it's
committed), so the previously-tolerated error case is now a hard store fault
instead of a handled error return.

Suggested next step

Please check the map-then-use ordering / error handling around
vregion_create() / vpages_alloc_and_map() for the small-L2 (NVL-S) path —
specifically whether the metadata write (metadata at 0x400fdd80) or first
use of the new vregion happens before/without confirming the map succeeded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants