Skip to content

feat(vm): host-driven suspend/resume for deflua and Elixir callbacks - #434

Open
dompdv wants to merge 5 commits into
tv-labs:mainfrom
dompdv:feat/deflua-suspend-resume
Open

dompdv wants to merge 5 commits into
tv-labs:mainfrom
dompdv:feat/deflua-suspend-resume

Conversation

@dompdv

@dompdv dompdv commented Oct 23, 2026

Copy link
Copy Markdown

Summary

Host-driven suspension/resumption of a Lua evaluation (plan A51).

A deflua or Elixir callback can return {:suspend, values} (or {:suspend, values, lua} to thread state) to park the running machine. Lua.eval!/2 and Lua.call_function/3 then return {:suspended, values, lua} with a continuation on the %Lua{}; the new Lua.resume/2 / Lua.resume!/2 re-enter the parked machine — the resume values become the suspended call's results inside Lua.

defmodule Scheduler do
  use Lua.API, scope: "scheduler"

  deflua yield(value), state do
    {:suspend, [value], state}
  end
end

lua = Lua.new() |> Lua.load_api(Scheduler)

{:suspended, [42], lua} =
  Lua.eval!(lua, "local x = scheduler.yield(42); return x + 1")

{:ok, [11], _lua} = Lua.resume(lua, [10])   # the suspended call "returns" 10

Design

  • The suspend travels as a Lua.VM.Suspend exception raised by the callback wrappers (execute_function/4, wrap_callback/3); the executor and dispatcher catch it at their native-call and engine-boundary sites and attach resumable machine layers (a stack, innermost first) to the exception, which crosses the tail-recursive BEAM frame to Lua.VM.execute/3 — the same channel VM errors already use to ferry raise-time state out. Zero happy-path cost: layers are built only in rescue blocks.
  • Lua.VM.Continuation.run/3 re-enters the layers innermost-first; the heap flows through the completion states, not the layers. A :max_instructions budget spans the whole suspend/resume cycle.
  • Heap effects made before a suspend are kept, as are effects made before a post-resume error (matching pcall semantics).
  • A suspend that cannot be resumed — across pcall/xpcall, a metamethod, a table.sort comparator, a string.gsub replacement function, a generic-for iterator, a load reader, a required module, or a re-entrant native call — is reported as the catchable Lua error "attempt to suspend across a native call boundary" (the Lua 5.2-style "yield across a C-call boundary" divergence). Full coroutine support remains a separate non-goal (coroutine.lua stays a documented suite skip).

Verification

  • mix test: 2917 passed (baseline 2870 + 44 new tests + 3 new doctests), 0 failures
  • mix test test/lua53_suite_test.exs --only lua53: 20/29 unchanged
  • mix compile --warnings-as-errors and mix format: clean
  • New test file: test/lua/suspend_resume_test.exs (44 tests: result placements, chained suspends, loops, dispatcher path, heap effects, error-after-resume, pcall/xpcall boundary, re-entrant deflua, call_function direct + pending_call, ArgumentError, encoded-value validation, :max_instructions across resume, decode: false, eval-between-suspend-and-resume, chunk clause, metamethod/comparator/gsub/iterator/load boundaries)

Plan: A51

dompdv and others added 5 commits October 23, 2026 13:44
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
A deflua or Elixir callback can now return {:suspend, values} (or
{:suspend, values, lua} to thread state) to park the running machine.
Lua.eval!/2 and Lua.call_function/3 return {:suspended, values, lua}
with a continuation on the %Lua{}; the new Lua.resume/2 and
Lua.resume!/2 re-enter the parked machine, the resume values becoming
the suspended call's results inside Lua. Heap effects made before a
suspend are kept, and a :max_instructions budget spans the whole
suspend/resume cycle.

The suspend travels as a Lua.VM.Suspend exception raised by the
callback wrappers; the executor and dispatcher catch it at their
native-call and engine-boundary sites and attach resumable machine
layers (a stack, innermost first) to the exception, which crosses the
tail-recursive BEAM frame to Lua.VM.execute/3 — the same channel VM
errors already use to ferry raise-time state out. Continuation.run/3
re-enters the layers innermost-first; the heap flows through the
completion states, not the layers.

A suspend that cannot be resumed — across pcall/xpcall, a metamethod,
a table.sort comparator, a string.gsub replacement function, a
generic-for iterator, a load reader, a required module, or a
re-entrant native call — is reported as the catchable Lua error
"attempt to suspend across a native call boundary".

Plan: A51

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
PR: tv-labs#434

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant