Skip to content

Lua: tuple returns as multiple results, hashtable natives printed in place - #1427

Open
Frotty wants to merge 6 commits into
masterfrom
lua-tuple-returns-hashtables
Open

Frotty wants to merge 6 commits into
masterfrom
lua-tuple-returns-hashtables

Conversation

@Frotty

@Frotty Frotty commented Oct 10, 2026

Copy link
Copy Markdown
Member

Two of the medium items from the Lua output audit: tuple returns, and the emulated hashtable natives.

Tuple returns as Lua multiple results

A function returning a tuple passed every component but the first through a global of the function, on Lua as on Jass: a global write per component in the function and a global read per component after each call. Lua has multiple results, so the function now returns its components together and a call takes them into locals of its own:

-- before
function cellCenter(side, idx)
	...
	cellCenter_return_x = temp603
	cellCenter_return_y = (cond_result49 + ((w9 + 0.5) * 64.))
	return cellCenter_return_x
end
c_x2 = cellCenter(side13, idx41)
c_y1 = cellCenter_return_y

-- after
function cellCenter(side, idx)
	...
	return temp603, (cond_result49 + ((w9 + 0.5) * 64.))
end
c_x2, c_y1 = cellCenter(side13, idx41)
  • EliminateTuples makes the shape on Lua. LuaMultipleResults states it and checks it after the elimination: a function returning two or more components has the flat tuple of them as its return type and returns tuple expressions of them; a call is a statement or the value of a results local, which only such calls write and only component selections read. To the optimizer a results local is an ordinary local which a call writes, so no pass needed to learn a new kind of assignment. The backend prints one Lua local per component, a multiple assignment for the call and a return of the values (two Lua AST nodes, LuaMultipleAssignment and LuaReturnValues).
  • A component no call reads used to vanish with its global. The garbage removal now drops such a result from every function of its group (the implementations a call can dispatch between, and the functions whose calls share a results local, as LocalMerger can merge two), keeping what the dropped value did in its place (LuaUnreadResults).
  • Copy propagation knew a variable or a constant as a value; a component of a results local is one too, until another call writes the local, so let p = f() reads the results where they were received instead of copying them.
  • Jass is unchanged.

Hashtable natives printed where they are called

LoadInteger, SaveInteger and the rest were Lua helper functions defined by literal bodies, so every call (about 219 loads and 218 saves in castle fight) cost a global lookup and a call, and the IM inliner could not see into them. They are now backend intrinsics like the KeyedMap operations, printed as the table operations they stand for:

x = (h.__wurst_ht_int[p] or __wurst_htEmpty)[c] or 0                 -- LoadInteger
;(h.__wurst_ht_int[p] or __wurst_htNewChild(h.__wurst_ht_int, p))[c] = v  -- SaveInteger
if h.__wurst_ht_int[p] then h.__wurst_ht_int[p][c] = nil end          -- RemoveSavedInteger
  • An operation is printed in place only when every operand is a literal, a variable or a read, because in place an operand can be read twice and after part of the body ran. Otherwise it stays the call of its helper, which is then defined, once. An instance id cast to int (this castTo int, how Table keys its hashtable) counts as its operand.
  • The loads and HaveSaved tests count as natives which only read, as they do on Jass (the __wurst_ rename had hidden them from that list), so an unused load is dropped.
  • __wurst_htEmpty (the shared child of an absent parent key, never written) and __wurst_htNewChild are main-chunk locals, declared when first used.
  • The printer separates an assignment whose target starts with ( from the statement before it with ;, which Lua otherwise joins to it as a call.
  • Behaviour is unchanged: a missing key reads the same defaults, a nil hashtable or a nil key on a store raises as the helpers did, and a missing string still reads nil.

Measurements

Castle fight and zombie defense built with -lua -inline -localOptimizations, against master 678bb436f:

castle fight zombie defense
uses of tuple return globals (_return_) 536 → 0 1,335 → 0
__wurst_LoadInteger( (calls and definition) 219 → 4 120 → 3
__wurst_SaveInteger( 218 → 2 72 → 2
SaveStr / HaveSavedInteger / FlushChildHashtable helper calls 22 / 9 / 17 → 0 / 0 / 3 93 / 14 / 10 → 0 / 0 / 0
global writes (SETTABUP) 18,826 → 18,511 18,090 → 17,346
global reads (GETTABUP) 88,251 → 87,706 77,205 → 76,585
static instructions 470,958 → 473,007 (+0.4%) 380,838 → 384,377 (+0.9%)

The static count grows because an operation printed in place is a few instructions more than a call; what runs is less. Stock Lua 5.3, loop overhead included: a load 73 → 47 ns, a save 78 → 30 ns, a call with a tuple of 2 results 68 → 41 ns, of 3 results 95 → 53 ns. Castle fight builds byte-identically twice and both scripts pass luac -p. Not run in game.

Checks

  • New LuaMultipleResultsTests (13) and LuaHashtableTests (9): output shape and run-time behaviour on the bundled Lua (dispatch, closures, nested tuples, recursion, discarded calls, a dropped result with an effect, an override group, the locals-table spill, a compile-time hashtable, operand evaluation order of the call form). Each was seen failing with its change undone.
  • DeterministicChecks.dispatchSignatureKeysDoNotDependOnIdentityHashes: fails on the old key. fsmSiblingsInSeparatePackagesBindRootSlotInAnyUnitOrder failed now and then on this branch (1 of 5 class runs; the two type variables printed the same identity hash T412 in the failing compilation); with the structural key the class passed 5 of 5 and the slot-binding classes (865 tests) pass.
  • 37 Lua-related test classes, 1,594 tests, before the last two commits; the 10 classes those touch (699 tests) after. CI runs the full suite.

Also in here

  • Dispatch slot key (LuaDispatchPreparation): methods of an override family are grouped into slots by the key of their signature, which was the printed type, and a type variable prints with an identity hash. The parameter T of State and the T of NoOpState therefore had different keys in most compilations and the same key in some, and the slot was named after either class. A type variable is now its owner, its position and its name. This branch did not cause it, but shifted the identity hashes enough to make it show.
  • An instance id cast to int (this castTo int) and an old-generics value of a variable cast to int count as the reads they print as, so Table, HashList and the old HashMap get the in-place forms.

Known gaps

  • A call whose results are returned unchanged is still received into locals and returned from them (a, b = f(); return a, b) rather than return f().
  • The stdlib's String package keys a hashtable by StringHash(c), a call, so those saves keep the helper call.

…bals

A function returning a tuple passed every component but the first through a
global of the function on Lua too: a global write per component in the
function, a global read per component after each call. Lua has multiple
results, so the function now returns its scalar components together
(return x, y) and a call takes them into locals of its own (a, b = f()).

EliminateTuples makes the shape on Lua (LuaMultipleResults states and checks
it): the function's return type is the flat tuple of its components, each
return returns a tuple expression of them, and a call is a statement or the
value of a results local, read only by component selections. To the optimizer
a results local is an ordinary local which a call writes. The backend prints
one Lua local per component, a multiple assignment for the call and a return
of the values.

The global of a component which no call read was garbage, value and all. The
garbage removal now drops such a result from every function of its group (the
implementations a call can dispatch between, and the functions whose calls
share a results local), keeping what the dropped value did in its place.
let p = f() becomes p_x = t.0; p_y = t.1 after the call, and the copies
stayed: copy propagation knew only a variable or a constant as a value.
A component of a results local is one too, until another call writes the
local, so the reads of p_x read t.0 and the copy is garbage.
Each emulated hashtable native was a call of a __wurst_ helper: a global
lookup and a call on every load and save. The backend now prints the table
accesses the helper consists of: a load is (h.T[p] or __wurst_htEmpty)[c]
with the default the helper answered, a save
(h.T[p] or __wurst_htNewChild(h.T, p))[c] = v, a remove tests the child
first, the flushes reset the subtables. An operation keeps the call of its
helper where an operand has an effect, since printed in place an operand is
read again or not at all; the helper is then defined, and only then.

LuaNativeLowering keeps the stubs in ImTranslator.luaHashtableStubs, so the
optimizer and the backend match them by identity (LuaHashtable). Loads and
HaveSaved tests count as reads (isLuaTableRead), so an unused one is dropped,
as the Jass natives already were. The helper bodies lose the tests for a
missing subtable, which InitHashtable always creates. The printer puts a ';'
before an assignment whose target starts with '(', which Lua would read as
the arguments of a call ending the previous line.
Table keys its hashtable by this castTo int. Inlined, that cast is an
operand of the hashtable native, and the store operand classification
counted every cast as one which may raise, so every inlined Table load, test
and remove kept the call of its helper. On Lua the cast is (x or 0), and a
cast from one class to another is x itself: neither raises nor changes
anything, so such a cast is classified as its operand is.
The methods of an override family are grouped into dispatch slots by the
key of their signature, which was the printed type. A type variable prints
with an identity hash (ImPrinter), so the parameter T of State and the T of
NoOpState had different keys in most runs and the same key in some, and the
FSM's slot was named after State in one compilation and after NoOpState in
the next (DeterministicChecks failed now and then). A type variable is now
its owner, its position there and its name.
…lace

HashList and the old HashMap key their hashtable by elem castTo int. On Lua
that cast of a variable is ((x == 0) and zero) or (x or 0), which only reads,
but the store operand classification counted it as an effect, so once the
loads counted as pure and the inliner passed the cast in directly, every
such load, test and store in zombie defense stayed a helper call (more than
before this branch).
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-10T13:13:35.671566Z 61a93f6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61a93f6138

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant