Add a hash map which keeps its keys as they are by Frotty · Pull Request #468 · wurstscript/WurstStdlib2

Frotty · 2026-08-17T21:44:20Z

HashMap casts every key to an int and stores it in a Table. That works for handles and for anything castable, but it loses the type on the way in: two keys which cast to the same int collide, and a key which is not castable cannot be used at all.

FastHashMap takes a bound on its key type instead, so hashing and comparing are done by the key's own implementation and the key is stored as itself:

implements Hashable<vec2>
    function hash(vec2 v) returns int
        return v.x.toInt() * 31 + v.y.toInt()
    function equals(vec2 a, vec2 b) returns boolean
        return a == b

let seen = new FastHashMap<vec2, unit>()
seen.put(caster.getPos(), caster)

Instances for int and string come with the package. Declaring one beside your own type is what makes it usable as a key.

Storage

One array per specialisation, carved into a section per instance, the way ArrayList works. A section is FASTHASHMAP_CAPACITY slots and does not grow; collisions probe linearly inside it, and a removed slot becomes a tombstone rather than empty so a probe which passed over it still finds keys put down beyond it.

A destroyed map hands its section back, so FASTHASHMAP_MAX_INSTANCES bounds maps alive rather than maps ever made. Every section is the same width, so released ones are a stack rather than the capacity-matched free list ArrayList keeps — any released section fits any new map, and there is nothing to compact. On the Jass target the configured total is also checked against JASS_MAX_ARRAY_SIZE, the storage being one fixed-size array.

The hash is computed here

Neither instance takes a shortcut, because a section fills up: colliding keys do not merely probe longer, the map eventually refuses them.

StringHash is unusable for this. It is case insensitive, so alpha and ALPHA would share a slot, and it collapses every partial multibyte slice to one constant. The string hash mixes each byte with its position and the length instead, so anagrams and prefixes separate too. Single bytes still decode through StringUtils.char, which recovers the case StringHash loses and which the library already depends on throughout; non-latin text is the remaining weak case, its lead bytes not decoding, and the doc says so.

The int hash returned the key itself at first, which sends everything strided by the capacity to slot zero — ids, handles and loop counters all arrive strided like that. Halves are mixed separately so the multiplications stay in range.

Every intermediate stays below 2^31 rather than relying on overflow, which wraps at 32 bits on Jass and does not on Lua. Nothing stores a hash, so differing values would not have been wrong, but the interpreter could then no longer stand in for the game while testing distribution.

Reporting and iteration

A full map reports a dropped write rather than discarding it silently — losing a store without a word is close to impossible to find from the outside. A key the map already holds stays writable, and isFull() asks beforehand.

Entries can be walked without allocating:

var slot = map.nextEntry(0)
while slot >= 0
    doSomething(map.keyAt(slot), map.valueAt(slot))
    slot = map.nextEntry(slot + 1)

Slot walking rather than an iterator object or a closure: nothing is allocated, and it avoids dispatching a bound through a closure, which is not supported on every target. A for in wrapper can be layered on this.

Compiler requirement

Type class bounds for T: generics, shipped in wurstscript/WurstScript#1226, #1228 and #1229. The bound is what makes the key's own hash and equals reachable from inside the generic, and it lowers to a direct call to the instance function after specialisation, on both Jass and Lua.

Testing

grill test reports 489/489, of which 25 are this package's. Beyond the obvious behaviour — put/get, replacement, missing keys reading as the value type's default, colliding keys, removal keeping later keys reachable, tombstone reuse, independent instances, two specialisations coexisting — the tests cover the three things that would otherwise pass quietly:

The hash itself, reached through a hashOf<T: Hashable> helper. The behavioural tests pass with an identity hash and with StringHash, because probing separates colliding keys however badly they hash; only observing the hash catches it. Against the old ones these fail with all sixteen strided keys in one slot and with alpha and ALPHA hashing alike.
Section reuse, by creating and destroying past the instance limit. Without it, that reports out of sections.
Iteration, including that tombstones are skipped and do not end the walk early.

Every test here was checked to fail before it passed.

Note

The container is also exercised from the compiler side in FastHashMapTests there — standalone and with the standard library in scope, on both targets — since it is the shape the bounds feature was built for. One gap worth stating: the standard-library-in-scope case is compiled but not executed on Lua, because the test runtime shim cannot initialise the library's own packages.

A later step could give Lua a native table instead of this probing, which would lift FASTHASHMAP_CAPACITY there; that is wurstscript/WurstScript#1255, and it is deliberately not part of this.

HashMap casts every key to an int and stores it in a Table. That works for handles and for anything castable, but it loses the type on the way in: two keys which cast to the same int collide, and a key which is not castable cannot be used at all. FastHashMap takes a bound on its key type instead, so hashing and comparing are done by the key's own implementation and the key is stored as itself. Instances for int and string come with the package; declaring one beside your own type is what makes it usable as a key. Storage is one array per specialisation carved into a section per instance, as ArrayList does it. A section is a fixed number of slots and does not grow, so a map which fills up refuses further keys rather than rehashing, and both the section size and the number of sections are configurable. Collisions probe linearly inside the section; a removed slot becomes a tombstone rather than empty, so a probe which passed over it still finds keys put down beyond it.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76b6d9ba72

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Three things, the first of which the build caught. The overview comment sat at package level with another doc comment after it, so it documented nothing and hotdoc rejected the position. It belongs on the class it describes, which is where LinkedList keeps its own. The section count was checked against the configured total and not against the array holding it. Raising FASTHASHMAP_CAPACITY while leaving FASTHASHMAP_MAX_INSTANCES alone - which the overview suggested doing - can put that total past JASS_MAX_ARRAY_SIZE, and sections were then handed out past the end of the array on a target where it cannot grow. Guarded as ArrayList.allocateStorage guards it, on the Jass target only, since Lua grows the table. A vacated slot kept the key and value it held. On Lua those are references the map no longer owns, so a removed entry and a cleared map held their last occupant for the lifetime of the specialisation. Released in both remove and clear, behind the same isLua branch ArrayList uses for the same reason; a tombstone is never read for its key, so this is safe. The overview also named FastHashMap_CAPACITY, which is not the constant.

Wurst re-exports MagicFunctions, which is where isLua comes from, but not ErrorHandling, so the two error() calls guarding a failed allocation resolved to nothing. The earlier hotdoc failure stopped compilation before name resolution ran, which is why this only surfaced once that was fixed. Verified against the whole library this time rather than pushed to find out: 474/474, including this package's fourteen.

Frotty · 2026-08-18T09:07:21Z

CI is green now. Three rounds, and both review findings are in.

Build failure (mine). The overview comment sat at package level with another doc comment after it, so it documented nothing and hotdoc rejected the position. Moved onto the class it describes, which is where LinkedList keeps its own. A second, masked error surfaced once that was fixed: Wurst re-exports MagicFunctions, which is where isLua comes from, but not ErrorHandling, so both error() calls resolved to nothing — the earlier failure had stopped compilation before name resolution ran.

P1, storage past the array bound — accepted. The section count was checked against the configured total and not against the array holding it, so raising FASTHASHMAP_CAPACITY while leaving FASTHASHMAP_MAX_INSTANCES alone — which the overview suggested doing — put that total past JASS_MAX_ARRAY_SIZE and handed out sections past the end of a fixed-size array. Guarded as ArrayList.allocateStorage guards it, on the Jass target only since Lua grows the table. The overview says so now, and no longer names a constant that does not exist.

P2, retention on Lua — accepted. A vacated slot kept the key and value it held, so a removed entry and a cleared map held their last occupant for the lifetime of the specialisation. Released in both remove and clear behind the same isLua branch ArrayList uses for the same reason; a tombstone is never read for its key, so it is safe.

Verified against the whole library rather than pushed to find out this time: grill test reports 474/474, with all fourteen of this package's test functions executing by name.

@codex review

chatgpt-codex-connector · 2026-08-18T09:10:00Z

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 6ea44a6ae6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Both instances leaned on something that collides on ordinary keys, and a section is FASTHASHMAP_CAPACITY slots wide, so a map whose keys collide refuses them once it fills rather than merely probing longer. StringHash is case insensitive, so alpha and ALPHA had one hash, and it collapses every partial multibyte slice to a single constant. The string hash now mixes each byte with its position and the length, so anagrams and prefixes separate too. Single bytes are still decoded through StringUtils.char, which recovers the case StringHash loses and which the library already depends on throughout; non-latin text stays the weak case, its lead bytes not decoding, and says so. The int hash returned the key itself, which sends everything strided by the capacity to slot zero - ids, handles and loop counters all arrive strided like that. Halves are mixed separately so the multiplications stay in range. Every intermediate stays below 2^31 rather than relying on overflow, which wraps at 32 bits on Jass and does not on Lua. Nothing stores a hash, so differing values would not have been wrong, but the interpreter could then no longer stand in for the game. The behavioural tests here pass with the old hashes as well, because probing separates colliding keys whatever they hash to. Reaching the requirement through hashOf is what observes the hash itself: against the old ones those tests fail with all sixteen strided keys in one slot, and with alpha and ALPHA hashing alike.

nextFree only ever grew and nothing was ever released, so FASTHASHMAP_MAX_INSTANCES counted maps ever made rather than maps alive. A map built per spell cast or per unit therefore exhausted the sections and every later one refused its keys with the section limit error - which is the ordinary way a map uses a container, so this had to be fixed before the container is usable in one. Every section is the same width, so released ones are a stack rather than the capacity-matched free list ArrayList keeps: any released section fits any new map, and there is nothing to compact. Emptied on the way out rather than on the way in, so the next map gets a clean section without paying for it and nothing keeps a reference the map no longer owns. ondestroy takes neither a hotdoc comment nor a return, hence the line comments and the inverted guard. The three tests fail without the reuse: creating past the instance limit reports out of sections, and the two checking that a reused section starts empty and that a live map keeps its own were written against that failure first.

Frotty · 2026-08-18T10:01:19Z

Two rounds since: a real hash, and the lifecycle gap that would have bitten in an actual map.

The hash is computed here now, not taken from StringHash or from identity. Both instances leaned on something that collides on ordinary keys, and a section is FASTHASHMAP_CAPACITY slots wide and fills up — so colliding keys do not merely probe longer, the map eventually refuses them. The string hash mixes each byte with its position and the length; single bytes still decode through StringUtils.char, which recovers the case StringHash loses and which the library already depends on throughout. The int hash mixed nothing at all, so everything strided by the capacity went to slot zero — ids, handles and loop counters all arrive strided like that. Every intermediate stays under 2^31 rather than relying on overflow, which wraps at 32 bits on Jass and does not on Lua.

Correction to something I wrote earlier in this PR: the interpreter does emulate StringHash — StringProvider delegates to Wc3StringHash, with Wc3StringHashTest checking parity against the Lua shim. The case insensitivity and the multibyte collapse are real and are the reason not to use it, but my claim that a test could not see them was wrong.

A destroyed map now hands its section back. nextFree only ever grew and nothing was released, so FASTHASHMAP_MAX_INSTANCES counted maps ever made rather than maps alive: a map creating one per spell cast or per unit exhausted the sections and every later one failed. That is the ordinary way a map uses a container, so it had to go before this is usable in one. Sections are all the same width, so released ones are a stack rather than the capacity-matched free list ArrayList keeps — any released section fits any new map, and there is nothing to compact.

On the tests. The first batch I wrote for the hash passed against the old hashes too, because probing separates colliding keys however badly they hash — coverage in name only. Reaching the requirement through a hashOf<T: Hashable> helper is what observes the hash itself, and against the old ones those fail with all sixteen strided keys in one slot and with alpha and ALPHA hashing alike. Same discipline for the lifecycle tests: with the reuse disabled they report out of sections.

grill test: 485/485, all 25 of this package's test functions.

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0521e2aba1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Two things a map needs before it is usable in one. put dropped a new key silently once the section was full. Losing a store without a word is close to impossible to find from the outside - the map simply does not have what you put in it - so it reports now. A key the map already holds stays writable, as before, and isFull is there to ask beforehand. The test which relied on the silent drop asks isFull instead. There was no way to see what a map holds. nextEntry walks to the next occupied slot and keyAt/valueAt read it, which allocates nothing - what a map iterating every frame needs - and avoids dispatching a bound through a closure, which is not supported on every target. A for-in wrapper can be built on top; this is the primitive under it. nextEntry skips tombstones, so a removed key is not visited and its slot does not stop the walk early. keyAt and valueAt report on a slot holding nothing rather than returning whatever the array has there. startSlot rather than from, from being reserved.

Frotty · 2026-08-18T10:06:29Z

Both gaps closed. grill test: 489/489.

A dropped write reports. put silently discarded a new key once the section was full — the map simply does not have what you put in it, which is close to impossible to find from the outside. It errors now. A key the map already holds stays writable, and isFull() is there to ask beforehand. testFullMapRefusesNewKeys relied on the silent drop, so it asks isFull instead of writing into a full map.

A map can be walked. There was no way to see what one holds:

var slot = map.nextEntry(0)
while slot >= 0
    doSomething(map.keyAt(slot), map.valueAt(slot))
    slot = map.nextEntry(slot + 1)

Slot walking rather than an iterator object or a closure: it allocates nothing, which is what a map iterating every frame needs, and it avoids dispatching a bound through a closure — a shape that is not supported on every target, as TypeClassTests.nestedClosuresInsideBoundedGenericAreRejectedForLua pins. A for in wrapper can be layered on this later; this is the primitive underneath it.

nextEntry skips tombstones, so a removed key is not visited and its slot does not end the walk early. keyAt/valueAt report on a slot holding nothing rather than handing back whatever the array happens to have there. Four tests cover it: every entry once, removed entries skipped, empty and cleared maps, and string keys iterating as themselves.

One naming note for the next reader: the parameter is startSlot because from is reserved.

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 629095c779

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Frotty · 2026-08-18T10:14:40Z

Description rewritten — it still described only the original container, and the branch has since grown a custom hash, section reuse on destroy, a reported rather than silent dropped write, and iteration. Six commits is a lot to reconstruct that from.

One thing worth stating explicitly for review: this branch predates #467, so a tip-to-tip diff shows PolygonTests.wurst as changed. It is not — against the merge base the PR touches exactly two files, FastHashMap.wurst and FastHashMapTests.wurst, which is what the Files tab shows. Nothing here reverts the polygon fix.

@codex review

chatgpt-codex-connector · 2026-08-18T10:18:35Z

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 629095c779

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Decoding each byte and mixing it cost about two native calls and three string comparisons per character: charAt is a SubString, and StringUtils.char is two comparisons, a StringHash and a round trip through the code table. Forty native calls for a twenty character key, against one for StringHash - not a trade a container called fast should make on every lookup. StringHash does the bytes now, and its two defects are patched at fixed cost instead. Two whole-string comparisons separate the cases which occur in practice, all lower, all upper and mixed, so alpha and ALPHA no longer share a hash; two mixed-case spellings of one word still collide and are separated by probing. Length is mixed in, which separates non-latin keys of different length where the multibyte collapse would give them one raw hash; same-length ones still collide and fall back on equals. Order sensitivity comes from StringHash itself. The tests still hold: against plain StringHash the case test fails.

… and say the limit counts live maps A FASTHASHMAP_CAPACITY of zero or less built a map whose first lookup took hash mod 0 and ended the thread; construction now reports it, through a pure sectionProblem that also holds the out-of-sections and Jass array-limit checks, so the three are tested by changing its arguments. On Lua error() returns, so keyAt on a slot that holds nothing went on to answer keys[base], whichever live key sat there; it answers a never-written key now, and verifyFastHashMapLuaGuardsIngame checks that where error() returns. The comment on FASTHASHMAP_MAX_INSTANCES still said sections are never reclaimed, which stopped being true when destroying a map hands its section back.

Frotty · 2026-10-01T13:56:11Z

@codex review

chatgpt-codex-connector · 2026-10-01T13:56:25Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-01T13:59:12.516701Z	`7b22de1`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector · 2026-10-01T13:59:11Z

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 7b22de1697

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Frotty · 2026-10-01T14:49:47Z

Closing. On Lua, where the performance matters, this is slower than the old-generic HashMap it set out to beat. Measured on the 3.0.0.24268 client in a test map, building, hitting, missing and churning maps of 8 and 24 keys under int, string and unit keys, it came out at about 0.5x overall: int keys 1.05x to 2.5x slower, string keys 3x to 8x slower. That is structural: hashing and probing written in Wurst can not compete with a table the compiler lowers to directly, and KeyedMap / FastKeyedMap already are that, through compiler intrinsics with no indirection, so there is nothing left for this to add on Lua.

chatgpt-codex-connector Bot reviewed Aug 17, 2026

View reviewed changes

Comment thread wurst/data/FastHashMap.wurst

Comment thread wurst/data/FastHashMap.wurst

Frotty added 2 commits August 18, 2026 10:28

Frotty added 2 commits August 18, 2026 11:49

chatgpt-codex-connector Bot reviewed Aug 18, 2026

View reviewed changes

Comment thread wurst/data/FastHashMap.wurst

Comment thread wurst/data/FastHashMap.wurst Outdated

chatgpt-codex-connector Bot reviewed Aug 18, 2026

View reviewed changes

Comment thread wurst/data/FastHashMap.wurst Outdated

Frotty added 3 commits August 18, 2026 13:18

Merge remote-tracking branch 'origin/master' into fhm-review

7b22de1

Frotty closed this Oct 1, 2026

Conversation

Frotty commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Storage

The hash is computed here

Reporting and iteration

Compiler requirement

Testing

Note

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Frotty commented Aug 18, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 18, 2026

Uh oh!

Frotty commented Aug 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Frotty commented Aug 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Frotty commented Aug 18, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 18, 2026

Uh oh!

Frotty commented Oct 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot commented Oct 1, 2026

Uh oh!

Frotty commented Oct 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Frotty commented Aug 17, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 1, 2026 •

edited

Loading