Skip to content

Support string keys in FastKeyedMap #1408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Frotty merged 2 commits into from
Oct 9, 2026
Merged

Support string keys in FastKeyedMap #1408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b5dcb1e
Support FastKeyedMap string-key intrinsics (PoC)
Frotty Oct 9, 2026
a691fc3
Report string-key signatures for malformed map fallbacks
Frotty Oct 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions ...script/src/main/java/de/peeeq/wurstscript/intermediatelang/interpreter/ILInterpreter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,21 +233,31 @@ public static LocalState runFunc(ProgramState globalState, ImFunction f, @Nullab
// through the int fallback, with the value in its integer representation.
if (JassKeyedMapLowering.isUnloweredPutNative(f)) {
// Looked up first: it checks the intrinsic's shape before any argument is read.
ImFunction put = JassKeyedMapLowering.fallbackOf(globalState.getProg(), f);
ImFunction put = JassKeyedMapLowering.fallbackOf(globalState.getProg(), f,
f.getParameters().size() > 1
? keyedMapCallType(globalState, f, caller, f.getParameters().get(1).getType()) : JassIm.ImVoid());
JassKeyedMapLowering.checkSpecialization(f,
keyedMapCallType(globalState, f, caller, f.getParameters().get(1).getType()),
keyedMapCallType(globalState, f, caller, f.getParameters().get(2).getType()));
runFunc(globalState, put, caller, args[0], args[1], keyedMapIntValue(globalState, args[2]));
return new LocalState();
}
if (JassKeyedMapLowering.isUnloweredGetNative(f)) {
ImFunction get = JassKeyedMapLowering.fallbackOf(globalState.getProg(), f);
ImFunction get = JassKeyedMapLowering.fallbackOf(globalState.getProg(), f,
f.getParameters().size() > 1
? keyedMapCallType(globalState, f, caller, f.getParameters().get(1).getType()) : JassIm.ImVoid());
ImType valueType = keyedMapCallType(globalState, f, caller, f.getReturnType());
JassKeyedMapLowering.checkSpecialization(f,
keyedMapCallType(globalState, f, caller, f.getParameters().get(1).getType()), valueType);
ILconst stored = runFunc(globalState, get, caller, args[0], args[1]).getReturnVal();
return new LocalState(keyedMapValueOfInt(globalState, stored, valueType));
}
if (JassKeyedMapLowering.isUnloweredMembership(f)) {
ImFunction fallback = JassKeyedMapLowering.membershipFallback(globalState.getProg(), f,
f.getParameters().size() > 1
? keyedMapCallType(globalState, f, caller, f.getParameters().get(1).getType()) : JassIm.ImVoid());
return runFunc(globalState, fallback, caller, args);
}

// --- local state & bind parameters ---
LocalState localState = new LocalState();
Expand Down
93 changes: 76 additions & 17 deletions ...pt/src/main/java/de/peeeq/wurstscript/translation/imtranslation/JassKeyedMapLowering.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ public static void transform(ImProg prog) {
lowerPut(prog, f);
} else if (GET_NATIVE.equals(name)) {
lowerGet(prog, f);
} else if ("keyedMapHasNative".equals(name) || "keyedMapRemoveNative".equals(name)) {
lowerMembership(prog, f);
}
}
}
Expand All @@ -50,48 +52,93 @@ public static boolean isUnloweredGetNative(ImFunction f) {
return !f.getTypeVariables().isEmpty() && GET_NATIVE.equals(intrinsicName(f));
}

public static boolean isUnloweredMembership(ImFunction f) {
return !f.getTypeVariables().isEmpty()
&& ("keyedMapHasNative".equals(intrinsicName(f)) || "keyedMapRemoveNative".equals(intrinsicName(f)));
}

public static ImFunction membershipFallback(ImProg prog, ImFunction f, ImType keyType) {
boolean has = "keyedMapHasNative".equals(intrinsicName(f));
if (f.getParameters().size() != 2 || !TypesHelper.isIntType(f.getParameters().get(0).getType())
|| !(has ? TypesHelper.isBoolType(f.getReturnType()) : f.getReturnType() instanceof ImVoid)) {
throw invalidSpecialization(f, "KeyedMap membership requires an int map and a handle or string key");
}
if (!supportedKey(keyType)) {
throw invalidSpecialization(f, "KeyedMap supports only handle and string keys");
}
String name = has ? "keyedMapHas" : "keyedMapRemove";
if (TypesHelper.isStringType(keyType)) {
name += "String";
}
for (ImFunction candidate : prog.getFunctions()) {
if (name.equals(intrinsicName(candidate)) && packageOf(candidate) == packageOf(f)
&& candidate.getParameters().size() == 2
&& TypesHelper.isIntType(candidate.getParameters().get(0).getType())
&& sameKeyRepresentation(candidate.getParameters().get(1).getType(), keyType)
&& (has ? TypesHelper.isBoolType(candidate.getReturnType()) : candidate.getReturnType() instanceof ImVoid)) {
return candidate;
}
}
throw invalidSpecialization(f, "The package must declare the " + name + " Jass fallback");
}

private static void lowerMembership(ImProg prog, ImFunction f) {
ImType keyType = f.getParameters().size() > 1 ? f.getParameters().get(1).getType() : JassIm.ImVoid();
ImFunction fallback = membershipFallback(prog, f, keyType);
ImExpr call = JassIm.ImFunctionCall(f.attrTrace(), fallback, JassIm.ImTypeArguments(),
JassIm.ImExprs(JassIm.ImVarAccess(f.getParameters().get(0)), JassIm.ImVarAccess(f.getParameters().get(1))),
false, CallType.NORMAL);
f.getBody().clear();
f.getLocals().clear();
f.getBody().add(f.getReturnType() instanceof ImVoid ? call : JassIm.ImReturn(f.attrTrace(), call));
}

/**
* The int fallback that the lowered put or get calls, found the way this pass finds it. Checks the
* (map, key[, value]) shape first and raises this pass's diagnostic when it is wrong, so running the
* program before the lowering cannot bypass it.
*/
public static ImFunction fallbackOf(ImProg prog, ImFunction f) {
return fallbackOf(prog, f, f.getParameters().size() > 1 ? f.getParameters().get(1).getType() : JassIm.ImVoid());
}

public static ImFunction fallbackOf(ImProg prog, ImFunction f, ImType keyType) {
boolean put = PUT_NATIVE.equals(intrinsicName(f));
boolean shaped = f.getParameters().size() == (put ? 3 : 2)
&& TypesHelper.isIntType(f.getParameters().get(0).getType())
&& (!put || f.getReturnType() instanceof ImVoid);
if (!shaped) {
throw invalidSpecialization(f, put
? "keyedMapPutNative requires an int map, a handle key, and an int-represented value"
: "keyedMapGetNative requires an int map, a handle key, and an int-represented result");
? "keyedMapPutNative requires an int map, a handle or string key, and an int-represented value"
: "keyedMapGetNative requires an int map, a handle or string key, and an int-represented result");
}
return findFallback(prog, packageOf(f), f, put ? PUT : GET_INT, put);
return findFallback(prog, packageOf(f), f, put ? PUT : GET_INT, put, keyType);
}

/**
* Raises this pass's diagnostic unless a call's resolved types are the ones it lowers: a handle
* Raises this pass's diagnostic unless a call's resolved types are the ones it lowers: a handle or string
* key, and a value with Jass's integer representation, an int or a class reference. The
* interpreter checks each call with it, so a specialization the Jass build rejects cannot pass
* there.
*/
public static void checkSpecialization(ImFunction f, ImType keyType, ImType valueType) {
if (LuaNativeLowering.isHandleType(keyType)
if (supportedKey(keyType)
&& (TypesHelper.isIntType(valueType) || valueType instanceof ImClassType)) {
return;
}
throw invalidSpecialization(f, PUT_NATIVE.equals(intrinsicName(f))
? "keyedMapPutNative requires an int map, a handle key, and an int-represented value"
: "keyedMapGetNative requires an int map, a handle key, and an int-represented result");
? "keyedMapPutNative requires an int map, a handle or string key, and an int-represented value"
: "keyedMapGetNative requires an int map, a handle or string key, and an int-represented result");
}

private static void lowerPut(ImProg prog, ImFunction f) {
if (f.getParameters().size() != 3 || !(f.getReturnType() instanceof ImVoid)
|| !TypesHelper.isIntType(f.getParameters().get(0).getType())
|| !LuaNativeLowering.isHandleType(f.getParameters().get(1).getType())
|| !supportedKey(f.getParameters().get(1).getType())
|| !TypesHelper.isIntType(f.getParameters().get(2).getType())) {
throw invalidSpecialization(f, "keyedMapPutNative requires an int map, a handle key, and an int-represented value");
throw invalidSpecialization(f, "keyedMapPutNative requires an int map, a handle or string key, and an int-represented value");
}
ImFunction fallback = findFallback(prog, packageOf(f), f, PUT, true);
ImFunction fallback = findFallback(prog, packageOf(f), f, PUT, true, f.getParameters().get(1).getType());
ImVar map = f.getParameters().get(0);
ImVar key = f.getParameters().get(1);
ImVar value = f.getParameters().get(2);
Expand All @@ -105,10 +152,10 @@ private static void lowerPut(ImProg prog, ImFunction f) {
private static void lowerGet(ImProg prog, ImFunction f) {
if (f.getParameters().size() != 2 || !TypesHelper.isIntType(f.getReturnType())
|| !TypesHelper.isIntType(f.getParameters().get(0).getType())
|| !LuaNativeLowering.isHandleType(f.getParameters().get(1).getType())) {
throw invalidSpecialization(f, "keyedMapGetNative requires an int map, a handle key, and an int-represented result");
|| !supportedKey(f.getParameters().get(1).getType())) {
throw invalidSpecialization(f, "keyedMapGetNative requires an int map, a handle or string key, and an int-represented result");
}
ImFunction fallback = findFallback(prog, packageOf(f), f, GET_INT, false);
ImFunction fallback = findFallback(prog, packageOf(f), f, GET_INT, false, f.getParameters().get(1).getType());
ImVar map = f.getParameters().get(0);
ImVar key = f.getParameters().get(1);
ImExpr call = JassIm.ImFunctionCall(f.attrTrace(), fallback, JassIm.ImTypeArguments(),
Expand All @@ -118,7 +165,10 @@ private static void lowerGet(ImProg prog, ImFunction f) {
f.getBody().add(JassIm.ImReturn(f.attrTrace(), call));
}

private static ImFunction findFallback(ImProg prog, WPackage owner, ImFunction source, String name, boolean put) {
private static ImFunction findFallback(ImProg prog, WPackage owner, ImFunction source, String name, boolean put, ImType keyType) {
if (TypesHelper.isStringType(keyType)) {
name = put ? "keyedMapPutString" : "keyedMapGetStringInt";
}
if (owner != null) {
for (ImFunction candidate : prog.getFunctions()) {
if (!name.equals(intrinsicName(candidate)) || packageOf(candidate) != owner) {
Expand All @@ -127,16 +177,17 @@ private static ImFunction findFallback(ImProg prog, WPackage owner, ImFunction s
boolean valid = put
? candidate.getParameters().size() == 3
&& TypesHelper.isIntType(candidate.getParameters().get(0).getType())
&& isHandle(candidate.getParameters().get(1).getType())
&& sameKeyRepresentation(candidate.getParameters().get(1).getType(), keyType)
Comment thread
Frotty marked this conversation as resolved.
&& TypesHelper.isIntType(candidate.getParameters().get(2).getType())
&& candidate.getReturnType() instanceof ImVoid
: candidate.getParameters().size() == 2
&& TypesHelper.isIntType(candidate.getParameters().get(0).getType())
&& isHandle(candidate.getParameters().get(1).getType())
&& sameKeyRepresentation(candidate.getParameters().get(1).getType(), keyType)
&& TypesHelper.isIntType(candidate.getReturnType());
if (!valid) {
throw new CompileError(candidate.attrTrace().attrErrorPos(),
name + " must keep the existing int-map/handle-key signature.");
name + " must keep the existing int-map/"
+ (TypesHelper.isStringType(keyType) ? "string" : "handle") + "-key signature.");
}
return candidate;
}
Expand All @@ -150,6 +201,14 @@ private static boolean isHandle(ImType type) {
return type instanceof ImSimpleType simple && "handle".equals(simple.getTypename());
}

private static boolean supportedKey(ImType type) {
return LuaNativeLowering.isHandleType(type) || TypesHelper.isStringType(type);
}

private static boolean sameKeyRepresentation(ImType fallback, ImType keyType) {
return TypesHelper.isStringType(keyType) ? TypesHelper.isStringType(fallback) : isHandle(fallback);
}

private static String intrinsicName(ImFunction f) {
return f.attrTrace() instanceof FuncDef fd
&& fd.attrHasAnnotation(CompilerIntrinsics.ANNOTATION)
Expand Down
12 changes: 6 additions & 6 deletions ...wurstscript/src/main/java/de/peeeq/wurstscript/translation/imtranslation/LuaKeyedMap.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,17 +92,17 @@ public static String nativeStubFor(ImFunction f) {
return switch (fd.getName()) {
case CREATE -> f.getParameters().isEmpty() && TypesHelper.isIntType(result)
? NATIVE_CREATE : null;
case PUT, PUT_NATIVE -> isKeyedPair(f, 3) && isValueType(f.getParameters().get(2).getType())
case PUT, PUT_NATIVE, "keyedMapPutString" -> isKeyedPair(f, 3) && isValueType(f.getParameters().get(2).getType())
&& result instanceof ImVoid
? NATIVE_PUT : null;
case GET -> isKeyedPair(f, 2) && isNilDefaultedType(result) ? NATIVE_GET : null;
case GET_NATIVE -> isKeyedPair(f, 2) ? nativeGetStub(result) : null;
case GET_INT -> isKeyedPair(f, 2) && TypesHelper.isIntType(result) ? NATIVE_GET_INT : null;
case GET_INT, "keyedMapGetStringInt" -> isKeyedPair(f, 2) && TypesHelper.isIntType(result) ? NATIVE_GET_INT : null;
case GET_REAL -> isKeyedPair(f, 2) && TypesHelper.isRealType(result) ? NATIVE_GET_REAL : null;
case GET_BOOL -> isKeyedPair(f, 2) && TypesHelper.isBoolType(result) ? NATIVE_GET_BOOL : null;
case GET_STR -> isKeyedPair(f, 2) && TypesHelper.isStringType(result) ? NATIVE_GET_STR : null;
case HAS -> isKeyedPair(f, 2) && TypesHelper.isBoolType(result) ? NATIVE_HAS : null;
case REMOVE -> isKeyedPair(f, 2) && result instanceof ImVoid ? NATIVE_REMOVE : null;
case HAS, "keyedMapHasNative", "keyedMapHasString" -> isKeyedPair(f, 2) && TypesHelper.isBoolType(result) ? NATIVE_HAS : null;
case REMOVE, "keyedMapRemoveNative", "keyedMapRemoveString" -> isKeyedPair(f, 2) && result instanceof ImVoid ? NATIVE_REMOVE : null;
case DESTROY -> isDestroy(f) ? NATIVE_DESTROY : null;
default -> null;
};
Expand All @@ -126,14 +126,14 @@ private static boolean isKeyedPair(ImFunction f, int count) {
}

/**
* Key types: an int, an erased type parameter which is the element itself, or a concrete
* Key types: an int, a string, an erased type parameter which is the element itself, or a concrete
* handle type. The last lets a library declare the operations with a {@code unit} key and a
* Jass body of {@code Table} plus {@code GetHandleId}, which is correct in source on every
* compiler, so a build which does not lower the operations degrades to the hashtable path
* instead of keying every element the same.
*/
private static boolean isKeyType(ImType t) {
return TypesHelper.isIntType(t) || t instanceof ImTypeVarRef || t instanceof ImAnyType
return TypesHelper.isIntType(t) || TypesHelper.isStringType(t) || t instanceof ImTypeVarRef || t instanceof ImAnyType
|| LuaNativeLowering.isHandleType(t);
}

Expand Down
4 changes: 3 additions & 1 deletion de.peeeq.wurstscript/src/main/resources/agent-docs/WURST_LANGUAGE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,9 @@ For the same reason, an old-style generic (`Box<T>`, `toInt<T>`) cannot be insta

### KeyedMap intrinsic representation

The compiler-owned `keyedMapPut`/`keyedMapGet` intrinsics have a map handle stored as `int`, a key represented by the native handle (or supported primitive key), and a value represented by its specialized native type. Keep their intrinsic declarations and signatures aligned with the shapes recognized by `LuaKeyedMap`; changing the declaration shape can prevent intrinsic recognition. A wrapper such as `KeyedMap<K: handle, V:>` should forward `K` and `V` directly, without `GetHandleId`, integer casts, or class-to-index boxing on the Lua path.
The compiler-owned `keyedMapPut`/`keyedMapGet` intrinsics have a map handle stored as `int`, a key represented by the native handle (or supported primitive key), and a value represented by its specialized native type. Keep their intrinsic declarations and signatures aligned with the shapes recognized by `LuaKeyedMap`; changing the declaration shape can prevent intrinsic recognition. `FastKeyedMap<K:, V:>` forwards `K` and `V` directly, without `GetHandleId`, integer casts, or class-to-index boxing on the Lua path. Its supported key types are handles and strings. The generic `keyedMapHasNative<K:>` and `keyedMapRemoveNative<K:>` accompany generic put/get; the fixed handle operations remain available to existing callers.

On Jass, string-key specializations select the same-package annotated fallbacks `keyedMapPutString`, `keyedMapGetStringInt`, `keyedMapHasString`, and `keyedMapRemoveString`. These take an int map and string key, with int values/results for put/get. The library uses `StringHash` only to choose a bucket and checks full string equality inside that bucket; a hash is never treated as a unique key. Removal frees its collision entry; destruction frees every remaining entry through an ownership list before releasing the backing Table. The interpreter selects the same fallbacks using resolved generic arguments. On Lua these operations use the string itself as a native table key, with no hashing or collision entries emitted. A low-level map must consistently use one key representation; mixing legacy handle operations with string operations on the same map is unsupported.

On Lua, the recognized operations lower to a native table keyed by the handle object itself: put is one `t[k] = v` store and get is one `t[k]` read (with typed defaults for primitive results). Do not introduce handle-id conversion or an index registry on this path. The source/Jass fallback still uses the hashtable-backed `Table` API and `GetHandleId`, because Jass has no Lua table identity keys. Jass supports only `int` and class-reference values through the existing int representation; other specializations are rejected rather than boxed. Generic reference values must be non-null; use `remove` for absence. Destroy releases the Jass `Table` and clears the Lua table in place, since aliases to a destroyed Lua object may still retain its store. `LuaKeyedMapTests` covers intrinsic recognition, direct Lua table access, native handle identity, generic integer/class values, Jass fallback, and retained-alias destruction; extend it when changing this lowering contract.

Expand Down
Loading
Loading