Skip to content

feat(compiler): implement PackedArray reference semantics from semantics to C codegen - #82

Merged
SuperIceCN merged 9 commits into
masterfrom
fix/frontend/packed-array-ref-sema
Sep 26, 2026
Merged

SuperIceCN merged 9 commits into
masterfrom
fix/frontend/packed-array-ref-sema

Conversation

@Iridium-Zero

Copy link
Copy Markdown
Contributor

Summary

Align Packed*Array storage and operation semantics with the Godot 4.5 interpreter's reference (shared-identity) semantics across frontend and backend, replacing the old by-value struct (COW) storage model with Variant-backed storage.

What changed

  • Backend storage model: every Packed*Array slot (locals, parameters, fields, statics, coroutine frames) is now a godot_Variant; identity flows through godot_new_Variant_with_Variant, and raw struct<->Variant crossing is restricted to the whitelisted helpers in include_451/gdcc/gdcc_packed_ref.h (PackedRefCNames).
  • Call emission: builtin methods/operators/constructors keep the native packed ABI and adapt through the internal-pointer getter / wrap_temp (PackedNativeAbiCallSupport); engine method ptrcall helpers materialize packed args into helper-owned slots and wrap packed returns; call_func wrappers preserve identity while ptrcall wrappers intentionally isolate it.
  • Iteration: per-family live iterators (intrinsic/for_packed_array_iter.h, GdccForPackedArrayIterType) hold a sharing Variant holder and re-check live size per access — appended elements are visited, shrink/realloc stays safe.
  • Frontend writeback: route-provenance-based分流 — DIRECT_SLOT snapshot commits and builtin engine-property mutating-call writebacks are dropped for packed; assignment/subscript routes on the same property still persist; static property and lambda capture routes are unblocked (FrontendWritableTypeWritebackSupport, FrontendCfgGraphBuilder, FrontendLoweringBodyInsnPass).
  • Verification: dual-run probe harness (interpreter vs compiled output over the same packed_ref_probes.gd, golden locked by Godot 4.5.2), Godot-independent coverage inventory anchor, fake-engine runtime smoke tests, codegen text anchors, and test_suite end-to-end resources (packed_call_func_identity.gd, packed_ref_full_usage.gd).
  • Documentation: consolidated fact source doc/module_impl/backend/packed_array_implementation.md; synchronized gdcc_c_backend.md, gdcc_low_ir.md, gdcc_type_system.md, gdcc_runtime_lib.md, gdcc_ownership_lifecycle_spec.md; code/test comments made self-contained and English-only.

Why

  • The old by-value godot_Packed*Array storage copied at the C++ Vector layer and detached on first write, diverging from the interpreter: local aliases, parameter mutations, live for-in iteration, and Variant round-trips were not shared.
  • Design trade-offs recorded in the doc: widening writeback on the struct model cannot fix alias sharing, and a gdcc-managed refcount box duplicates the engine's Variant refcounting while still breaking identity at Variant boundaries — both rejected in favor of Variant-backed storage.

Affected packages/files

  • src/main/java/gd/script/gdcc/backend/c/** (CGenHelper, PackedRefCNames, PackedNativeAbiCallSupport, CBuiltinBuilder, IndexStoreInsnGen, CBodyBuilder, CHrxIdentityCatalog, ...)
  • src/main/java/gd/script/gdcc/frontend/** (writeback route support, CFG builder, lowering pass)
  • src/main/c/codegen/include_451/gdcc/gdcc_packed_ref.h, intrinsic/for_packed_array_iter.h, template_451/entry.h.ftl
  • src/test/resources/packed_ref_semantics/**, src/test/java/gd/script/gdcc/backend/c/build/packedref/**, src/test/test_suite/unit_test/**
  • doc/module_impl/backend/packed_array_implementation.md and the five compiler spec docs

Validation

  • script/run-gradle-targeted-tests.sh --tests "PackedArrayReferenceSemanticsDualRunTest,PackedRefSemanticsGoldenInventoryTest,ProbeOutputTest,ProbeGoldenComparisonTest,CConstructInsnGenEngineTest,COperatorInsnGenTest,CConstructInsnGenTest,PackedRefStorageModelSmokeTest,BuiltinCastInsnGenTest,CPackUnpackVariantInsnGenTest,IndexStoreInsnGenTest,IndexStoreInsnGenEngineTest,GdccPackedRefRuntimeSmokeTest,CallMethodInsnGenTest,ExplicitCastSupportTest,CCodegenTest,FrontendLoweringBodyInsnPassTest"
  • ./gradlew clean build --no-daemon --info --console=plain

Result: BUILD SUCCESSFUL — 4644 tests, 0 failures (includes Godot 4.5.2 + Zig gated dual-run and integration suites; interpreter and compiled outputs match the full golden line-by-line).

Risks / Notes

  • ptrcall ABI boundary intentionally does NOT preserve identity (engine ABI limit; extension-to-extension ptrcall paths only). Ordinary GDScript<->GDExtension calls go through call_func (Variant ABI) and preserve identity; locked by runtime and codegen-anchor tests.
  • variant_get_ptr_internal_getter is a semi-internal GDExtension API (also used by godot-cpp); guarded by init-time availability checks with fail-fast.
  • Hot reload treats changed packed C-type fingerprints as fail-closed (rejects reload instead of rebinding incorrectly).
  • Optional follow-ups recorded in the doc: cleanup of redundant script-property/container-element writebacks, iterator size-eval fusion (measured 12.5 -> 6.9 ns/element potential).

Key behaviors covered (Optional)

  • 24-row interpreter-locked semantics matrix: alias/parameter/script-property/static/container-element/signal/lambda/coroutine sharing, live iteration, += rebinding, duplicate() independence, content equality and Dictionary key hashing, same-family as COW copy with fresh identity.
  • Builtin engine property: mutating-call not persisted (getter copy), explicit reassignment and subscript write persisted (read-modify-write).
  • Supplementary dual-run cases: complex control-flow branch mutation, return-value identity forms, engine method packed arg/return boundary (StreamPeerBuffer.set/get_data_array), PackedStringArray mutation/equality.

Diff stats (Optional)

  • 79 files changed, 7451 insertions(+), 366 deletions(-)

Breaking changes (Optional)

  • Behavior contract change (intentional, interpreter-aligned): mutating calls on builtin engine properties (e.g. poly.polygon.push_back(x)) no longer persist; the interpreter also discards them. Assignment routes on the same property still persist.
  • Internal C ABI change: packed parameter/return/field storage changed from godot_Packed*Array to godot_Variant (internal only; hot-reload fingerprints invalidate old connections fail-closed).

Related docs (Optional)

  • doc/module_impl/backend/packed_array_implementation.md
  • doc/gdcc_c_backend.md, doc/gdcc_low_ir.md, doc/gdcc_type_system.md, doc/gdcc_runtime_lib.md, doc/gdcc_ownership_lifecycle_spec.md

- Add terminal DIRECT_SLOT commit steps when mutating calls use a local snapshot
- Preserve alias-published receiver behavior and gate writeback for supported local slots
- Validate writable-route step shape and cover CFG, lowering, and runtime behavior
- Specify Variant-backed storage to match Godot's shared mutation semantics
- Document ABI boundaries, iteration, conversions, operators, and property behavior
- Define interpreter-based behavior goldens and phased implementation coverage
- Add a Godot interpreter and gdcc dual-run probe harness with golden comparisons
- Track probe coverage and phase-based enablement, including compile-blocked cases
- Record measured PackedArray behavior and update the implementation plan
- Add Variant-backed helpers for shared aliases and explicitly whitelisted conversions
- Resolve packed-array internal getters per translation unit and fail fast on invalid use
- Add Zig-gated runtime coverage for identity, conversion boundaries, and failure paths
- Record Phase B implementation and validation results
- Store PackedArray values as shared Variant holders and route copies, casts, and destruction through the packed-reference helpers
- Adapt builtin calls, operators, constructors, and ptrcall boundaries to the native packed-array ABI
- Make packed-array iteration observe live mutations without caching element pointers
- Add codegen and runtime coverage for identity, conversions, ABI isolation, and iteration
- Record Phase C implementation and validation results
- Skip redundant snapshot and static-property commits for shared PackedArray values
- Prevent mutating calls on engine-property copies from persisting, while retaining assignment and script-property writeback
- Enable packed-array mutation through static properties and lambda captures
- Cover all ten packed-array types and promote deferred interpreter comparison probes
- Stub and register `variant_new_nil` so iterator setup never destroys uninitialized Variant storage
- Require dual-run tests to match the full golden matrix and add an independent inventory check
- Expand probes for control flow, return-value sharing, engine method boundaries, and PackedStringArray mutation
- Add runtime and codegen coverage for call_func identity and ptrcall ABI boundaries
- Align type, ownership, runtime, and backend documentation with the Variant-backed storage model
- Replace the phased plan with a maintained source of truth for implemented storage, ABI, ownership, and conversion contracts
- Synchronize related compiler docs and code/test references with the finalized documentation
- Keep the interpreter comparison goldens and coverage inventory aligned with the documented behavior
@SuperIceCN
SuperIceCN merged commit 02f7210 into master Sep 26, 2026
4 checks passed
@SuperIceCN
SuperIceCN deleted the fix/frontend/packed-array-ref-sema branch September 26, 2026 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants