Rendering: stop first-use stutters on WebGL and WebGPU

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
 	"name": "melonjs",
-	"version": "20.9.0",
+	"version": "20.10.0",
 	"description": "Build games with melonJS — 24 guides to the 2D, 2.5D and 3D HTML5 game engine, its conventions and idioms, so generated code runs the first time.",
 	"author": {
 		"name": "melonJS",

packages/melonjs/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## [20.10.0] (melonJS 2) - _unreleased_
+
+**Highlights:** a game no longer stutters the first time an effect, a blend mode or a new kind of object appears. Shaders, pipelines and render targets are now prepared when the object is added, or during `loader.preload()`, rather than in the frame that first draws it, where a single first draw was measured at up to 140ms. Dropping a batch of meshes no longer stalls the frame either.
+
+### Fixed
+- WebGL: releasing retained geometry no longer stalls the frame, a stutter whenever a scene dropped a batch of meshes. Teardown asked a blocking `is*` query about every buffer and vertex array before deleting it, a round trip per object, and now only asks once the context has been lost, the only case where the answer can be no
+- WebGL: a `ShaderEffect` hosted on a `Mesh` is compiled during `loader.preload()`, on every call including a later one that brings a level's own effects, rather than on the frame it first draws. Preloading the shader built its standalone program, but a mesh host splices the body into the mesh shader, which is a different program again, so a game that declared every effect in its manifest still stuttered the first time each one appeared
+- WebGPU: a post effect's pipelines are compiled when the effect is attached (`addPostEffect`, assigning `shader`, or the tone curve `setToneMapping` builds), off the frame, rather than in the frame that first draws it, a stutter measured at up to 140ms per effect on desktop. Attaching effects when a pool or a stage is built, rather than when the object is shown, gives the compile time to finish
+- WebGL: a `ShaderEffect` hosted on a `Mesh` is also linked when it is attached, so an effect built inline and never preloaded no longer stalls its first draw either. The links are spread over the following frames, so a stage building a pool of effect-carrying meshes does not pay for all of them in one
+- Effects: switching a disabled effect on no longer creates a render target in that frame. A chain gets its targets for every effect attached to it, enabled or not, when the effect is attached
+- Rendering: adding a renderable to the world starts preparing its draw, so one added before it is first shown no longer pays for it on that frame. On WebGPU that is the pipeline for its blend mode, culling and transparency, compiled off the frame, and on both GPU backends an advanced blend mode (`overlay`, `difference` and the rest) gets its blend shader and offscreen target, a first draw measured at up to 136ms
+- WebGL and WebGPU: the shader warm-up runs again after a lost context or device is restored. It never did, so every shader went back to compiling on the frame it next drew
+- WebGPU: the mip-chain generator is built during `loader.preload()` when textures are filtered (`textureFilter` resolving to `"linear"`), instead of on the frame the first mipmapped texture is uploaded
+- Sprite: a video the browser refuses to start, as it does before the user has interacted with the page, no longer logs an uncaught error on every frame. The refusal is handled once, and playback starts on the next click, touch or key press
+- WebGL: ETC1 compressed textures no longer draw black. They were uploaded with a call ETC1 does not accept, which left the texture empty; they now upload as ETC2, which reads the same data, or level by level when the browser only supports ETC1
+- WebGPU: the smallest mip levels of a compressed texture (the 2×2 and 1×1 levels of a 4×4-block format) are no longer rejected and left empty. Each upload now covers whole blocks, as WebGPU requires
+
 ## [20.9.0] (melonJS 2) - _2026-10-18_
 
 ### Added

packages/melonjs/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "melonjs",
-	"version": "20.9.0",
+	"version": "20.10.0",
 	"description": "melonJS Game Engine",
 	"homepage": "http://www.melonjs.org/",
 	"type": "module",

packages/melonjs/skills/melonjs-effects-and-shaders/SKILL.md

@@ -453,8 +453,15 @@ With `prewarm` on (the default since 20.6), `loader.preload()` warms on its way
 through — automatically, on every call — and compiles the GPU-side objects for
 every shader it holds — on WebGPU that is the
 WGSL module, which is otherwise built inside the draw path the first time the
-effect is used. Effects constructed inline are not in the loader, so nothing can
-bring them forward.
+effect is used.
+
+Since 20.10 the rest is warmed when the effect is **attached**, inline effects
+included: `addPostEffect` (or assigning `shader`) compiles the pipelines that
+effect will draw with on that renderable — on WebGPU the costly part, keyed by
+the renderable's blend mode and target as well as the shader — and on WebGL
+links the spliced program when the host is a mesh. It finishes off the frame,
+so attach when you build (a pool, a stage starting), not when you show: an
+effect attached and drawn in the same frame still compiles on that frame.
 
 One caveat if you are moving an inline effect to a file: a template literal can
 interpolate constants into the source (`vec2(${ASPECT}, 1.0)`), and a static
@@ -470,8 +477,8 @@ uniform values — fine when the look is uniform, and the reason to clone (and
 pay a link) when it is not.
 
 Note this covers **your** effects only. The engine's own mesh/quad shader
-variants are compiled lazily per feature combination on first draw and cannot be
-preloaded — see `melonjs-loading-assets` for the warm-up that covers those.
+variants are covered by the loading-screen warm-up and by the per-renderable one
+— see `melonjs-loading-assets`.
 
 ### Screen-reading builtins
 

packages/melonjs/skills/melonjs-loading-assets/SKILL.md

@@ -111,13 +111,31 @@ the assets are in and **before** `LOADER_COMPLETE` fires, so it happens while
 the progress bar is still up. Every `preload()` warms, not just the first, so a
 per-level preload warms whatever that level brought with it.
 
-The one precondition: **a game that never calls `loader.preload()` is never
-warmed**, because that is the only hook. If you load assets another way, call
+The one precondition for this part: **a game that never calls
+`loader.preload()` does not get it**. If you load assets another way, call
 `await app.renderer.prewarm()` yourself. On
 WebGL that links the mesh batchers' variants, for the unlit tier and the lit
 one that inherits from it; on WebGPU it compiles the
-WGSL module of every shader the loader holds — which is the reason to declare
-effects as assets rather than build them inline (see above).
+WGSL module of every shader the loader holds, and the mip-chain blitter when
+textures are filtered.
+
+**4. Everything that depends on the renderable — warmed as you build, since
+20.10.** The rest cannot be done at preload, because it depends on what draws:
+on WebGPU a *pipeline* is keyed by blend mode, culling, transparency and the
+format of the target as well as by shader, and that compile is the expensive
+part (tens of milliseconds, up to 140ms measured, per effect). So, behind the
+same `prewarm` setting, the engine warms a renderable when it gains an effect
+(`addPostEffect`, or assigning `shader`) and when it is added to the world:
+its effect pipelines, its own draw, the render targets its chain composites
+through, the advanced-blend shader if it uses one. Inline effects are covered
+too; on WebGL that means linking a mesh-hosted effect's spliced program.
+
+That work starts after your code has run and finishes off the frame, so it
+needs **at least a frame between setting a renderable up and drawing it**. The
+pattern that gets it for free is the one you want anyway: build pools (and
+attach their effects) when the stage starts, then show and hide them during
+play. Something created and drawn in the same frame still compiles on that
+frame.
 
 Cost to a game that uses none of it is about 60ms of preload on a desktop GPU.
 Set it to `false` only for a target where linking is slow enough that the

packages/melonjs/skills/melonjs-performance/SKILL.md

@@ -245,9 +245,12 @@ Since 20.6 the engine's own programs are built during `preload()` instead —
 `loader.preload()` calls `renderer.prewarm()` itself, on every call, when the
 application's `prewarm` setting is on, which is the default. So on a current
 version this is one cause you can rule out before looking further, **unless the
-game does not use the preloader at all**, which is the one way to miss it — see `melonjs-loading-assets`. It does not
-cover effects you construct inline, and it does not cover the rest of what a
-first frame pays for, below.
+game does not use the preloader at all**, which is the one way to miss it — see `melonjs-loading-assets`. Since 20.10 the
+pipelines that depend on the renderable (effects, blend modes, render targets)
+are warmed when an effect is attached or a renderable is added to the world, as
+long as that happens at least a frame before it is drawn: a pool built at stage
+start is covered, an object created and drawn in the same frame is not. None of
+this covers the rest of what a first frame pays for, below.
 
 How to tell them apart, all from the page:
 
@@ -364,6 +367,7 @@ In rough order for a typical 2D game:
 | memory grows with long music | missing `stream: true` |
 | slow only on some machines | Canvas fallback — no GPU tilemap path |
 | a freeze the first time a level shows, never again | shader variants linking on first draw — warm them behind the loading screen |
+| a freeze the first time an effect appears mid-play (a shield, a hit flash) | the effect attached in the same frame it is drawn — attach it when the pool is built, so the warm-up has a frame to finish |
 | the same scene stutters once per entry | a derived texture rebuilt per stage: the texture cache keys on the image object, so each rebuild is a new upload |
 
 ## Related skills

packages/melonjs/src/application/settings.ts

@@ -279,10 +279,13 @@ export type ApplicationSettings = {
 	 * screen, instead of paying for it on the frame that first draws (GPU
 	 * backends — the Canvas renderer has nothing to warm and ignores this).
 	 *
-	 * Today that means the built-in shader programs, which is all
-	 * {@link Renderer#prewarm} does. Named for the intent rather than the
-	 * current contents: anything else worth doing before the first frame
-	 * belongs behind the same switch, and should not need a second setting.
+	 * Named for the intent rather than the contents: anything worth doing
+	 * before the first frame belongs behind this one switch. Today that is
+	 * {@link Renderer#prewarm} during `loader.preload()` (the built-in shader
+	 * programs, and the shaders declared as assets), and then, as the game
+	 * runs, warming each renderable when it gains a post effect or is added
+	 * to the world: its effect pipelines, its own draw, the render targets an
+	 * effect chain composites through. That second part needs no preload.
 	 *
 	 * A GPU backend does not finish a shader when handed the source; it
 	 * finishes it the first time something is drawn with it. That puts the
@@ -299,10 +302,11 @@ export type ApplicationSettings = {
 	 * `false` for a target where linking is slow enough that the preload
 	 * itself would suffer.
 	 *
-	 * The warm-up is run by `loader.preload()` — every call, not just the
-	 * first — so a game that preloads gets it automatically and a game that
-	 * never calls `preload()` never gets it at all. Call
-	 * {@link Renderer#prewarm} directly in that case.
+	 * The loading-screen warm-up is run by `loader.preload()` — every call,
+	 * not just the first — so a game that preloads gets it automatically and
+	 * a game that never calls `preload()` does not. Call
+	 * {@link Renderer#prewarm} directly in that case. It runs again on its
+	 * own after a lost context or device is restored.
 	 *
 	 * Shaders declared as assets (`{type: "shader"}`) are covered too, so a
 	 * level's own effects are warmed by declaring them alongside that level's

packages/melonjs/src/renderable/container.js

@@ -538,6 +538,9 @@ export default class Container extends Renderable {
 					registerChildBody(cchild, worldContainer);
 				});
 			}
+			// it will draw from the next frame on: build what that draw needs
+			// now (honours the `prewarm` setting)
+			worldContainer?.app?.renderer?._queueWarm?.(child);
 		}
 
 		if (
@@ -608,6 +611,8 @@ export default class Container extends Renderable {
 						registerChildBody(cchild, worldContainer);
 					});
 				}
+				// see addChild
+				worldContainer?.app?.renderer?._queueWarm?.(child);
 			}
 
 			if (

packages/melonjs/src/renderable/renderable.js

@@ -695,18 +695,28 @@ export default class Renderable extends Rect {
 			this.postEffects.length = 0;
 		} else {
 			this.postEffects = [value];
+			value._renderer?._queueWarm?.(this);
 		}
 	}
 
 	/**
 	 * Add a post-processing shader effect to this renderable.
+	 *
+	 * With the `prewarm` application setting on (the default), what the
+	 * effect will need to draw here is compiled from this call, off the
+	 * frame, rather than on the frame the renderable first draws with it.
+	 * That takes a moment to finish, so attach effects when the renderable
+	 * is built (a pool, a stage starting) rather than when it is shown.
 	 * @param {GLShader|ShaderEffect} effect - the effect to add
 	 * @returns {GLShader|ShaderEffect} the added effect
 	 * @example
 	 * mySprite.addPostEffect(new DesaturateEffect(renderer));
 	 */
 	addPostEffect(effect) {
 		this.postEffects.push(effect);
+		// compile what this effect will need now rather than on the frame the
+		// renderable first draws with it (honours the `prewarm` setting)
+		effect?._renderer?._queueWarm?.(this);
 		return effect;
 	}
 

packages/melonjs/src/renderable/sprite.js

@@ -16,6 +16,10 @@ import Renderable from "./renderable.js";
 // flicker interval in ms (~15 flashes per second)
 const FLICKER_INTERVAL_MS = 33;
 
+// the gestures a browser counts as the user allowing playback, the same set
+// the audio backend listens for to unlock sound
+const VIDEO_GESTURES = ["touchstart", "touchend", "click", "keydown"];
+
 /**
  * additional import for TypeScript
  * @import {TextureAtlas} from "./../video/texture/atlas.js";
@@ -265,6 +269,9 @@ export default class Sprite extends Renderable {
 				 */
 				this.removeStatePauseListener = on(STATE_PAUSE, () => {
 					this.image.pause();
+					// a paused game starts nothing: update() asks again once
+					// it runs, as it always has
+					this._cancelVideoPlay();
 				});
 
 				// Keep a video-frame callback pending while this sprite is
@@ -282,6 +289,24 @@ export default class Sprite extends Renderable {
 				 * @internal
 				 */
 				this._lastVideoFrameVersion = -1;
+				/**
+				 * a `play()` is in flight, or was refused until a user gesture
+				 * @ignore
+				 * @internal
+				 */
+				this._videoPlayPending = false;
+				/**
+				 * bumped to orphan an in-flight `play()` request
+				 * @ignore
+				 * @internal
+				 */
+				this._videoPlayRequest = 0;
+				/**
+				 * the document listener waiting for that gesture, while armed
+				 * @ignore
+				 * @internal
+				 */
+				this._videoGestureListener = undefined;
 				if (typeof this.image.requestVideoFrameCallback === "function") {
 					this.image.version ??= 0;
 					const tick = () => {
@@ -877,8 +902,8 @@ export default class Sprite extends Renderable {
 		if (this.isVideo) {
 			if (this.animationpause) {
 				this.image.pause();
-			} else if (this.image.paused) {
-				this.image.play();
+			} else if (this.image.paused && this._videoPlayPending !== true) {
+				this._playVideo();
 			}
 			// set-only (never assign false — that would stomp a dirty flag
 			// set earlier in the tick, e.g. by a tween on a paused sprite)
@@ -1009,6 +1034,93 @@ export default class Sprite extends Renderable {
 		);
 	}
 
+	/**
+	 * Start the video and handle the browser's answer. Before the user has
+	 * interacted with the page, browsers refuse playback: calling `play()`
+	 * again on every frame only logged an uncaught error per frame, so a
+	 * refusal now waits for the next gesture instead. Any other failure
+	 * (e.g. a `pause()` interrupting the request) retries on the next frame.
+	 * @ignore
+	 * @internal
+	 */
+	_playVideo() {
+		const request = ++this._videoPlayRequest;
+		this._videoPlayPending = true;
+		// `Promise.resolve`: browsers before the promise-returning `play()`
+		// return undefined
+		Promise.resolve(this.image.play()).then(
+			() => {
+				this._videoPlayPending = false;
+			},
+			(error) => {
+				if (request !== this._videoPlayRequest) {
+					// orphaned by a pause or destroy() while it was pending
+					return;
+				}
+				if (error?.name === "NotAllowedError") {
+					this._awaitVideoGesture();
+				} else {
+					this._videoPlayPending = false;
+				}
+			},
+		);
+	}
+
+	/**
+	 * Wait for the next user gesture and start the video from inside its
+	 * handler, the moment a browser that refused playback allows it.
+	 * @ignore
+	 * @internal
+	 */
+	_awaitVideoGesture() {
+		if (this._videoGestureListener !== undefined) {
+			return;
+		}
+		this._videoGestureListener = () => {
+			this._removeVideoGestureListener();
+			this._videoPlayPending = false;
+			if (this.animationpause !== true && this.image.paused) {
+				this._playVideo();
+			}
+		};
+		for (const type of VIDEO_GESTURES) {
+			globalThis.document.addEventListener(
+				type,
+				this._videoGestureListener,
+				true,
+			);
+		}
+	}
+
+	/**
+	 * Drop a pending `play()` request and stop waiting for a gesture.
+	 * @ignore
+	 * @internal
+	 */
+	_cancelVideoPlay() {
+		this._videoPlayRequest++;
+		this._videoPlayPending = false;
+		this._removeVideoGestureListener();
+	}
+
+	/**
+	 * @ignore
+	 * @internal
+	 */
+	_removeVideoGestureListener() {
+		if (this._videoGestureListener === undefined) {
+			return;
+		}
+		for (const type of VIDEO_GESTURES) {
+			globalThis.document.removeEventListener(
+				type,
+				this._videoGestureListener,
+				true,
+			);
+		}
+		this._videoGestureListener = undefined;
+	}
+
 	/**
 	 * Destroy function<br>
 	 * @ignore
@@ -1021,6 +1133,7 @@ export default class Sprite extends Renderable {
 		this.offset = undefined;
 		if (this.isVideo) {
 			this.removeStatePauseListener();
+			this._cancelVideoPlay();
 			if (typeof this._videoFrameHandle !== "undefined") {
 				this.image.cancelVideoFrameCallback(this._videoFrameHandle);
 				// drop the frame-counter stamp so a later non-Sprite consumer