NativeAOT builds and allocation work: findings and plan #4

Summary

ytx can ship NativeAOT builds as an extra way to install it. The existing dotnet tool stays as it is. AOT first needs one small code change: a source-generated JsonSerializerContext.

With that change in place, the AOT build:

• Works end to end. It fetched from YouTube and produced output byte-identical to v1.0.7.
• Starts much faster. Startup drops from about 38 ms to 8.6 ms.
• Uses less CPU and memory. CPU per call falls about 6x, and peak memory halves.
• Needs no .NET runtime.

Network time is about 90% of a real run. A person running ytx once won't notice the change, but agents and batch jobs that run it many times will.

Cutting string allocations is not worth chasing for speed. ytx's own code is about 2% of run time and about 18% of allocations. The rest is network and YoutubeExplode. Two small, readable changes halve ytx's own cost. A full zero-allocation rewrite is measurable but makes no difference users would notice.

How this was measured:

• One M-series Mac (osx-arm64), with .NET SDK 10.0.401 and 8.0.425.
• Videos aCi6BhrvjhM (17 min) and Lex Fridman #501 NYFGCESmikA (5h15m, 16,284 captions, 1.1 MB of JSON output).
• Windows and osx-x64 binaries were built but never run.
• Nothing was committed or pushed.

Related: #3, the full review.

Part 1: NativeAOT

The code change it needs

Publishing as-is gives four trim and AOT warnings, all from Program.cs. YoutubeExplode and AngleSharp produce none, which was checked with TrimmerSingleWarn=false. YoutubeExplode is marked IsTrimmable.

src/Ytx/Program.cs(88,33): warning IL2026: Using member 'System.Text.Json.JsonSerializer.Deserialize<TValue>(String, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. ...
src/Ytx/Program.cs(88,33): warning IL3050: Using member 'System.Text.Json.JsonSerializer.Deserialize<TValue>(String, JsonSerializerOptions)' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. ... Use System.Text.Json source generation for native AOT applications.
src/Ytx/Program.cs(176,24): warning IL2026 / IL3050: same two, for 'JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)'

The unmodified AOT binary fetched from YouTube, then failed when writing the JSON:

Error: Reflection-based serialization has been disabled for this application. Either use the source generator APIs or explicitly configure the 'JsonSerializerOptions.TypeInfoResolver' property.

With --metadata-only it still exited 0, which is wrong. The stdin JSON path crashed with an unhandled exception.

The fix is about 10 lines. Add a YtxJsonContext with [JsonSerializable] for Input and Output, and pass .Input and .Output type info into the existing options. After the change:

• There are no warnings.
• All 26 tests pass.
• Output is byte-identical to v1.0.7 for a full run, stdin input, and the error and exit-code paths.
• The current build without AOT also gets about 6 ms faster on startup.

It works on net8, net9 and net10, so merge it whatever you decide about AOT.

Results (net10.0, osx-arm64)

How to read the table:

• Startup columns: 30 runs after 3 warm-up runs, with no network. "JSON path" is stdin JSON input that fails ID validation before any network call.
• Network columns: 8 rounds against aCi6BhrvjhM, with every build run once per round so network drift hits them all equally.
• Compare CPU time and medians, not averages. Network variance is large: standard deviations were 1–6 s, with one outlier at 21 s.
• Build types: "framework-dependent" needs a .NET runtime installed. "Self-contained" bundles the runtime into one file. ReadyToRun (R2R) precompiles code to cut startup time.

/usr/bin/time -l on --metadata-only confirms the pattern:

Notes on the table:

• Avoid trimmed self-contained without R2R. It was the worst build here, because trimming drops the precompiled framework code and everything is compiled at run time.
• Skip the size-optimised AOT build. It saves only 1 MB, and UseSystemResourceKeys would make the Error: {ex.Message} text worse.
• Where the time goes. On the AOT --metadata-only call, about 780 ms of wall time breaks down as about 9 ms of startup, about 47 ms of CPU, and network for the rest.
• What the runtime costs today. The .NET runtime adds about 30 ms of startup plus about 250 ms of compile and type-loading CPU per real call. On a full run that is within network noise.

Platform findings

• Linux. linux-arm64 AOT was built in mcr.microsoft.com/dotnet/sdk:10.0-noble-aot. It is 11.5 MB, plus an 18 MB .dbg file, and works end to end.
  • It loads OpenSSL at run time. A bare ubuntu:24.04 image fails with "The SSL connection could not be established" until libssl3t64 and ca-certificates are installed.
  • A binary built on noble needs GLIBC_2.34, so debian:11 fails. Build on ubuntu-22.04 or older to reach more systems.
• macOS. osx-x64 cross-compiles from arm64, but it couldn't be run because the test machine has no Rosetta.
  • The binary is only ad-hoc signed. That works for a Homebrew formula or curl, but a browser download is quarantined.
  • Notarising would need an Apple Developer ID.
• Windows. win-x64 needs a Windows runner and is untested.
• Debug symbols. By default the .dSYM (about 30 MB) lands in the publish output and the RID-specific package. CopyOutputSymbolsToPublishDirectory=false removes it, which was verified.

Packaging options

Option A, recommended now: GitHub Release assets plus package managers.

• Keep solrevdev.ytx on NuGet exactly as it is. It still targets net8, net9 and net10, and was verified to pack the same.
• Attach a per-RID AOT archive (.tar.gz or .zip) and SHA256SUMS to each GitHub Release.
• Publish a Homebrew tap formula and a Scoop bucket or winget manifest that point at those assets.

Option B, at a 2.0 that requires the .NET 10 SDK: a hybrid RID-specific tool package.

This uses the .NET 10 feature with PublishAot=true and ToolPackageRuntimeIdentifiers=osx-arm64;osx-x64;linux-x64;linux-arm64;win-x64;any. It was verified locally in experiments/ridtool/Ytx.RidTool.csproj:

• dotnet pack builds a 2 KB pointer package.
• dotnet pack -r osx-arm64 builds an 11 MB AOT package for that RID.
• dotnet pack -r any -p:PublishAot=false builds a 0.5 MB fallback.
• Installing with SDK 10 links straight to the native binary, with no shim. --version then takes 7.3 ms, against 33.5 ms for the current shim.

Option B breaks users on SDK 8, so it can't ship as a minor release. Installing under SDK 8 fails with:

The settings file in the tool's NuGet package is invalid: Format version is higher than supported. This tool may not be supported in this SDK version. Update your SDK.

Other limits of option B:

• AOT packages must be built on the matching OS. Building for another CPU architecture works, but building for another OS does not.
• RID packages must be pushed before the pointer package.
• The net8 and net9 targets become pointless.

csproj change

<PropertyGroup>
  <IsAotCompatible>true</IsAotCompatible>   <!-- analyzers flag IL2026/IL3050 at normal build time -->
</PropertyGroup>
<PropertyGroup Condition="'$(PublishAot)' == 'true'">
  <InvariantGlobalization>true</InvariantGlobalization>
  <CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>
</PropertyGroup>

Publish command:

dotnet publish src/Ytx -c Release -f net10.0 -r <rid> -p:PublishAot=true

Workflow changes (publish.yml)

Today one ubuntu job bumps the version, packs, publishes and creates the release. It would need splitting up:

1. version job. Bump the version and output it.

2. native matrix job. For each target, publish with -p:Version=..., name the binary ytx, archive it, and upload it as an artifact.

   RID	Runner
   osx-arm64	macos-latest
   osx-x64	macos-latest (cross-arch)
   linux-x64	ubuntu-22.04
   linux-arm64	ubuntu-22.04-arm
   win-x64	windows-latest

3. publish job. Download the artifacts, write SHA256SUMS, optionally attest provenance, and run gh release upload after gh release create.

4. Package manager job. Bump the Homebrew tap and the Scoop or winget manifest. This needs a PAT for the tap repo, or wingetcreate or winget-releaser.

The runner labels ubuntu-22.04 and ubuntu-22.04-arm were not checked for current availability.

Is AOT worth it?

• For a person running it once: no. It saves about 0.2 s on a 1–2 s call that waits on the network.
• For agents spawning it many times, or batch jobs: yes.
  • About 6x less CPU per call (47–73 ms against 300–400 ms).
  • Half the memory (53–61 MB against 111–132 MB).
  • Fast failure paths such as --help and bad input drop from 38–56 ms to 9 ms.
• For distribution: yes. No .NET runtime is needed, which helps Homebrew, containers and CI.

Part 2: Allocations and CPU

Where the time and memory go

These figures come from an instrumented build of v1.0.7 on .NET 10, Apple M1, workstation GC, running on the 5h15m video:

The short video tells the same story. For the 17-minute video, ytx's share is 17 ms of about 1.5 s, and 1.7 MB of 44 MB.

About 98% of the run is network and YoutubeExplode. Perfect code on our side would save about 20–25 ms out of about 2 s. Network jitter alone ranges from 0.5 s to 10 s for the same caption fetch.

YoutubeExplode 6.6.2 allocates about the same: 39–44 MB for the watch page and about 97 MB for captions. Its release notes since 6.5.6 contain no performance work. Upgrade anyway for the client switches in 6.5.7 and 6.6.2 and the HtmlParser concurrency fix in 6.6.1.

Findings in the current code

• NormalizeCaption allocates for almost every caption. The pattern \s+ matches every single space, so Regex.Replace returns a new string even when nothing changes. It also looks up the static regex cache on every call.
• Serialising to a string costs 26 MB on a cold run. JsonSerializer.Serialize to a string sizes its buffer for the worst-case escape of the whole value, about 6 bytes per char on a 1.1M-char string. It then produces a 2.2 MB UTF-16 string, which Console.WriteLine encodes again.
• Each caption creates four temporary strings. These are the timestamp, the link, the formatted line and the normalised text. The StringBuilder grows in chunks, and ToString().Trim() copies the result once more.
• new JsonSerializerOptions is built on every run. Reflection setup is a real part of the 13–14 ms cold serialise time. The source-gen context from Part 1 fixes this.

BenchmarkDotNet results

These use BenchmarkDotNet 0.15.2 with .NET 10.0.12 on an Apple M1, with network excluded by replaying captured captions.

• Baseline is the current v1.0.7 code.
• Tier 1 is the readable fixes below.
• Tier 2 is a pooled-buffer, chunked Utf8JsonWriter rewrite.

Every variant was checked for byte-identical output to v1.0.7, in both indented and compact modes. The checks covered both videos plus edge cases:  , NBSP, U+2028, U+0085, U+3000 and hours above 100.

BenchmarkDotNet measures warm, steady-state code, while a CLI runs once, cold. So each variant was also run once in a fresh process, 5 times, on the 5h data with no network:

Recommended changes (Tier 1)

1. Skip the regex when a caption is already clean. This takes normalisation from 754 KB to under 1 KB and makes it 6x faster.

[GeneratedRegex(@"\s+")] private static partial Regex Ws();
internal static string NormalizeCaption(string text)
{
    if (string.IsNullOrWhiteSpace(text)) return "";
    // \s+ matches every ordinary space, so the regex would allocate for nearly every caption.
    if (NeedsNormalizing(text)) text = Ws().Replace(text, " ").Trim();
    return text.Contains(" ") ? text.Replace(" ", " ") : text;
}
static bool NeedsNormalizing(ReadOnlySpan<char> t)
{
    if (char.IsWhiteSpace(t[0]) || char.IsWhiteSpace(t[^1])) return true;
    for (int i = 1; i < t.Length; i++)
        if (t[i] == ' ' ? t[i - 1] == ' ' : char.IsWhiteSpace(t[i])) return true;
    return false;
}

2. Format numbers straight into the StringBuilder. This removes the temporary strings for each caption line.

if (rawSb.Length > 0) { rawSb.Append(' '); mdSb.Append('\n'); }   // replaces AppendLine + TrimEnd's newline
rawSb.Append(text);
var ts = caption.Offset; int h = (int)ts.TotalHours;
if (h > 0) mdSb.Append($"- [{h:00}:{ts.Minutes:00}:{ts.Seconds:00}");
else       mdSb.Append($"- [{ts.Minutes:00}:{ts.Seconds:00}");
mdSb.Append($"](https://www.youtube.com/watch?v={videoId}&t={(int)ts.TotalSeconds}s) ").Append(text);

3. Serialise straight to stdout with the source-gen context. This is shared with the AOT change.

[JsonSourceGenerationOptions(WriteIndented = true)]
[JsonSerializable(typeof(Output))] partial class OutputContext : JsonSerializerContext { }
static readonly OutputContext Indented = new(new() { WriteIndented = true,  Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping });
static readonly OutputContext Compact  = new(new() { WriteIndented = false, Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping });

using var stdout = Console.OpenStandardOutput();
JsonSerializer.Serialize(stdout, output, (compact ? Compact : Indented).Output);
stdout.WriteByte((byte)'\n');

Together these are about 30 lines, stay readable and keep the output byte-identical. They halve ytx's own CPU time and allocations, saving about 20 ms on a cold 5h run. The benchmark's verify harness can become a golden-output test.

Not worth doing

• Tier 2: pooled ArrayPool<char> buffers, hand-written digit formatting, chunked Utf8JsonWriter.WriteStringValueSegment and a manually written JSON object.
  • It cuts allocations by 99% but saves only about 3 ms more than Tier 1, which is invisible next to 2 s of network.
  • It is also harder to keep correct, because of surrogate splitting, trim behaviour and the   fallback.
• Cutting YoutubeExplode's ~150 MB: this would mean forking it or bypassing its parsing. GC pauses here are a few milliseconds.

Suggested plan

1. Source-gen JsonSerializerContext and stream to stdout. This unblocks AOT and is a small win for every build.
2. Tier 1 changes to NormalizeCaption and caption formatting. Add a golden-output test.
3. IsAotCompatible=true, so the analyzers catch regressions in every build.
4. AOT Release assets. Use a native matrix job in publish.yml with checksums, then add a Homebrew tap and Scoop or winget.
5. Later: consider a 2.0 hybrid RID-specific tool package that requires the .NET 10 SDK.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GheLxQMy85bM3Cb7mWY1CT