Octo: ggml inference engine (diffusion + L1/proprio), LIBERO client, and open-loop evaluation

[DuyBaoDOCer]

What

Adds a complete Octo inference engine in C++/ggml (src/models/octo.cpp) covering the full forward pipeline from image + language observation to predicted action chunk, plus an open-loop evaluation harness that measures parity against the PyTorch model.

Core C++ engine (src/models/octo.{h,cpp}):

• SmallStem16 vision tokenizer (primary 256x256 + wrist 128x128 camera streams) as a ggml graph
• Native T5-base text encoder with SentencePiece/unigram tokenizer (vocab baked into GGUF)
• Language projection + repeat_task_tokens, block transformer with block-wise attention mask
• Diffusion action head with golden noise replay for deterministic parity testing
• L1 action head + proprio tokenizer (head_type=l1) for fine-tuned/ALOHA checkpoints
• Resident weight-loading path: weights + embedding table loaded once; inference dispatched to model backend (CUDA when available, CPU otherwise)
• window_size and action-buffer geometry (horizon × dim) read from GGUF metadata
• Stats (ms_vision / ms_inference / ms_total) populated in predict()

GGUF converter (scripts/convert_octo_to_gguf.py):

• Supports the published octo-small-1.5 diffusion checkpoint and fine-tuned PyTorch L1/proprio checkpoints (--ckpt-format pytorch --ckpt … --step …)

LIBERO client and server wiring:

• VlaCppClient extended with wrist zero-fill fallback and data-driven unnorm_dataset key resolution
• OctoL1Client subclass sends raw proprio via req.state (teacher-forced)
• LIBERO libero_object unnorm + gripper-channel output

Open-loop evaluation harness (eval/client/run_open_loop_octo.py):

• Teacher-forced runner over a real RLDS episode
• Reports MAE / MSE / RMSE + gripper accuracy in both normalized and original action units
• Saves GT-vs-prediction plot and NumPy trace for auditing

Test suite (tests/, tests/py/):

• Full golden-trace parity vs OctoPt oracle (diffusion, 20/20 cases)
• Statistical action-distribution parity (N=512 samples)
• Preprocessing parity (rotate180 + resize 256)
• LIBERO window=1 parity and window=2 regression
• Stagewise CPU parity for the L1+proprio path (T0–T5)
• Resident-path parity wired into ctest
• Open-loop output compared against OctoPt golden trace

Bug fixes:

• Handle flat (single-dataset) octo.dataset_statistics shape
• Bake window_size=1 for fine-tuned PyTorch checkpoints
• Fix SentencePiece/protobuf ABI conflict causing vla-server abort on startup
• Derive open-loop plot title from checkpoint path (was hardcoded jitter2525)

Docs (docs/octo_open_loop_vla_cpp_en.md):

• Open-loop evaluation report for the jitter-adapted Octo checkpoint (step 4500) on aloha_carrot_easy_rlds ep0

Why

Provides a self-contained, PyTorch-free inference path for Octo — suitable for embedded/edge deployment (vla-server over ZMQ, no Python runtime on robot). The evaluation harness confirms the C++ port is numerically indistinguishable from the original model before any deployment use. Fine-tuned L1/proprio checkpoints (ALOHA task suite) are first-class: they require a different action head and proprio tokenizer not present in the upstream diffusion checkpoint.

Verified

• Builds clean under -Wall -Wextra (first-party code)
• ctest passes
• Numeric output unchanged (vla_predict_check diff), or the change is meant to move it and a LIBERO sweep is below

Open-loop parity — vla.cpp C++ engine vs OctoPt, step-4500 checkpoint, aloha_carrot_easy_rlds ep0 (149 steps, 19 inference calls):

Metric	vla.cpp	PyTorch	|diff|
original MAE	0.008952	0.008951	4.4e-7
original MSE	0.000212	0.000212	2.0e-8
original RMSE	0.014549	0.014549	6.8e-7
gripper accuracy	99.33%	99.33%	0.0
normalized MAE	0.032435	0.032432	2.7e-6

All metrics agree to within ~1e-5. Golden-trace parity: 20/20 cases pass (diffusion).
Stagewise L1+proprio parity: all T0–T5 stages pass on CPU. Full ctest: 25/25.

Archs and backends tested:

• x86-64 · CUDA 12.6 · RTX 4050 Laptop (sm_89)
• x86-64 · CUDA 12.8 · A100 (sm_80, Colab)