Skip to content

ci: fix Extensions Tests and Go bindings CI (runner/toolchain bitrot) - #2337

Open
sivukhin wants to merge 5 commits into
mainfrom
ci/crr-extensions-install-libclang
Open

sivukhin wants to merge 5 commits into
mainfrom
ci/crr-extensions-install-libclang

Conversation

@sivukhin

@sivukhin sivukhin commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

The Extensions Tests workflow jobs (CR SQLite C Tests, CR SQLite Rust Tests, Extensions Tests) fail on main with ~140 Rust compile errors when building the cr-sqlite extension:

error[E0432]: unresolved import `crate::bindings::sqlite3_index_info_sqlite3_index_constraint`
error[E0609]: no field `bind_blob` on type `sqlite3_api_routines`
              note: available fields are: `_address`

Root cause

libsql-sqlite3/ext/crr/rs/sqlite-rs-embedded/sqlite3_capi generates its Rust bindings from the SQLite C headers at build time with bindgen = "0.68.1" (the extensions-tests job does the same via cargo xtask build-bundled). bindgen loads a system libclang (via clang-sys) to parse the headers.

On the current GitHub ubuntu-latest runner image, clang-sys picks up a libclang that cannot properly parse the SQLite headers, so bindgen falls back to emitting opaque placeholder structs — sqlite3_api_routines comes out with only an _address field and the nested sqlite3_index_info_sqlite3_index_constraint* types are missing, which is exactly what the _address-only error notes show.

Installing libclang-dev alone does not fix it: clang-sys still selects the wrong libclang on the runner. The binding generation is correct once bindgen is pointed at a working libclang (verified in a clean ubuntu:24.04 container — libclang 16/17/18 all produce the full sqlite3_api_routines and the correct nested-struct names). The crr sources are unchanged; this is purely a runner/toolchain resolution issue.

Fix

Install libclang-18-dev and set LIBCLANG_PATH=/usr/lib/llvm-18/lib on the three Extensions Tests jobs, so clang-sys uses a known-good libclang. This pins the toolchain, not the runner image, so it stays correct across runner-image updates and does not expire when an older image (e.g. ubuntu-22.04) is retired. It aligns with how the repo's other bindgen workflows provision libclang (rust.yml, nemesis.yml).

Files changed

  • .github/workflows/extensions-test.yml — add LIBCLANG_PATH env and libclang-18-dev install to c-tests, rs-tests, and extensions-tests.

Verification

  • Reproduced the CI failure signature (available fields are: _address) from the failing run logs and traced it to bindgen emitting opaque structs.
  • In a clean ubuntu:24.04 container, confirmed the crr bindings generate correctly with libclang-dev present and LIBCLANG_PATH set (only the crate's expected nightly feature gate remained, proving the bindings themselves are correct).
  • The runner-specific libclang selection only reproduces on the affected CI image, so CI on this PR is the gate confirming the fix restores a green build.

🤖 Generated with Claude Code

The cr-sqlite extension jobs (CR SQLite C Tests, CR SQLite Rust Tests)
and the Extensions Tests job build Rust bindings from the SQLite C
headers with bindgen (bindgen = "0.68.1" in
sqlite-rs-embedded/sqlite3_capi; and cargo xtask build-bundled). bindgen
loads the runner's system libclang at build time to parse the headers.

Without libclang-dev installed, on current GitHub runner images that
libclang cannot parse the SQLite headers and bindgen falls back to
emitting opaque placeholder structs instead of the real definitions. The
generated `sqlite3_api_routines` ends up with a single `_address` field
and the nested `sqlite3_index_info_sqlite3_index_constraint` types are
missing, so the crr build fails with ~140 errors:

    error[E0432]: unresolved import `crate::bindings::sqlite3_index_info_sqlite3_index_constraint`
    error[E0609]: no field `bind_blob` on type `sqlite3_api_routines`
                  note: available fields are: `_address`

Install libclang-dev in these jobs so bindgen has a working libclang and
parses the headers correctly, matching how the other bindgen-based
workflows in this repo already provision it (rust.yml, nemesis.yml). This
keeps the jobs on ubuntu-latest and is not tied to a specific, soon to be
retired runner image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Installing libclang-dev alone was not enough on the GitHub ubuntu-latest
runner: clang-sys still picked up a libclang that cannot parse the SQLite
headers, so bindgen kept emitting opaque placeholder structs
(sqlite3_api_routines with only an `_address` field, missing
sqlite3_index_info_sqlite3_index_constraint types) and the crr build
failed with ~140 errors.

Install libclang-18-dev explicitly and point bindgen at it with
LIBCLANG_PATH=/usr/lib/llvm-18/lib, so clang-sys uses a known-good
libclang that parses the headers correctly. Verified in a clean
ubuntu:24.04 container that libclang 16/17/18 generate the correct
bindings (full sqlite3_api_routines, correct nested struct names). This
pins the toolchain rather than the runner image, so it does not depend on
a soon to be retired image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sivukhin sivukhin changed the title ci: fix cr-sqlite extension build by installing libclang-dev for bindgen ci: fix cr-sqlite extension build by pinning libclang via LIBCLANG_PATH Nov 20, 2026
sivukhin and others added 2 commits November 20, 2026 11:53
The Extensions Tests job's `cargo test --features=extensions` builds
wabt-sys v0.6.1, whose bundled CMakeLists declares a pre-3.5
cmake_minimum_required. CMake 4.x on the runner refuses that and the
build fails:

    CMake Error at CMakeLists.txt:17 (cmake_minimum_required)
    thread 'main' panicked ... build script failed, must exit now

Set CMAKE_POLICY_VERSION_MINIMUM=3.5 for the job so CMake accepts the
old minimum version from this third-party dependency.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The golang-bindings job builds sql-experimental into a static library
(libsql_experimental.a) and links it into go-libsql via cgo. Newer Go
toolchains reject static libraries that contain an embedded LLVM bitcode
section, so the test build fails:

    disallowed LTO section ".llvmbc" in #cgo LDFLAGS: .../libsql_experimental.a
    FAIL github.com/tursodatabase/go-libsql [build failed]

Build sql-experimental with RUSTFLAGS="-C embed-bitcode=no" so rustc does
not embed the .llvmbc section. The release profile does not enable LTO,
so dropping embedded bitcode is safe and does not affect the native code.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sivukhin sivukhin changed the title ci: fix cr-sqlite extension build by pinning libclang via LIBCLANG_PATH ci: fix Extensions Tests and Go bindings CI (runner/toolchain bitrot) Nov 20, 2026
@sivukhin

Copy link
Copy Markdown
Contributor Author

Added a Go-bindings fix on top of the Extensions Tests fixes so the whole PR goes green:

  • golang-bindings: the sql-experimental static library carried an embedded LLVM bitcode (.llvmbc) section, which newer Go toolchains reject during cgo linking (disallowed LTO section ".llvmbc"). Built it with RUSTFLAGS="-C embed-bitcode=no" so the .a has no bitcode section. The release profile has no LTO, so this is safe.

All three Extensions Tests jobs are already green here; this makes the Go bindings job green too.

The previous RUSTFLAGS="-C embed-bitcode=no" approach did not help: the
repo's .cargo/config.toml sets build rustflags (--cfg tokio_unstable),
and the RUSTFLAGS env var replaces rather than extends it, so the flag
both risked dropping tokio_unstable and did not remove the embedded
bitcode. The go-libsql cgo build still failed with:

    disallowed LTO section ".llvmbc" in #cgo LDFLAGS: .../libsql_experimental.a

Instead, strip the .llvmbc/.llvmcmd sections from the built static
library with objcopy before copying it into go-libsql. objcopy removes
the sections from every archive member and the library still links
cleanly (verified). This is independent of cargo config and the build
cache, and the sections are only used for LTO, which the release build
does not perform.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants