Skip to content

feat(cuda): add CUDA 13.0.0 jobs to the compile workflow - #1439

Merged
martindevans merged 3 commits into
SciSharp:masterfrom
modelpath-dev:feat/1360-cuda13
Sep 27, 2026
Merged

martindevans merged 3 commits into
SciSharp:masterfrom
modelpath-dev:feat/1360-cuda13

Conversation

@modelpath-dev

@modelpath-dev modelpath-dev commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Why

Martin asked to split the CUDA 13 work so the compile workflow can land first. Maintainers can then run a Compile build and refresh the development binaries package before the managed loader or NuGet packages change.

This is part 1 of 3 for #1360.

Scope

  • .github/workflows/compile.yml: add 13.0.0 to the cublas matrix, use Jimver/cuda-toolkit@v0.2.36 for non-12.4.0 installs, and gather cu13.0.0 Windows/Linux artifacts next to cu12.4.0.

Out of scope here: NativeLibraryWithCuda, LLamaSharp.Backend.Cuda13 nuspecs, LLamaSharp.Runtime.targets, and docs.

Tradeoffs

Kept the existing 12.4.0 toolkit pin on @v0.2.15 and only moved newer matrix values to @v0.2.36, so the CUDA 12 path stays unchanged while 13 can install.

Blast Radius

Only the Compile workflow and its published native artifact layout change. No managed package or runtime load path changes until parts 2 and 3.

Verification

  • Patch-id of the full three-part tip matches the previous combined feat(cuda): add CUDA 13.0.0 jobs to the compile workflow #1439 tip (757b5dd6…).
  • This PR's diff is only .github/workflows/compile.yml (+33/−4).
  • dotnet is not installed on this machine, so Compile CI on GitHub is the live check for the new matrix jobs.

Stack:

  1. This PR (build)
  2. Loader (opens next)
  3. NuGet packaging (opens after loader)

@martindevans

Copy link
Copy Markdown
Member

Thanks for putting this together so quickly!

Would you mind spitting the PR into 3 parts:

  1. Just the build changes. That way we can merge those, run a build, and update the binaries package used for development
  2. The new loader. We can merge this once the binaries are available for it to use.
  3. The nuget changes. We have a separate issue with our package management/ownership at the moment, we need to get that resolved before we can create new nuget packages.

Sorry for the extra work.

@modelpath-dev

Copy link
Copy Markdown
Contributor Author

Ok @martindevans, working on it

Extend the cublas matrix and Gather Binaries layout so a Compile run can produce cu13.0.0 artifacts next to cu12.4.0 without changing the managed loader or NuGet packages yet.
@modelpath-dev modelpath-dev changed the title Add CUDA 13 backend alongside CUDA 12 feat(cuda): add CUDA 13.0.0 jobs to the compile workflow Sep 18, 2026
@modelpath-dev

Copy link
Copy Markdown
Contributor Author

@martindevans Split as you asked. Same code as before, three PRs:

  1. This PR — compile workflow only (build / Gather Binaries for cu13.0.0)
  2. feat(cuda): probe cuda13 native libraries before cuda12 #1440 — loader + unit tests
  3. feat(cuda): add LLamaSharp.Backend.Cuda13 packages #1441 — Backend.Cuda13 nuspecs, Runtime.targets, docs

Please merge in that order. I will rebase 1440 onto master after this one lands, then 1441 after 1440.

Comment thread .github/workflows/compile.yml
@martindevans

Copy link
Copy Markdown
Member

Alright this all looks good to me! Thanks for splitting it up, it's much easier to handle this way.

Could you start a test run over in your repo (I would do it myself, but unfortunately I can't do it here until it's merged, and I don't have permission to do it in your fork).

To do that:

  1. Go to actions tab
  2. On the left, find the Update Binaries action and click it
  3. On the top right corner of the action click Run workflow
  4. Click the green Run workflow in the popup, this will build against llama.cpp master branch
  5. Copy the run URL into a comment here

@martindevans

Copy link
Copy Markdown
Member

It looks like your test run worked (https://github.com/modelpath-dev/LLamaSharp/actions/runs/36240372366), so I'll merge this and start a run in this repo. Once it's done I'll upload the binaries and we can start using those in dev.

@martindevans
martindevans merged commit 05a5cff into SciSharp:master Sep 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants