[spencercjh]

Most of this Pull Request was generated or revised with the assistance of AI tools(Raft with GPT-5.6 Sol High). I have reviewed the resulting content and take full responsibility for its accuracy, security, licensing compliance, and inclusion in this project.

Important

Mock-only limitation: this PR does not provide full-capacity MI300X mocking. A real MI300X reports about 196288 MiB; the generic mock expands that into 196288 Device Plugin API objects, so kubelet rejects the roughly 6.76 MiB ListAndWatch response at its default 4 MiB gRPC limit. The released AMD device plugin does not use this representation: it registers ten logical amd.com/gpu splits per physical card and carries full memory/CU values in the Node annotation. This limitation therefore applies to this mock implementation, not to the real AMD plugin or, by itself, to HAMi Scheduler.

The 32 GiB example in this PR is only a transport-safe smoke test for annotation decoding, mock resource registration, and Scheduler Filter/Bind. It must not be cited as full MI300X capacity coverage.

What

• replace the hard-coded AMD device inventory with hami.io/node-amd-register JSON annotation decoding
• register configured AMD memory and core resources through the existing mock device-plugin manager
• validate the AMD count, memory, and core resource names before initialization
• initialize AMD atomically with the existing device map and reject commonWord collisions
• document a transport-safe Scheduler smoke test, the real AMD split count, and the mock-only high-memory boundary

Why

The previous AMD mock inferred a fixed device model and capacity from the count resource. That diverges from HAMi's current AMD scheduling contract and cannot represent the device inventory described by the Node annotation. This change makes the reducer consume the same device facts that HAMi schedules against, while explicitly retaining the generic mock transport limitation above.

AMD resource semantics

• the configured count resource (normally amd.com/gpu) remains an external health gate; the mock does not recreate or sum it
• the released AMD device plugin uses count: 10 and advertises ten amd.com/gpu splits per physical card; examples and fixtures preserve that scheduling contract
• gpumem is the sum of devmem for healthy annotation entries only
• each healthy physical GPU contributes 100 to gpucores, because HAMi exposes AMD core requests as percentages
• annotation devcore remains the physical CU count used by the Scheduler to convert percentages into CUs

Known blocker: full MI300X memory in the mock

This Draft does not provide a valid full-memory MI300X mock. The released AMD device plugin reports roughly 196288 MiB for one MI300X VF. The generic mock represents each scalar MiB as one Device Plugin API object, producing a roughly 6.76 MiB ListAndWatch response; kubelet's default gRPC receive limit is 4 MiB, so it drops amd.com/gpumem and the Node shows 0.

Shortening mock IDs can make one card fit but fails again for multiple cards; splitting ListAndWatch is invalid because every response must be complete. Under the current scalar-resource design, preserving MiB semantics requires a matching AMD memoryFactor in both HAMi Scheduler and this mock, or a different end-to-end representation. This PR implements neither. Until that contract exists, use the reduced 32 GiB fake inventory only for the limited smoke scope stated above.

Impact

AMD mocking is enabled when all three AMD resource names are configured. An entirely empty AMD block remains disabled; partial, invalid, mixed-vendor, duplicate-resource, or conflicting configurations fail closed before replacing the global device map. Existing NVIDIA, Hygon, and Ascend behavior is unchanged.

Validation

• go test ./... on Go 1.21 and the host toolchain
• go test -race ./...
• go vet ./...
• focused AMD/config tests, including collision and failed-initialization atomicity
• scoped golangci-lint
• gofmt -d
• git diff --check

K3s v1.32.5 + HAMi v2.10.0 E2E confirmed both sides of the mock boundary: a nominal MI300X annotation failed at exactly 6,770,170 > 4,194,304 bytes before Scheduler participation, while a separate 32 GiB control run registered all AMD resources and scheduled a Pod to Running; its 50% core request was correctly converted to 152 of the annotation's 304 physical CUs.

[hami-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: spencercjh
Once this PR has been reviewed and has the lgtm label, please assign archlitchi for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Warning

Review limit reached

Next included review available in 16 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e2790d4f-8b43-40b6-8c7c-13fe0eb646b8

📥 Commits

Reviewing files that changed from the base of the PR and between d3b1687 and 84be8fb.

📒 Files selected for processing (5)

• README.md
• internal/pkg/api/device/amd/device.go
• internal/pkg/api/device/amd/device_test.go
• internal/pkg/config/amd_test.go
• internal/pkg/config/config.go

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[spencercjh]

Local K3s test: AMD vGPU

Tested with the released HAMi chart v2.10.0.

Setup

1. Created a clean one-node K3s v1.32.5 cluster.
2. Installed HAMi v2.10.0 with devicePlugin.enabled=false, mockDevicePlugin.enabled=true, and scheduler.leaderElect=false.
3. Built this PR locally and used mock-device-plugin:pr27-090ecaf in the Mock DaemonSet.
4. Added amd.com/gpu=1 to Node status as the health gate.
5. Added the documented MI300X Node annotation:

hami.io/node-amd-register=[{"id":"AMD-MOCK-0","count":1,"devmem":196608,"devcore":304,"type":"AMD-MI300X","health":true,"numa":0}]

Documented MI300X capacity: FAIL

The Mock correctly calculated and logged:

Add resources gpumem=196608 gpucores=100
Device Registered gpumem 196608
Device Registered gpucores 100

However, the kubelet rejected the memory device-plugin response:

ListAndWatch ended unexpectedly for device plugin
rpc error: code = ResourceExhausted desc = grpc: received message larger than max (6770170 vs. 4194304)
resource=amd.com/gpumem

The resulting Node status was therefore:

{
  "amd.com/gpu": "1",
  "amd.com/gpucores": "100",
  "amd.com/gpumem": "0"
}

So the README MI300X example (devmem: 196608) cannot currently be used on a real kubelet. The mock represents each MiB as one device-plugin device, and the 196608-device ListAndWatch response exceeds the default 4 MiB gRPC receive limit.

Smaller capacity and Scheduler path: PASS

For a separate control run, I changed only devmem to 32768 and restarted the task-local Mock DaemonSet after the failed stream. The Node then showed:

{
  "hami.io/node-amd-register": "[{...AMD-MOCK-0...devmem:32768...}]",
  "amd.com/gpu": "1",
  "amd.com/gpumem": "32768",
  "amd.com/gpucores": "100"
}

I created this Pod:

resources:
  limits:
    amd.com/gpu: "1"
    amd.com/gpumem: "512"
    amd.com/gpucores: "50"

It was Running on the test Node through hami-scheduler. The Pod annotations showed the expected physical device and converted core request:

hami.io/amd-devices-to-allocate=AMD-MOCK-0,AMD-MI300X,512,152:;
hami.io/amd-devices-allocated=AMD-MOCK-0,AMD-MI300X,512,152:;
hami.io/vgpu-node=k3d-hami-mock-pr27-server-0

This confirms the annotation decoding, resource registration, Node presentation, and HAMi scheduling path work for a capacity below the gRPC message limit. The documented MI300X 196608 MiB case is still a real failure and needs a representation that does not create one mock device per MiB. I did not change product code. The test cluster was removed after collecting the result.

[moezdil]

Old PRs and issues are being closed because they are no longer relevant. Or it was closed because of th
inactivity or PR/issues owners.
If you believe the issue is still relevant, please submit feedback.