Skip to content

[eno][tombstone-recovery 5/7] Record Ready-synthesis inventories as Secrets - #661

Merged
Ruinan Liu (ruinan-liu) merged 8 commits into
mainfrom
users/ruinanliu/ruinanliu/backup-5-inventory-record
Oct 16, 2026
Merged

Ruinan Liu (ruinan-liu) merged 8 commits into
mainfrom
users/ruinanliu/ruinanliu/backup-5-inventory-record

Conversation

@ruinan-liu

Copy link
Copy Markdown
Collaborator

Summary

Add inventory recording to the tombstone recovery controller. Once an eligible Composition's current synthesis is Ready, save its resource inventory as Secrets in the downstream cluster's kube-system namespace.

Switch recovery from ConfigMap inventories to Secret snapshots.

Behavior

  • Record only when the operator is enabled, the Composition is in scope and opted in, and its current synthesis is Ready with a finished recovery decision.
  • Read all current ResourceSlices before preparing inventory; stop on missing, terminating, unreadable, or malformed slices.
  • Include resource identities, labels, and readiness/deletion-group annotations. Exclude tombstones and Patch resources.
  • Split inventory deterministically into Secret chunks within the 1 MiB data limit.
  • Use stable names: <comp-name>-<inventory-lineage>-<chunk-index>.
  • Record empty inventory as one Secret containing [].
  • Reuse existing Secrets on retries and avoid rewriting unchanged snapshots.

Recovery and Safety

  • Find snapshots by lineage and select the newest source synthesis timestamp.
  • Require a complete set of chunks with consistent snapshot metadata.
  • Skip missing, mixed, malformed, or timestamp-ambiguous snapshots with InventoryInvalid; do not generate tombstones from ambiguous history.
  • Recheck Composition identity, synthesis UUID, and recording eligibility immediately before writes.
  • Use resourceVersion-protected updates to prevent overwriting concurrent changes.
  • Preserve inventory lookup across Composition recreation without requiring the previous Composition UID.

Limitations and Scope

  • Multi-Secret replacement is not atomic. A partial replacement may cause recovery to skip historical cleanup; this is an intentional best-effort tradeoff.
  • Ignore leftover chunks after a smaller replacement; general inventory garbage collection and Composition-deletion handling remain separate work.
  • No ConfigMap compatibility is retained because that inventory format has not been released.
  • Inventory contains resource identities and deletion metadata, not full resource manifests.

@ruinan-liu Ruinan Liu (ruinan-liu) changed the title [eno] Record Ready-synthesis inventories as Secrets [eno][tombstone-recovery 5/7] Record Ready-synthesis inventories as Secrets Oct 8, 2026
@ruinan-liu
Ruinan Liu (ruinan-liu) force-pushed the users/ruinanliu/ruinanliu/backup-5-inventory-record branch from 3a8ed81 to d928400 Compare October 9, 2026 20:52
@ruinan-liu
Ruinan Liu (ruinan-liu) changed the base branch from users/ruinanliu/backup-2-core to main October 9, 2026 20:53
@ruinan-liu
Ruinan Liu (ruinan-liu) marked this pull request as ready for review October 12, 2026 16:56
@ruinan-liu
Ruinan Liu (ruinan-liu) force-pushed the users/ruinanliu/ruinanliu/backup-5-inventory-record branch from cf3920f to 214fa64 Compare October 13, 2026 20:07
Comment thread internal/controllers/tombstonerecovery/inventory.go
@xiazhan

xiazhan commented Oct 14, 2026

Copy link
Copy Markdown
Collaborator

The secret or configmap has 1mb size limit. Does control-plane-core beyond this limit? I think it doesn't because we only store meta data.

Comment thread internal/controllers/tombstonerecovery/inventory.go
@ruinan-liu

Copy link
Copy Markdown
Collaborator Author

The secret or configmap has 1mb size limit. Does control-plane-core beyond this limit? I think it doesn't because we only store meta data.

Yes, you are right. It won't hit the limit. Inventory only stores resource identity (group/version/kind/namespace/name), labels and the eno.azure.io/readiness-group and eno.azure.io/deletion-group annotations, not specs or data. If it ever exceed 1mb, recording splits it across multiple secrets.

@ruinan-liu Ruinan Liu (ruinan-liu) left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reply to comment

Comment thread internal/controllers/tombstonerecovery/recording.go

@ruinan-liu Ruinan Liu (ruinan-liu) left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reply to comment

@ruinan-liu Ruinan Liu (ruinan-liu) left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reply to comment

@ruinan-liu
Ruinan Liu (ruinan-liu) merged commit f7cf6bc into main Oct 16, 2026
99 of 100 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants