Skip to content

Add per-VNI multisite ingress replication (Cisco NX-OS) - #577

Merged
adamtrizuljak-sap merged 1 commit into
mainfrom
feat/nve-multisite-ingress-repl
Sep 23, 2026
Merged

adamtrizuljak-sap merged 1 commit into
mainfrom
feat/nve-multisite-ingress-repl

Conversation

@adamtrizuljak-sap

@adamtrizuljak-sap adamtrizuljak-sap commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Adds support for per-VNI multisite ingress replication on Border Gateway (BGW) nodes. Because this is a Cisco NX-OS–specific feature with no OpenConfig equivalent, it is exposed through a new provider-specific configuration resource, EVPNInstanceConfig, rather than on the vendor-neutral core API.

When enabled, BUM traffic for a VNI is replicated to remote VTEP peers in the multisite domain via ingress replication. Three modes are supported: Disabled (default), Enabled, and EnabledOptimized.

This maps to the NX-OS CLI:

member vni {vni}
multisite ingress-replication

and the YANG leaf multisiteIngRepl at:

System/eps-items/epId-items/Ep-list[epId=1]/nws-items/vni-items/Nw-list[vni={vni}]/multisiteIngRepl

⚠️ Setting multisiteIngRepl: Enabled requires evpn multisite border-gateway to be enabled, which is not yet supported in network-operator. See the Testing section for details. This PR is a pre-requisite for supporting evpn multisite border-gateway.

Changes

API

  • New EVPNInstanceConfig resource under api/cisco/nx/v1alpha1 with the MultisiteIngressReplication enum field (Disabled/Enabled/EnabledOptimized, default Disabled), registered as an EVPNInstance dependency.

Provider (Cisco NX-OS)

  • EnsureEVPNInstance reads multisiteIngressReplication from the referenced EVPNInstanceConfig and maps it to the VNI's device enum.
  • VNI struct/mapping updates in nve.go for the three modes.

Generated / scaffolding

  • New CRD (base + Helm chart), RBAC, deepcopy, API reference docs, PROJECT and
    kustomization entries.

Samples

  • New EVPNInstanceConfig sample plus Tiltfile wiring; the EVI sample now references it via providerConfigRef (commented out by default).

Generated artifacts

  • Regenerated CRDs (config/crd/bases), Helm chart CRD copy, and API reference
    docs to reflect the new enum field and its Disabled default.

Notes

  • The field is optional and defaults to Disabled, so existing EVPNInstance resources are unaffected.

Testing

  • Provider payload golden tests cover all three enum values (disable, enable, enableOptimized).
  • gNMI integration test (evpninstance.txtar) updated: a default EVPNInstance now emits "multisiteIngRepl": "disable".
  • make manifests generate helm fmt lint test and make test-gnmi PROVIDER=cisco-nxos-gnmi all pass.

Simulated device

Tested against a simulated Cisco N9K-C9300v in Clabernets. Results of reading the path using gnmic -a 127.0.0.1 --port 9339 -u *** -p *** --skip-verify get -e json --values-only --path 'System/eps-items/epId-items/Ep-list[epId=1]/nws-items/vni-items/Nw-list[vni=100010]/multisiteIngRepl'

spec.multisiteIngressReplication gNMI value
unset / "Disabled" ["disabled"]
Enabled [] - causes controller error
EnabledOptimized ["enableOptimized"]

Setting multisiteIngRepl: Enabled produces the following error. This is expected, because evpn multisite border-gateway is not yet supported in network-operator. This PR is a pre-requisite for supporting evpn multisite border-gateway.

ERROR	Failed to reconcile resource	{"controller": "evpninstance", "controllerGroup": "networking.metal.ironcore.dev", "controllerKind": "EVPNInstance", "EVPNInstance": {"name":"vxlan-100010","namespace":"default"}, "namespace": "default", "name": "vxlan-100010", "reconcileID": "d48672c4-2103-4197-a001-f72862a26d9e", "error": "gnmiext: failed to perform set rpc: rpc error: code = Internal desc = descr:internal processing error [error:operation-failed msg:Please enable 'evpn multisite  border-gateway' before enabling 'evpn multisite ingress-replication'][error:operation-failed msg:Please enable 'evpn multisite  border-gateway' before enabling 'evpn multisite ingress-replication' Commit Failed]"}

YANG

Reference https://github.com/YangModels/yang/blob/main/vendor/cisco/nx/10.6-3/Cisco-NX-OS-device.yang

Allowed values of multisiteIngRepl:

typedef nvo_MultisiteIngReplStateT {
    type enumeration {
        enum disable {
            value 1;
            description "Disable 'multisite ingress-replication'";
        }
        enum enable {
            value 2;
            description "Enable 'multisite ingress-replication'";
        }
        enum enableOptimized {
            value 3;
            description "Enable 'Optimized multisite ingress-replication'";
        }
    }
    default "disable";
}

@hardikdr hardikdr added the area/switch-automation Automation processes for network switch management and operations. label Sep 18, 2026
@hardikdr hardikdr added this to Roadmap Sep 18, 2026
@adamtrizuljak-sap
adamtrizuljak-sap force-pushed the feat/nve-multisite-ingress-repl branch from aa68674 to 729a3f7 Compare September 18, 2026 08:35
@adamtrizuljak-sap
adamtrizuljak-sap force-pushed the feat/nve-multisite-ingress-repl branch from 729a3f7 to 7ca72f8 Compare September 18, 2026 08:46
@adamtrizuljak-sap adamtrizuljak-sap changed the title Support per-VNI multisite ingress replication in EVPNInstance Add per-VNI multisite ingress replication (Cisco NX-OS) Sep 18, 2026
@adamtrizuljak-sap
adamtrizuljak-sap marked this pull request as ready for review September 18, 2026 08:52
Comment thread api/cisco/nx/v1alpha1/evpninstanceconfig_types.go Outdated
@adamtrizuljak-sap
adamtrizuljak-sap force-pushed the feat/nve-multisite-ingress-repl branch from 7ca72f8 to a79c024 Compare September 22, 2026 15:37
Comment thread api/cisco/nx/v1alpha1/evpninstanceconfig_types.go Outdated
@adamtrizuljak-sap
adamtrizuljak-sap force-pushed the feat/nve-multisite-ingress-repl branch from a79c024 to 994c9f4 Compare September 23, 2026 09:03
…ecific configuration

MultisiteIngressReplication does not have direct support in Openconfig -> we make it provider-specific configuration

Signed-off-by: Adam Trizuljak <adam.trizuljak@sap.com>
@adamtrizuljak-sap
adamtrizuljak-sap force-pushed the feat/nve-multisite-ingress-repl branch from 994c9f4 to 38009cb Compare September 23, 2026 12:13
@github-actions

Copy link
Copy Markdown

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/ironcore-dev/network-operator/api/cisco/nx/v1alpha1 0.00% (ø)
github.com/ironcore-dev/network-operator/internal/provider/cisco/nxos 11.02% (+1.21%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/ironcore-dev/network-operator/api/cisco/nx/v1alpha1/evpninstanceconfig_types.go 0.00% (ø) 4 (+4) 0 4 (+4)
github.com/ironcore-dev/network-operator/api/cisco/nx/v1alpha1/zz_generated.deepcopy.go 0.00% (ø) 598 (+38) 0 598 (+38)
github.com/ironcore-dev/network-operator/internal/provider/cisco/nxos/nve.go 69.23% (-3.50%) 13 (+2) 9 (+1) 4 (+1) 👎
github.com/ironcore-dev/network-operator/internal/provider/cisco/nxos/provider.go 2.11% (+1.73%) 3416 (+1287) 72 (+64) 3344 (+1223) 👍

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/ironcore-dev/network-operator/internal/provider/cisco/nxos/nve_test.go

@adamtrizuljak-sap
adamtrizuljak-sap merged commit e54bc31 into main Sep 23, 2026
26 checks passed
@adamtrizuljak-sap
adamtrizuljak-sap deleted the feat/nve-multisite-ingress-repl branch September 23, 2026 12:21
@github-project-automation github-project-automation Bot moved this to Done in Roadmap Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/switch-automation Automation processes for network switch management and operations. size/L

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants