Conversation
Export SONIC_BASE_CONFIG_PATH in sonic_golden_test.sh, pointing it at
the in-repo files/sonic/config_db.json next to the existing
SONIC_PORT_CONFIG_PATH, and regenerate all nine goldens against it.
The harness set no base config path before this, so the generator fell
back to /etc/sonic/config_db.json -- a path the Containerfile creates
inside the conductor image, and which does not exist on the host the
harness runs on. Every device started from {}, and the goldens pinned
output for a configuration that never occurs in production. The path
itself stopped being hardcoded when SONIC_BASE_CONFIG_PATH was added;
the harness simply never set it.
Regenerating adds 24 tables to every golden -- those populated in the
shipped base and never emitted by the generator, among them FEATURE,
TELEMETRY, SWITCH, ZTP and the POLICY_* and ECMP_LOADSHARE_* groups.
They reach real switches by passing through untouched, so nothing
covered them before. The union of populated tables across the set goes
from 38 to 62 and no golden loses one.
DEVICE_METADATA.localhost changes content rather than gaining presence,
inheriting default_config_profile, frr_mgmt_framework_config,
intf_naming_mode and type. That is INHERITED_TABLE_KEYS working as
documented: purely additive, with no generated field overwritten.
The inherited type is LeafRouter on all nine devices, spine and OOB
included, because the shipped base hardcodes it and the generator never
sets that field. That is what production gets; the empty base was hiding
it.
Verified by regenerating against a fresh stack: 3483 unit tests pass,
the committed-artifact validator accepts all nine enriched goldens, and
the golden coverage gate still passes.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Roger Luethi <luethi@osism.tech>
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="tests/e2e/sonic_golden_test.sh" line_range="207" />
<code_context>
export SONIC_EXPORT_DIR="${EXPORT_DIR}"
export SONIC_EXPORT_IDENTIFIER="hostname"
export SONIC_PORT_CONFIG_PATH="${REPO_ROOT}/files/sonic/port_config"
+export SONIC_BASE_CONFIG_PATH="${REPO_ROOT}/files/sonic/config_db.json"
echo ">>> Generating SONiC configurations (tests/e2e/generate.py)"
</code_context>
<issue_to_address>
**issue (bug_risk):** All nine generated configurations now inherit `DEVICE_METADATA.localhost.type` as `LeafRouter` from the shared base, including the spine and OOB devices. The committed goldens therefore pin role metadata that is incorrect for those devices, and deploying those generated configs labels non-leaf switches as leaf routers.
**Triggers:** When the generated SONiC configuration is consumed for the spine or OOB fixtures.
**Suggested fix:** Make the base metadata role-neutral or set `DEVICE_METADATA.localhost.type` from the NetBox device role during generation, then regenerate the goldens.
</issue_to_address>Sourcery assessment
Approval pending. 1 finding to address first.
Blocking findings: tests/e2e/sonic_golden_test.sh:207
| export SONIC_EXPORT_DIR="${EXPORT_DIR}" | ||
| export SONIC_EXPORT_IDENTIFIER="hostname" | ||
| export SONIC_PORT_CONFIG_PATH="${REPO_ROOT}/files/sonic/port_config" | ||
| export SONIC_BASE_CONFIG_PATH="${REPO_ROOT}/files/sonic/config_db.json" |
There was a problem hiding this comment.
issue (bug_risk): All nine generated configurations now inherit DEVICE_METADATA.localhost.type as LeafRouter from the shared base, including the spine and OOB devices. The committed goldens therefore pin role metadata that is incorrect for those devices, and deploying those generated configs labels non-leaf switches as leaf routers.
Triggers: When the generated SONiC configuration is consumed for the spine or OOB fixtures.
Suggested fix: Make the base metadata role-neutral or set DEVICE_METADATA.localhost.type from the NetBox device role during generation, then regenerate the goldens.
Exports
SONIC_BASE_CONFIG_PATHintests/e2e/sonic_golden_test.sh, pointing itat the in-repo
files/sonic/config_db.json, and regenerates all nine goldensagainst it.
Why
The harness set no base config path, so the generator fell back to
/etc/sonic/config_db.json— a path the Containerfile creates inside theconductor image, and which does not exist on the host the harness runs on. Every
device started from
{}and logged the base config as not found, so the goldenspinned output for a configuration that never occurs in production.
The path stopped being hardcoded when
SONIC_BASE_CONFIG_PATHwas added; theharness simply never set it. This is the one-line follow-up that takes advantage
of it, and it closes the base-config gap described under "Scope of the coverage
claim" in the series context issue.
What the diff contains
24 tables appear in every golden that appeared in none before — those populated
in the shipped base and never emitted by the generator:
They reach real switches by passing through untouched, so nothing covered them
before. The union of populated tables across the golden set goes from 38 to 62,
and no golden loses a table — which is what the regeneration coverage guard
checks.
DEVICE_METADATA.localhostis the one table that changes content rather thangaining presence: it inherits
default_config_profile,frr_mgmt_framework_config,intf_naming_modeandtype. That isINHERITED_TABLE_KEYSworking as documented — purely additive, no generatedfield overwritten.
One thing worth a second look
The inherited
typeisLeafRouteron all nine devices, spine and OOBincluded, because the shipped base hardcodes it and the generator never sets
that field. So every switch deployed from this base is labelled
LeafRouterregardless of role. That is pre-existing production behaviour, not something this
change introduces — but it was invisible while the goldens had no base, and it is
worth deciding whether it matters before it is pinned as expected output.
Reviewing this
The diff is
+3681 / -17, but only one line of it is logic. The nine goldens arenear-identical
+409blocks of the same base content, so: read the harness line,read one golden in full, spot-check the rest.
Verified
Regenerated against a fresh NetBox stack, not a warm one, so the result is
reproducible in CI:
test_validator_artifactsaccepts all nine enriched goldens — the merged basecontent validates, which is the question the regeneration existed to answer
python-osism-sonic-e2ejob will re-derive these goldens on this PR; itsfile matcher covers both
files/sonic/andtests/e2e/🤖 Generated with Claude Code