Skip to content

Prepare Ruby serial ID metadata system tests - #7856

Draft
hhan2024 wants to merge 9 commits into
mainfrom
ffe-ruby-serial-id-metadata
Draft

hhan2024 wants to merge 9 commits into
mainfrom
ffe-ruby-serial-id-metadata

Conversation

@hhan2024

@hhan2024 hhan2024 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stacked directly on #7820. Return Ruby OpenFeature evaluation details, including actual SDK flag metadata, from the parametric adapter.

Dependency / enablement

Depends on the Ruby SDK provider fix: DataDog/dd-trace-rb#6400.
That change copies the evaluated split serial ID into __dd_split_serial_id when span enrichment is enabled, preserving zero and omitting a missing ID.

The serial-ID manifest intentionally remains missing_feature: no released Ruby SDK version is yet verified to contain this patch. Keep this PR draft until the SDK change is merged and its release floor is assigned; then replace the manifest marker with that version.

Validation

  • Candidate Ruby SDK 2.44.0-dev with provider patch: all 5 serial-ID tests XPASS with -F missing_feature (positive, zero, missing, targeting, repeated).
  • 24 dynamic-evaluation regression tests pass.
  • SDK OpenFeature suites: Ruby 3.1, 3.2 and 4.0, each with latest and minimum OpenFeature SDK; 316 examples per run, zero failures (1,896 total).
  • ./format.sh, Ruby syntax check, SDK RuboCop (1,978 files), and full SDK Steep type check pass.

No serial IDs are synthesized or normalized by the test adapter, and shared assertions are unchanged.

hhan2024 and others added 8 commits September 24, 2026 18:30
Every UFC split has a stable per-org serialId used by downstream consumers
(APM span enrichment, exposure caching, evaluation-metrics correlation) to
identify the exact (allocation, variant) pair that was assigned. Today only
the Node server SDK (openfeature-js-client#269) exposes this via
flagMetadata["__dd_split_serial_id"]; this test generalizes the contract
so every SDK can be validated the same way.

Adds a dedicated fixture covering: a plain serialId, serialId=0 (a valid ID
that must not be dropped by a truthiness check), a split with no serialId
(must not fabricate one), and two allocations with different serialIds
selected via targeting.

All languages are marked missing_feature: verified directly against the
Node.js and Go parametric weblog handlers that neither returns flagMetadata
from POST /ffe/evaluate today.

The new feature_flags_serial_id_metadata marker uses NOT_REPORTED_ID
pending a real feature-parity page.
Exclude the UFC configuration from evaluation-case discovery, disable the
new suite for Rust, and require successful evaluation before accepting an
absent serial ID. Apply the repository formatter to the new test module.

Environment: Datadog workspace
Fail evaluations that omit flagMetadata while allowing null or empty metadata when a split has no serial ID.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
Tighten the missing-serial-ID test so an SDK cannot serialize an absent UFC serial ID as a null metadata entry.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
Exercise serial-ID flag metadata under the required span-enrichment configuration for positive, zero, and missing UFC serial IDs.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
Fix the import-policy failure when the PR is tested against current main.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
Bring in the exact upstream facade so imports work on the PR head as well as merged main.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
Return SDK evaluation details so serial ID metadata tests can exercise Ruby support.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

utils/build/docker/ruby/parametric/server.rb                            @DataDog/system-tests-reviewers

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7ea591b | Docs | View more details | Give us feedback!

Start the provider without injecting the UFC fixture after Remote Config acknowledgement.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
Base automatically changed from ffe-serial-id-metadata-system-test to main October 16, 2026 17:03

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants