Repository navigation
Conversation
Port FFL-3407 coverage onto current main and exclude the PHP native IPC regression from other SDKs. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost> Co-authored-by: Vickie Boettcher <vickie.fridge@datadoghq.com>
Separate context count and value bounds, validate reported capacity drops in an isolated scenario, and add contract regression coverage. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
|
|
❌ ErrorsYour PR has failed checks. Please review the issues below and take necessary action before merging. 🚦 12 Pipeline jobs failed
ℹ️ InfoNo other issues found (see more)❄️ No new flaky tests detected Useful? React with 👍 / 👎 This comment will be updated automatically if new data arrives.🔗 Commit SHA: 2f0b9f6 | Docs | View more details | Give us feedback! |
Require complete context at the field limit and preserve results, identity, and counts when oversized contexts are pruned or omitted. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Merge main at f8d7b76, including the gqlgen fixture upgrade; preserve the feature-flag evaluation harness changes. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Retain compact targeting keys from #8133 while preserving isolated 12000-evaluation accounting and native cap-boundary coverage. Add compact-payload and multi-window contract regressions; document why doubling the workload cannot guarantee degradation across arbitrary flushes. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Merge main at 3f9f99b and retain all FFE regression coverage under the upstream TEST_THE_TEST relocation. Resolve README instructions and migrate added tests off the removed contract-only scenario. Environment: Datadog workspace Co-Authored-By: Codex GPT-5 <noreply@localhost>
Keep the shared suite focused on context bounds and overload accounting; cover native IPC degradation through the PHP API-to-wire tests. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Describe flush timing without claiming exact-cap coverage in SDK tests. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Keep the shared contract, run command, and isolation requirement; link to implementation details. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Check workload properties and cover each malformed capture once while retaining valid flush layouts. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Remove the redundant class-level missing-feature declaration. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Pass Jackson Long attributes through OpenFeature numeric context instead of replacing them with null. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Build typed OpenFeature values instead of stringifying numeric, boolean, and structured evaluation attributes. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Bring in current main and rerun CI after investigating the Node shutdown timing failure and Payara startup failure. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6be98ef02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Remove the FFE self-tests added by this PR while preserving the pre-existing coverage. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
sameerank
left a comment
There was a problem hiding this comment.
A couple non-blocking suggestions!
aarsilv
left a comment
There was a problem hiding this comment.
Nice work! Approving with some comments you can take or leave as you see fit.
Preserve the isolated stress scenario in the refactored FFE module and retain typed .NET context conversion alongside the opt-out scenarios. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Preserve longer capture waits while allowing telemetry to flush, gate Go drop accounting at 2.11.0, and apply shared outcome checks to oversized values. Clarify counter units and batch semantics. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Include the matching Java development provider download from #8188 and current main updates. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Include the latest main changes while preserving the FFE repair scope. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Preserve the FFE harness repairs alongside the merged SDK identity tests from PR 8153. Environment: Datadog workspace Co-Authored-By: Codex GPT-6 <noreply@localhost>
Why
FFL-3407 makes the shared flag-evaluation tests check SDK behavior reliably, without depending on flush timing or which context fields an SDK keeps when it reaches a limit.
What changed
Long; pass them through OpenFeature's numeric API instead of replacing them with null. The existing full-consent test verifies that the numericorg_idsurvives unchanged.org_id.No production SDK timing changes or test-only SDK APIs are needed. The PHP and Python manifest edits preserve the existing disabled status when renaming the stress test; enabling supported SDK versions remains a separate follow-up.
PHP-specific oversized-input and same-request recovery coverage belongs in the PHP API-to-wire suite and is proposed in dd-trace-php #4364. This PR focuses on shared SDK contracts. PHP activation remains separate work in FFL-3020.
How to test
Run the contract tests and repository checks:
With the system-tests runner and Docker available, build a Go weblog and run the SDK-backed checks:
For Node.js, build with
./build.sh nodejs -w express5and repeat the two runs with--library nodejs.After merging main at
d454d0d07, all 242 focused framework tests passed. Following review fixes, 157 focused checks and full formatting/static checks passed. Scratch checks confirmed Go version gates and the 15-second minimum without shortening longer waits; mutations of a live value-bounds capture were rejected for incorrect counts, identity, outcomes, and errors. No synthetic self-tests were added to the repository. See the PR checks for current CI results.Fresh review validation on Go 2.11.0 and Node.js 6.20.0, through Agent 7.84.2: all three context-boundary cases and the isolated stress scenario (accounting plus two schema checks) passed for each SDK. Node was forced past its manifest gate; Go ran with its normal gate. Both stress runs used the 15-second library wait. These runs do not force drops or prove every possible flush schedule.
Earlier forced SDK runs covered the 13 evaluation tests and isolated stress test, including cases disabled by manifests:
These SDK runs had no skips and preceded the latest main merge. The .NET and PHP results are development-build results, not validation of published 3.57.0/1.27.0 releases. Separate agentless scenarios were outside this local SDK validation matrix. No all-SDK green result is claimed.
Review and follow-up
Please review the shared context rules, workload accounting, and isolated scenario/CI integration.
FFE reviewer selection
Reviewers: @leoromanovsky, @sameerank
Why: Flag evaluation on EVP — The PR changes the shared system tests for flag-evaluation telemetry sent over EVP. Reviewers come from the matched pod.
🤖 Automated selection.
Selection details
Initiative: project / flag-evaluation-evp
Why this match: The PR changes the shared system tests for flag-evaluation telemetry sent over EVP. It adds context-bound checks for fully consented evaluation data. These verify that 256 fields are kept at the limit and that oversized values are omitted without truncation, while identity and the evaluation count stay intact. It also replaces the degradation test with an isolated high-cardinality stress scenario. That scenario checks that delivered evaluations plus reported capacity drops add up to the full workload. Finally, it fixes the Java and .NET weblogs so numeric and typed JSON context reaches OpenFeature with its original types, which the full-consent payload tests depend on. All of this is about EVP evaluation payloads, consent and delivery, and none of it is onboarding diagnostics or bandits. The catalog has no entries with supplemental review enabled, so no additional areas apply.
The PR author was excluded. Existing reviewers and teams were preserved.
Debug or fix a problem
Workflow run · Bits Code session
dry_run=false. Unchanged completed runs are skipped. If the receipt saysneeds_reconciliationor a write outcome is uncertain, ask the workflow owner to reconcile GitHub and the receipt before retrying; do not delete the lock blindly.Evidence and run details
✅ Complete — reviewer coverage verified. The workflow checked pending requests or submitted reviews on the evaluated head.
Generated asynchronously by the FFE reviewer-routing workflow.
Changed paths:
tests/ffe/test_flag_eval_evp.py,tests/ffe/utils/evp.py,tests/ffe/README.md,utils/_context/_scenarios/feature_flagging.py,utils/_context/_scenarios/__init__.py,.github/workflows/run-end-to-end.yml,utils/build/docker/java/spring-boot/src/main/java/com/datadoghq/system_tests/springboot/featureflag/FeatureFlagEvaluatorController.java,utils/build/docker/dotnet/weblog/Controllers/FeatureFlagEvaluatorController.cs,manifests/golang.ymlEvaluated head:
9b6ae1cbd822fcba0a2cf10da318188dbfcd11beRegistry revision:
revision:1946aa2a-530e-487a-84a7-83aff44a1089Pinned skill source:
d214cfc65ae154b97d39af5cf32caf48aaec9bf0Instructions: workflow-injected snapshot (no plugin loaded)
Instruction snapshot SHA-256:
f23b2a88c13c0da90e2dd58246a14f257af91e2195c555039bd555b52e6f3516