Skip to content

Omit onboarding credentials from pytest failure reports - #8203

Open
dmalysza wants to merge 3 commits into
mainfrom
damian.malysza/no-jira/system-tests-ci-header-output
Open

dmalysza wants to merge 3 commits into
mainfrom
damian.malysza/no-jira/system-tests-ci-header-output

Conversation

@dmalysza

@dmalysza dmalysza commented Oct 28, 2026 •

Copy link
Copy Markdown

Onboarding backend failures expose DD-API-KEY and DD-APPLICATION-KEY through pytest failure locals. SDS-flagged CI logs identify the shared onboarding header producer, also used by auto_inject and dd-trace-py.

Use pytest's native __tracebackhide__ in the three helpers that retain credential-selection values or authentication headers: _headers, _make_request and _query_for_profile. Pytest omits those internal frames and their locals entirely. Public caller frames and the existing method, URL and timeout error remain visible. Credential values, request preparation, key-pair selection, retries, backoff, validation and ordinary logs are unchanged. This addresses the onboarding header producer; other credential detections require separate attribution.

Key files to review

  • Credential-handling helpers: only pytest's traceback filtering changes. Check that the three credential-retaining frames are omitted while the public caller and existing failure details still identify the failing operation.

Validation

  • Existing tests only: ./run.sh TEST_THE_TEST tests/test_the_test/test_junit.py tests/test_the_test/test_itself.py passed all 3 selected tests, covering JUnit failure reporting and scenario coherence. No unit tests or assertions are added by this PR.
  • ./format.sh --check: all checks passed, including mypy, import policy, formatting, YAML, manifest checks, shellcheck and Node linters.
  • git diff --check: passed. The final diff contains only 9 lines in the three existing credential-retaining helpers.
  • No live authenticated requests or CI infrastructure provisioned locally. CI is rerunning for the final commit.

Assisted by Codex (GPT-6).

Preserve credential string values for HTTP authentication while redacting their repr in request and key-selection locals.

Assisted by Codex (GPT-6).
@dmalysza
dmalysza requested review from a team as code owners October 28, 2026 01:17
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T01:19:29.575363Z 70c3d48 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

utils/onboarding/backend_interface.py                                   @DataDog/system-tests-core

Use native traceback filtering to omit authentication locals while retaining the public caller and operation error. Restore ordinary credential strings and HTTP preparation.

Assisted by Codex (GPT-6).
@dmalysza dmalysza changed the title Hide onboarding API keys in pytest failure locals Omit onboarding credentials from pytest failure reports Oct 28, 2026
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Pipelines  Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a9a9a4b | Docs | View more details | Give us feedback!

Keep the native pytest traceback omission scoped to the existing credential-retaining helpers and validate with existing repository checks.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-6 <noreply@localhost>
@gh-worker-ownership-write-b05516
gh-worker-ownership-write-b05516 Bot removed the request for review from a team October 28, 2026 03:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant