Skip to content

processor_sql: preserve resource and scope groups when filtering logs - #12506

Merged
edsiper merged 2 commits into
masterfrom
fix/sql-group-markers
Oct 26, 2026
Merged

edsiper merged 2 commits into
masterfrom
fix/sql-group-markers

Conversation

@edsiper

@edsiper edsiper commented Oct 2, 2026 •

Copy link
Copy Markdown
Member

SQL currently evaluates resource/scope transport markers as log rows. A WHERE clause can delete those markers and strip metadata from surviving OTLP logs; projection can also alter the envelopes. This remains reproducible after the FTS merge.

Skip non-normal log events before applying SQL row processing. Existing group normalization removes envelopes whose logs were all dropped.

For two resource groups containing IDs 1–4, SELECT * FROM STREAM WHERE id = 1; retains ID 1 with its original resource/scope metadata, removes the now-empty second group, and reports one logical output record. The fix is independent of processor fusion in #12505.

Rebased onto master 05048e35d. Validation:

  • Unmodified post-FTS master fails four metadata-preservation cases; its two empty-batch cases pass.
  • This branch's six integration cases pass normally and under strict Valgrind: keep all groups, partial/whole-group drops, and empty batch, in input/output contexts with filesystem storage and two-route fan-out.
  • Tests compare resources/scopes, schema URLs, body, timestamps, severity, trace/span IDs, flags, attributes and logical counters.
  • All six Memcheck processes report zero errors and zero definite/indirect/possible leaked bytes.
  • flb-it-processor_sql passes; full rebased PR range passes prefix lint.

From the repository root, with FLUENT_BIT_BINARY pointing at this branch's build:

python -m pytest tests/integration/scenarios/processor_sql -q
VALGRIND=1 VALGRIND_STRICT=1 python -m pytest tests/integration/scenarios/processor_sql -q
ctest --test-dir build -R '^flb-it-processor_sql$' --output-on-failure

No public configuration, packaging or bundled-library changes. Target: master.

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T21:38:12.799398Z 1a2990f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@edsiper
edsiper merged commit 7d80307 into master Oct 26, 2026
58 of 65 checks passed
@edsiper
edsiper deleted the fix/sql-group-markers branch October 26, 2026 21:37

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant