Repository navigation
processor: fuse compatible body content modifier chains - #12505
Merged
Merged
Conversation
|
Caution Review failedThe pull request is closed. Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
edsiper
force-pushed
the
perf/5.2-candidates
branch
from
October 26, 2026 19:30
4c27c12 to
d9662be
Compare
edsiper
marked this pull request as ready for review
October 26, 2026 21:38
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compatible native body-edit chains still spend CPU materializing CFL objects after the FTS merge. Fuse a complete compatible segment into one MessagePack decode/edit/encode pass for ordered body
insert,upsert,deleteandrenameactions.The planner respects starting stages and filter boundaries, checks every unit, acquires stage locks in configuration order, and updates each stage's metrics. Conditions, mixed plugins, group markers, legacy records and unsupported/deep payloads retain the current FTS windowed/CFL path. Partial private output is discarded before fallback.
The rebase preserves FTS's per-invocation codecs, bounded windows, window flags and
flb_processor_run_with_count()interface. Known logical counts propagate through raw segments. Fully decoded CFL chains count live records for following stages, fixing stale per-stage input/drop accounting after native drops; that bug reproduces on unmodified post-FTS master.The SQL envelope fix remains independent in #12506. Worker codec isolation is already in master; #12511 now supplies additional regression tests rather than a production prerequisite.
Validation
From the repository root, with
FLUENT_BIT_BINARYpointing at this branch's build:python -m pytest tests/integration/scenarios/processor_content_modifier -q VALGRIND=1 VALGRIND_STRICT=1 python -m pytest tests/integration/scenarios/processor_content_modifier -q ctest --test-dir build -R '^flb-(it-(processor|processor_conditional|mp_chunk_cobj)|rt-processor_content_modifier)$' --output-on-failureFresh post-FTS measurements
Post-FTS master
05048e35dversus the reviewed candidate, verified-O3 -DNDEBUG, Linux/Ryzen 9 7940HS, CPU 7 affinity, alternating fresh processes, four clients, 1,000 records per batch and 10,000 warmup records. Confirmation uses five pairs per case, 800,000 measured records/run for 12 fields and 200,000 for 128 fields.These replace the pre-FTS performance claims. Compatible plain-log fusion remains useful; CPU savings do not always increase throughput. Mixed Forward chains with a CFL-only hash tail changed CPU by only 1.2–1.8%, so no meaningful mixed-chain speedup is claimed.
Grouped OTLP declines raw fusion and exercises live counting instead. Initial ingestion controls varied by about ±6%, so not every observed grouped-fallback change is attributed to this PR. No general fallback speedup or memory-saving claim is made. The host was shared; confirmation ran after the competing benchmark and integration tests finished, without reserving a CPU.
Across the initial and confirmation campaigns, all 160 runs delivered exactly 44 million measured records, plus 1.6 million warmup records. The measured review candidate also contained #12506's SQL fix; none of these benchmark pipelines invokes SQL. Binary/payload hashes, per-run CSVs, invocation metadata and the full report are committed under
benchmarks/results/post-fts-12505.Repeat with
--modifiers 0for a control. SetFLB_DEBUG=OffandFLB_RELEASE=OffwithCMAKE_BUILD_TYPE=Release, and inspect the actual compiler flags.No public configuration or bundled-library changes. Stress tests and Memcheck are not a race-detector proof. Target: master.