Skip to content

feat: count only records the destination received (BP-1324) - #345

Merged
Dylan-M merged 3 commits into
dylanmyers/bp-1302-docs-refreshfrom
dylanmyers/bp-1324-output-delivery-count
Nov 2, 2026
Merged

Dylan-M merged 3 commits into
dylanmyers/bp-1302-docs-refreshfrom
dylanmyers/bp-1324-output-delivery-count

Conversation

@Dylan-M

@Dylan-M Dylan-M commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Proposed Change

Make generator.count only count a record once the destination has it, such as: a successful socket write, a Splunk ACK, or an OTLP export with nothing rejected. A record that fails or is dropped after it was queued is replaced so that a run ends with exactly the count delivered. Delivery is at least once, so a destination can receive duplicates. docs/finite-generation.md lists the delivery point for each output.

Breaking changes:

  • The end-of-run log field records_emitted is now records_delivered.
  • Validation rejects output.otlp-grpc.batchTimeout: 0 and an output.hec.batchTimeout of 0 or less. With HEC ACKs on, it also rejects an ackPollInterval or ackTimeout of 0 or less. A prometheus-remote-write batchTimeout: 0 is rejected when a count is set.
  • otlp-grpc metric and trace exports now use requestTimeout, as logs do, instead of batchTimeout. Stop no longer cancels an export in flight.

stdout now buffers metrics and spans along with logs. Filegen replay retries a failed line up to three times after the rest of the pass, so retried lines arrive out of order. After the last retry it drops the line with an error log. Permit.Forgo then lowers that cycle's count, so the run still ends.

Checklist
  • Changes are tested
  • CI has passed

@Dylan-M
Dylan-M requested review from a team as code owners October 30, 2026 14:29
@Dylan-M
Dylan-M added this pull request to stack #347 October 30, 2026 14:29
@Dylan-M Dylan-M changed the title dylanmyers/bp 1324 output delivery count feat: count only records the destination received (BP-1324) Oct 30, 2026
@Caleb-Hurshman Caleb-Hurshman self-assigned this Oct 30, 2026
@Dylan-M
Dylan-M force-pushed the dylanmyers/bp-1324-output-delivery-count branch from 5f8b086 to 0e4d5de Compare October 30, 2026 16:20
@Dylan-M
Dylan-M added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit 9de5320 Nov 2, 2026
19 of 31 checks passed
@Dylan-M
Dylan-M deleted the dylanmyers/bp-1324-output-delivery-count branch November 2, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants