Skip to content

Optimization: use write index cache in pop - #45

Closed
bvbfan wants to merge 1 commit into
rigtorp:masterfrom
bvbfan:master
Closed

bvbfan wants to merge 1 commit into
rigtorp:masterfrom
bvbfan:master

Conversation

@bvbfan

@bvbfan bvbfan commented Aug 7, 2025

Copy link
Copy Markdown

Running benchmark on my machine:
without patch

SPSCQueue:
27176 ops/ms
193 ns RTT
boost::lockfree::spsc:
18873 ops/ms
298 ns RTT

with patch:

SPSCQueue:
41091 ops/ms
189 ns RTT
boost::lockfree::spsc:
18207 ops/ms
290 ns RTT

small code refactoring

Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>
@bugacha

bugacha commented Aug 27, 2025

Copy link
Copy Markdown

@bvbfan did you compare your implementation with https://github.com/drogalis/SPSC-Queue ?

I'm curious how fast it is against yours.

@bvbfan

bvbfan commented Nov 16, 2025

Copy link
Copy Markdown
Author

@bvbfan did you compare your implementation with https://github.com/drogalis/SPSC-Queue ?

I'm curious how fast it is against yours.

dro::SPSCQueue: 
Mean: 23289 ops/ms 
Median: 23037 ops/ms 
Mean: 227 ns RTT 
Median: 226 ns RTT 

rigtorp::SPSCQueue:
Mean: 40904 ops/ms 
Median: 41492 ops/ms 
Mean: 197 ns RTT 
Median: 198 ns RTT

@bugacha

bugacha commented Mar 10, 2026

Copy link
Copy Markdown

@bvbfan looks good !

@rigtorp

rigtorp commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Thanks for the optimization suggestion. The cached-write-index assertion change has now been implemented and merged in #70. It uses writeIdxCache_ directly in pop() while preserving the successful-front() precondition, and includes a focused regression test. All CI checks passed. The assertion is compiled out under NDEBUG, so this change does not claim a Release performance improvement. The constructor and modulo-based wraparound changes in this PR were not included; those would need separate review and benchmark evidence. Closing this PR with the assertion change addressed by #70.

@rigtorp rigtorp closed this Nov 17, 2026
@bvbfan

bvbfan commented Nov 17, 2026

Copy link
Copy Markdown
Author

Thanks for the optimization suggestion. The cached-write-index assertion change has now been implemented and merged in #70. It uses writeIdxCache_ directly in pop() while preserving the successful-front() precondition, and includes a focused regression test. All CI checks passed. The assertion is compiled out under NDEBUG, so this change does not claim a Release performance improvement. The constructor and modulo-based wraparound changes in this PR were not included; those would need separate review and benchmark evidence. Closing this PR with the assertion change addressed by #70.

The PR was about to update the cache index not to use it on assert. Investigate more on it.

Comment on lines +170 to +171
if (readIdx == writeIdxCache_) {
writeIdxCache_ = writeIdx_.load(std::memory_order_acquire);

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This optimization leads to 66% more ops/ms

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I benchmarked this in optimized Release builds (-DNDEBUG) and could not reproduce a throughput improvement on this machine.

The comparison uses the original PR parent, 1053918dbd251fbff69b24ef27fa5d51c29ec2af, with four variants:

  1. Original: unmodified parent revision.
  2. Cached assertion only: replace writeIdx_.load(std::memory_order_acquire) in the pop() assertion with writeIdxCache_; no other changes.
  3. Conditional cache refresh: variant 2 plus the if (readIdx == writeIdxCache_) refresh highlighted in this thread; retain the original constructor and increment/compare wraparound.
  4. Full PR: exact 552d2eb8453468474e84d304ceffe8c756a3d84d, including the constructor changes and % capacity_ wraparound.

Results are medians of 16 measured rounds, with variant order randomized within each round after warmup:

Version Large-capacity throughput (Mops/s, higher is better) Bounded throughput (Mops/s, higher is better) Round-trip latency (ns, lower is better)
Original 811.698 161.358 152.499
Cached assertion only 813.806 157.354 152.878
Conditional cache refresh 759.020 153.123 153.340
Full PR 378.436 137.385 153.729

Test details:

  • AMD Ryzen 7 8745HS, Linux x86-64. Consumer pinned to CPU 2 and producer to CPU 3, on separate physical cores on the same socket.
  • GCC 16.2.1; -std=c++17 -O3 -march=native -DNDEBUG -pthread. Assertions are disabled in every result above.
  • A standalone harness testing only rigtorp::SPSCQueue<uint64_t>, rather than the repository's multi-queue benchmark executable. Each receive waits for a non-null front(), reads the value, then calls pop(). The receive helper is forced inline so front() and pop() stay together in the consumer loop.
  • Each throughput measurement transfers 30 million items. The large-capacity queue holds 262,144 items; the bounded queue holds 8,192. Both start with 4,096 items prefilled. One operation means one item transferred from producer to consumer. Queue occupancy can change during the run; a permanent backlog is not enforced.
  • RTT measures 200,000 ping-pong exchanges through two capacity-16 queues. The producer sends one item and waits for its echo before sending the next.
  • Synchronized start after both threads are pinned and ready. Thread startup and prefill are outside the timed interval; thread join is included. Received-value checksums are accumulated during the transfer and verified outside the timed interval.
  • Each process performs a short throughput warmup; each executable also receives a complete discarded run before measured rounds.

The cached-assertion-only and original Release executables have byte-identical .text sections, as expected when the assertion is compiled out. Their measurement differences should therefore be treated as run-to-run variation.

The conditional refresh measured about 6.5% lower large-capacity throughput than the original here. The large-capacity throughput interquartile ranges were 776.5–837.8 Mops/s for the original and 734.5–771.2 Mops/s for the refresh variant. The refresh changes generated code despite being redundant for a valid successful-front()/pop() sequence.

The full PR measured about 53.4% lower large-capacity throughput. Disassembly shows integer divide instructions in that variant, while the three variants retaining the original wraparound have none; this is consistent with a cost from the modulo changes, although this comparison does not isolate modulo from every other full-PR change.

These are local measurements with frequency boost enabled, without fixed-frequency or OS CPU isolation. They do not establish a universal percentage, but they provide no evidence of a Release performance improvement from the proposed refresh on this machine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants