Skip to content

Prevent overdue timers from starving I/O polling - #922

Open
radkesvat wants to merge 1 commit into
ithewei:masterfrom
radkesvat:fix/overdue-timer-polling
Open

radkesvat wants to merge 1 commit into
ithewei:masterfrom
radkesvat:fix/overdue-timer-polling

Conversation

@radkesvat

Copy link
Copy Markdown
Contributor

Problem

An overdue timer currently causes hloop_process_events() to jump directly to timer processing, skipping the backend I/O poll:

if (blocktime_us < 0) goto process_timers;

If a repeating timer continually overruns its interval, the loop can take this branch on every subsequent iteration. Ready sockets and posted custom events can then remain undiscovered for as long as the overrun continues, even though every timer callback returns normally.

The relevant code is the overdue-timer branch in hloop_process_events(). The condition can sustain itself because repeating timers are rescheduled before their callbacks execute.

Example: a small timer overrun stalls network work

Consider a server with a repeating 1 ms timer whose callback performs 2 ms of finite work. These timings are illustrative.

  1. The timer expires. The loop advances its next deadline past the current timer-collection time, reinserts it into the timer heap, and schedules its callback.

  2. The callback runs for 2 ms and returns. Its next deadline is now already overdue. Meanwhile, a client sends data and another thread posts a custom event. The socket and the custom-event wake descriptor become readable.

  3. The next iteration skips polling. Because the timer is overdue, hloop_process_events() jumps to process_timers. The readable socket and wake descriptor are not discovered.

  4. The same sequence repeats. The timer is rescheduled and its callback overruns again. Timer callbacks keep executing, while the client’s data and the posted custom event remain unprocessed.

This can make an application appear partially alive: timer callbacks continue running, but readiness-driven network handling stops making progress. Clients may wait despite their data already being available, and work submitted from other threads may remain queued.

Native events posted through hloop_post_event() are affected because their queue is drained by the wake descriptor’s read callback. C++ EventLoop::queueInLoop() ultimately uses this posting path.

The failure occurs between finite callbacks: after a callback returns, the loop repeatedly misses the opportunity to discover other ready work.

Change

This change uses a zero poll timeout when the nearest timer is due or overdue, then continues through the existing I/O, clock-update, STOP-check and callback-dispatch path. Future-timer timeout rounding, the caller's timeout cap, timer repeat scheduling and callback priorities are preserved.

The new bounded regression uses real socket reads and posted events while forcing a repeating timer overdue for eight loop turns. It covers both timer heaps and remains active in Release builds. Static Linux CMake builds also check the actual backend timeout, exactly-due deadlines, future waits, the no-I/O path and STOP handling. The test is registered in both build systems and the Make test runner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant