[SDK] Consolidate deployment log reading into fetch_logs

[V2arK]

Problem

Deployment log reading was spread over three stateful readers (get_deployment_logs, get_deployment_logs_range, deployment_log_session), and none of them could stream one pod's logs lazily with bounded memory. Tailing a running deployment meant re-creating a reader over overlapping windows and keeping a seen dict in user code to drop the server's late-arrival re-deliveries — bookkeeping the SDK should absorb.

Change

• fetch_logs(deployment_id, revision_number, pod, start_time=None, end_time=None, chunk_size=10) is the whole surface. pod is required (discover names with get_deployment_pods()). start_time defaults to the current time and is resolved when fetch_logs is called rather than at the first next(), so lines logged while the generator sits unstarted are not skipped.

• chunk_size is the number of lines requested per round trip (max_lines on the wire), so it inherits the server's 1..5000 ceiling and is validated eagerly at the call. Each server page is yielded as one chunk: usually up to chunk_size lines, smaller when lines below start_time or already-delivered re-deliveries are filtered out of the page, larger when one millisecond holds more than chunk_size lines — the server never splits a millisecond across pages. A bulk read of history wants a large chunk_size; the log read path is rate-limited upstream, and a small chunk_size over a large window multiplies requests.

• With end_time set the iterator terminates once the window is delivered or the store has no more lines to give, whichever comes first — an end_time in the future does not keep it polling until then. Without end_time the iterator never terminates: once caught up it yields an empty chunk each time nothing new is stored yet, and the caller decides when to sleep or break. A page filtered away entirely yields nothing at all rather than that empty chunk, which means only that the stream is caught up.

  for chunk in cclient.fetch_logs(dep, rev, pod):
      if not chunk:
          time.sleep(2)
          continue
      ...

• The dedup anchor lives inside the generator for its whole lifetime and is trimmed by time (LOG_DEDUP_RETENTION_MS), which is what bounds its memory. Every stored line is delivered at most once and callers need no cross-call dedup.

• A page the store answers as busy (HTTP 503 — the read path is rate limited upstream on a bucket shared by every caller) is retried inside the loop, where the anchor is still in scope, with exponential backoff and jitter. There is no server-issued cursor, so a page request is a pure function of its anchor and re-issuing it can neither duplicate nor skip lines. Only 503 retries, and only here: the single-page readers lose nothing when a page fails and their callers can decide for themselves whether to wait. Nothing on this path sends Retry-After today — neither the API's HTTPException nor the ingress limiter — so the SDK does not read one.

• If a page request fails for any other reason, or the retries run out, the iterator raises and, like any generator, cannot be resumed — but every chunk already yielded is complete and none is left half-built. The docstring gives the resume recipe: re-anchor a new fetch_logs on the last delivered event's timestamp, which re-delivers only the lines sharing that millisecond.

• Lines inside a chunk are always ascending by (timestamp, id). The server orders a page by nanosecond timestamp only, never by the id's hash suffix, so lines sharing one nanosecond can arrive in either id order; the SDK re-inserts them.

• get_deployment_logs, get_deployment_logs_range, deployment_log_session and the DeploymentLogSession class carry typing_extensions.deprecated (PEP 702, the 3.10-compatible backport of warnings.deprecated) and keep working unchanged. typing-extensions is now a declared dependency rather than a transitive one. SDK-internal paging goes through the private _fetch_log_page, so it does not trip the warning. The pytest.ini deprecation filter is gone; the deprecated readers' tests assert the warning with pytest.warns(DeprecationWarning).

• get_deployment_pods() is unchanged.

• README is one short paragraph plus one example each for a window read and a tail. examples/sdk/get_deployment_logs.py shows the same two shapes over a single generator, with no seen dict and no overlap window.

Test plan

./.venv/bin/python -m pytest tests/ -q                                    # 119 passed
python -m pylint --rcfile ./scripts/pylintrc ./centml ./tests             # 10.00/10
python -m black --skip-string-normalization --skip-magic-trailing-comma \
    --line-length 120 --check ./centml ./tests                            # 24 files unchanged
mypy centml tests                                                         # no issues in 24 source files

End-to-end against the local v2 environment at d4bd070: the real API behind the k3d
ingress, real Loki behind it, synthetic lines pushed under the labels deployment 1729
revision 9 already uses.

Check	Result
Pod discovery	get_deployment_pods() returned 10 pods, terminated ones included
Event shape	DeploymentLogEvent(id='<nanoseconds>-<hash>', timestamp=<epoch ms>, message=..., pod=...); the wire carries no pod, the SDK attaches it
Bounded read	250 lines delivered exactly once at chunk_size 1 / 10 / 100 / 5000 — 314 / 34 / 5 / 2 requests, 0 duplicates, iterator terminates on its own
Wire page size	at chunk_size=7 every request sent max_lines=7; 46 requests over the same window
Full-page shape	at chunk_size=10, chunks of 9 with a final 7: the server withholds each full page's trailing millisecond and the next page re-covers it
Single-millisecond burst	12 lines sharing one millisecond arrived whole in one chunk at chunk_size=5 (chunk lengths 4, 12, 4)
Window bounds	start_time and end_time both inclusive; an inner window trims both sides
Laziness	0 requests after constructing the generator, 1 after the first next()
Open-ended tail	3 seed lines, then 3 empty chunks while idle without returning, then all 5 lines pushed mid-stream through the same generator; 0 duplicates
start_time pinned at the call	a generator left unstarted for 3 s still delivered the 4 lines written during them
Eager validation	chunk_size 0 / 5001, negative bounds and start_time > end_time each raised ValueError at the call, before any request
Unknown pod	not an error — fetch_logs(pod='no-such-pod') yielded nothing
Unknown revision	NotFoundException 404, {"detail":"Revision number 99999 not found for deployment 1729"}
Upstream rate limit	an unpaced chunk_size=1 read of 300 lines: 365 requests, 64 backoffs, 300/300 delivered, 0 duplicates, 42 s. The same shape died with ServiceException(503) at request ~32 before the retry
Deprecated readers	still return data; get_deployment_logs() emits one DeprecationWarning, deployment_log_session() emits two — one for the factory, one for the session class it hands back

[anandj91]

Thanks for the thorough work, but this is much more than what I asked for. What I want is one simple function: fetch a pod's log lines within a time window and yield them in caller-sized chunks, so the same call serves a one-off read of the past and tailing a running deployment. Roughly two thirds of the new code (direction flag, multi-pod merge with backpressure, the migration guides) implements things nobody requested, and three of my earlier comments were not followed (start_time default, per-pod scope, deprecation decorator). Please cut it down per the inline comments. The forward single-pod walker you already have is essentially the whole feature.

[anandj91]

This is the main design issue. Because the generator stops on the first empty page, the README and example have callers re-create it every two seconds with a 30 second overlap and keep a seen dict to dedupe by id. That is exactly the re-delivery bookkeeping the PR description says callers should never have to think about, moved into user code.

Keep the anchor inside the generator across pages (you already do this with held), and when end_time is None and a page comes back empty, yield an empty chunk instead of returning. Then one generator serves both shapes: with end_time set it terminates after passing it; without, the caller decides when to sleep or break:

for chunk in cclient.fetch_logs(dep, rev, pod):
    if not chunk:
        time.sleep(2)
        continue
    ...

[V2arK]

Done in 9ccc81a. With end_time unset the generator no longer returns on an empty page: it flushes any partial chunk, then yields an empty chunk. The dedup anchor now lives for the whole generator, so the re-delivery span is filtered inside the SDK and cross-call dedup is gone from user code.