create a the log hooks for cucascade - #175
Open
aminaramoon wants to merge 2 commits into
Open
Conversation
aminaramoon
marked this pull request as draft
July 30, 2026 23:15
aminaramoon
marked this pull request as ready for review
July 31, 2026 06:01
Contributor
Author
|
/ok to test d269f96 |
ran-yuan-rui
reviewed
Aug 4, 2026
| # topology-only build, which does not produce cucascade_objects. | ||
| if(TARGET cucascade_objects) | ||
| target_sources(cucascade_objects | ||
| PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/logging.cpp) |
Contributor
There was a problem hiding this comment.
seems like we only build logging.cpp but no stderr_sink.cpp?
ran-yuan-rui
reviewed
Aug 4, 2026
| namespace cucascade::log { | ||
| inline namespace v1 { | ||
|
|
||
| void stderr_sink(void*, record const& rec) noexcept |
Contributor
There was a problem hiding this comment.
this function has the same name within one in logging.cpp
rapids-bot Bot
pushed a commit
that referenced
this pull request
Aug 26, 2026
The existing footer-probe path starts with a blocking suffix GET on a short-lived easy handle. The process-wide CURLSH shares DNS and TLS-session state, but not its connection cache, so repeated probes each create a new TCP/TLS connection. This adds `rest_ioctx::resolve_footer_objects(paths, on_result, stop)`. A per-call curl multi keeps at most `footer_resolve_max_inflight` transfers active and reuses connections while the call is running. Results are delivered one input at a time, serially on the caller's thread, without waiting for the whole batch. This PR adds the cuCascade mechanism only; Sirius scan-preparation wiring will follow separately once the sirius-side io parity settles. Each entry follows `open_io_object(path, parquet_footer_probe)` behavior. A verified 206 returns the suffix window. Responses that cannot establish a valid suffix window fall back to HEAD. A zero-byte probe window uses HEAD directly. Retries re-authorize each attempt and preserve ETag and perf-counter attribution. Each input occurrence receives exactly one callback, including duplicate paths. Per-entry failures do not cancel siblings. Cancellation reports `operation_canceled` for entries not yet delivered. If a callback throws, remaining entries are canceled before the first exception is rethrown. Invalid submissions fail before any callback or network request. The returned `io_object` carries the path, object size, and validation tag, but not the footer bytes. Footer bytes are returned in a separate payload backed by an ioctx-wide `footer_resolve_stash_budget` lease. The reservation is released when the payload is freed. Concurrent calls are FIFO-serialized per ioctx, and `footer_resolve_max_inflight = 0` disables the API. Supporting changes: - `exec::admission_control` gains `try_acquire()`, `reserved()`, and `peak_reserved()`. - `rest_perf_snapshot` reports current and peak footer-payload reservations. - The loopback server gains keep-alive, per-key counters and response scripts, and a deterministic GET barrier. The PR also moves `mock_authorizer.hpp` from `include/cucascade/io/rest/` to `test/io/rest/`. It is a test-only fixture that was being installed and included in public API documentation. The move preserves its contents and updates its three in-tree includes; the move itself requires no CMake change. Fresh installs no longer contain the header. ## Validation The 20 footer-resolve cases, all failing before the mechanism landed, cover: - suffix/HEAD parity, retries, ETags, and perf counters; - streamed completion, duplicate inputs, cancellation, and callback failures; - per-entry error isolation; - memory-budget accounting, FIFO admission, and payload lifetime; - connection reuse under a bounded in-flight limit. `cucascade_io_tests`: 135/135 green, 1624 assertions. Sirius rebuilt successfully with cuCascade `c0a2607` on `integrate_cucs_io` at `8e7ce16b`. `make s3-test` passed 90 of 92 cases; the two remaining failures are the existing retry-log observability cases tracked in #175. The later test-only header move does not affect that result. Authors: - Yu (https://github.com/ran-yuan-rui) Approvers: - Amin Aramoon (https://github.com/aminaramoon) URL: #188
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.