Repository navigation
feat(sdk): 0.2.0 parity with Arc main + opt-out installation telemetry - #29
Merged
Merged
Conversation
…emetry Requests carry Arc-Client-Installation-Id (random UUID persisted under the user config dir, env override for containers, in-memory fallback when the dir is unwritable) and a User-Agent naming the Python version and platform. Arc relays distinct ids in its own opt-out telemetry, as it does for arcli; the SDK never contacts anything but the configured Arc server. Opt out with DO_NOT_TRACK=1, ARC_CLIENT_SEND_INSTALLATION_ID=0 or send_installation_id=False. Tests isolate the config dir so no test mints an id in the real home directory.
Add trigger(measurement=) (Arc 2026.09.2+), history(cycle_id=), cycles(), cycle(id), too_wide() and clear_too_wide() (Arc 2027.01.1+) to the sync and async compaction clients. Correct the module docstring: Arc made every compaction route admin-only, reads included.
Wrap /api/v1/reconciliation as client.reconciliation (sync and async). trigger() defaults to a dry run; act mode requires dry_run=False plus confirm=True and sends Arc's dry_run=false&act=true pair. Runs are returned as ReconciliationRun models; status() stays a raw dict.
Add AuthClient.cache_stats() and invalidate_cache() (sync and async) for the admin-only /api/v1/auth/cache routes. Stats are returned as a raw dict; invalidation loses no data, so it takes no confirm flag.
create(databases=) scopes a backup, list(include_foreign=) includes backups another instance wrote, and restore(mode=) selects merge or replace. All three need Arc 2027.01.1+ and are omitted unless set, so older servers behave as before. Restore still requires confirm=True; mode is validated client-side and left to the server's effective default when omitted.
delete.delete(allow_unreachable=True) sends the body field and databases.delete(allow_unreachable=True) sends ?allow_unreachable=true (Arc 2027.01.1+). Both are omitted by default. Docstrings spell out the relaxed cluster safety check: the delete runs and rows in unreachable files or on unflushed nodes are left in place.
#15) Add client.diagnostics (sync and async): - readiness(write=...) for /ready and /ready/write; a 503 probe answer is returned as ReadinessStatus (not_ready / not_write_target + reason) - JSON metrics, timeseries, and Prometheus text (public routes) - admin logs with client-side range checks for values Arc would silently replace with defaults - admin memstats / duckdb-memory, and free_os_memory gated by confirm=True because it forces a stop-the-world GC
ExecuteCQResponse gains optional watermark_status (set, moved or pending), which Arc 2027.01.1+ returns from a manual run once the cluster has adopted continuous queries. None on older servers.
… and policies Add delete_file, demote_writer, assign_compactor and converge_voters (Arc 2026.09.2+) and stream_claims, release_stream_claim, abort_compaction_job, policies and adopt_policies (Arc 2027.01.1+) to the sync and async cluster clients. Destructive controls (delete_file, demote_writer, converge_voters with dry_run=False, release_stream_claim with abandon=True, abort_compaction_job, adopt_policies) require confirm=True. converge_voters plans by default. adopt_policies raises when a standalone server answers 200 with success false.
) Add client.edge_sync (sync and async) covering every operator route: - hub: /api/v1/sync-spokes (list/get/register/rotate/enable/disable/ delete) and /api/v1/bundle-import (import, history) - spoke: /api/v1/spoke-sync (status, ledger, run, requeue, dismiss, export, revert, ack) rotate_spoke_secret, delete_spoke and dismiss_failed require confirm=True. Spokes are returned as Spoke/SpokeSecret models; other responses stay raw dicts. The HMAC spoke-to-hub transport routes (/api/v1/sync/file, /api/v1/sync/reconcile) are intentionally not wrapped.
Query POSTs only read, but _request retried status codes for GET/HEAD/OPTIONS only, so Arc's replication catch-up gate (503 + Retry-After) and the 27.01 cold-tier/vanished-partition 503s failed queries outright. Requests can now be marked idempotent=True (status retries like a GET; transport retries keep the mutation rule) or carry a retry_if predicate. A 503 carrying X-Arc-Ingest-Fenced or X-Arc-Ingest-Refused now raises ArcIngestRefusedError(reason, retry_safe); is_retryable_write_refusal accepts every reason except wal_not_durable, which may have applied part of the request. Adds ArcPartialWriteError and status_code on ArcQueryError and ArcIngestionError.
- Query executors no longer rewrap typed errors: authentication, permission,
scoped-access, license, rate-limit and not-supported errors surface as
themselves. Generic HTTP and transport failures stay ArcQueryError (with
status_code and __cause__) so except ArcQueryError keeps working.
- All four query POSTs are sent idempotent=True.
- profile=True on query/query_msgpack sends x-arc-profile and returns
QueryResponse.profile (QueryProfile).
- query_arrow/query_pandas/query_polars accept dictionary= and
compression=('zstd'|'lz4').
- X-Arc-Query-ID is exposed as QueryResponse.query_id and arc.query_id in
Arrow schema metadata.
- query_measurement() wraps GET /api/v1/query/:measurement.
- msgpack: documented as stable; an incomplete body (Arc's only signal for a
stream that failed after the 200) raises ArcQueryError.
- write_line_protocol/write_point take precision= (ns|us|ms|s) sent as ?precision=; write_point sends the timestamp unconverted when it is set. - tag_columns is no longer a no-op: write_columnar/write_dataframe and the buffered writers send the payload's tag_keys (Arc 2027.01). - Writes resend the 503 refusals Arc documents as 'nothing was written'; wal_not_durable and every other failure are never resent. - Arc's 400 'partial write: N points dropped' raises ArcPartialWriteError (dropped, others_written). Buffered writers discard that batch, keep flushing other measurements, and report the error once. - Auth, permission and rate-limit errors on writes surface as themselves.
create_token(permissions=[]) dropped the empty list, so Arc created a read,write token instead of an RBAC-only one. Optional token fields are now tested with 'is not None' (sync and async). The auth client also stops rewrapping rate-limit, server and connection errors as ArcAuthenticationError, which made them look like a bad token.
- HealthResponse models what Arc sends (time, uptime, uptime_sec, storage, license); version stays as an optional, always-None field. - readiness()/write_readiness() return ReadyResponse with status, reason and tier for /ready and /ready/write; ready() stays a bool and no longer retries a 503 'not ready' with backoff. - databases.get_measurement_schema() and rebuild_measurement_schema() cover the stable-schema registry routes (#16).
The Arrow trailer adapter walks httpcore private attributes that are identical in 1.0.0, 1.0.5 and 1.0.9; 1.0.9 is the floor because it is the first release that accepts h11 0.16. An exact pin in a library blocks any future 1.0.x security fix for every downstream app. If a release moves the attributes, capture is skipped and trailer-announcing Arrow responses fail closed with ArcQueryError.
…0-parity # Conflicts: # src/arc_client/models/__init__.py
…eadiness API Arc checks its ingest fence once per request in AdmitRequest and again per measurement in checkIngestGate (internal/ingest/arrow_writer.go), so a fenced or refused 503 to a multi-measurement line-protocol or row-format body can arrive after earlier measurements were buffered. Resending it duplicates rows. Such refusals are now raised once with retry_safe=False. Also drop DiagnosticsClient.readiness/ReadinessStatus, which duplicated client.readiness()/write_readiness(); the readiness tests now exercise the client methods, which probe once instead of retrying a 503.
- Buffered writers no longer lose partial-write errors: they are queued, reported before any terminal failure, and several are folded into one ArcPartialWriteError (dropped summed). A second background partial no longer overwrites the first, and close()/flush() report them even when another measurement fails. - A retry-safe refusal (fenced, wal_disk) keeps the batch and leaves the buffered writer usable; auto-flushes inside write() leave it for the next flush. Only wal_not_durable and other uncertain failures are terminal. - Buffered write() validates record tags before buffering, so a bad tag name raises at once instead of poisoning the writer. - MessagePack refusals are resent again for multi-measurement records: Arc checks the fence once per msgpack batch (ArrowBuffer.Write). Only multi-measurement line protocol stays gated, and that check now runs only when Arc actually refuses a write. - Query POSTs retry 503 only. Arc's query 429 is governance (rate limit or quota) without Retry-After and does not clear within the backoff. - The installation id never fails a request: an unresolvable home directory falls back to an in-memory id. It is resolved once per HTTP client, so no env lookups or first-run file I/O run per request or on an event loop.
This was referenced Nov 2, 2026
This was referenced Nov 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the SDK to parity with Arc main, including the planned 2027.01.1 features, and adds opt-out installation telemetry relayed through Arc. 0.2.0 ships with Arc 2027.01.1.
Closes #11, #12, #13, #14, #15, #16, #17.
Fixes for released Arc
query,query_arrow,query_msgpack,estimate) retry 503 withRetry-After. This covers replication catch-up, a vanished partition, and an unavailable cold tier. A governance 429 is raised at once.create_token(permissions=[])andupdate_token(permissions=[])used to drop the empty list, so Arc silently created a read/write token. They now send it.X-Arc-Ingest-FencedorX-Arc-Ingest-RefusedraisesArcIngestRefusedErrorand is resent automatically. Two cases are never resent:wal_not_durable, which may have partly applied.httpcoreis bounded>=1.0.9,<1.1instead of pinned exactly.Parity
ArcPartialWriteErrorfor Arc 2027.01's400 "partial write". It is never retried, and buffered writers discard that batch and report it once.precision=on line-protocol writes.tag_keys(thetag_columns=parameter now does something).profile=True, Arrowdictionary=/compression=, andquery_idforqueries.kill().query_measurement(), plus the measurement schema read and rebuild.HealthResponsenow matches Arc's/health.readiness()/write_readiness()return the reason, andready()is a single probe.databases,include_foreignand restoremode(Complete backup filter and restore-mode parameters #11).allow_unreachableon deletes, and continuous-querywatermark_status.confirm=True. Parameters that need Arc 2026.09.2+ or 2027.01.1+ say so in their docstrings.Telemetry
Requests send
Arc-Client-Installation-Id, a random UUID persisted under the user config directory, plus a User-Agent that names the Python version and platform. Arc relays the distinct IDs in its own opt-out telemetry, as it already does for arcli; the SDK never contacts anything but the configured Arc. Details:DO_NOT_TRACK=1,ARC_CLIENT_SEND_INSTALLATION_ID=0, orsend_installation_id=False.ARC_CLIENT_INSTALLATION_IDto a fixed UUID.feat/telemetry-sdk-clients) and telemetry.basekick.net (feat/sdk-installations); those PRs follow.Verification
pytest tests/unit: 1050 passed on 3.12. On 3.9 with the pandas/polars extras: 1061 passed.ruff check,ruff format --checkandmypyare clean, anduv lock --checkpasses.1943fe87); commit messages and docstrings cite the files.75512fb. The other 2, refactors of the duplicated sync/async error mapping and of the retry-policy plumbing, are deferred.Notes for reviewers
ready()no longer retries a 503. A query 429 is not retried. MessagePack queries are stable, and a stream that fails after the 200 raises instead of returning partial rows.