Repository navigation
feat(sdk): v0.2.0 — feature parity with Arc server + hardening - #2
Conversation
Bring the Python SDK up to date with the Arc server and fix several confirmed bugs. Sync and async paths are both covered throughout. New capabilities: - query_msgpack(): experimental columnar MessagePack query response with native timestamp decoding; raises ArcNotSupportedError on 501 (server built without the duckdb_arrow tag). - Bulk import client (client.import_): CSV, Parquet, line-protocol file, and TLE; accepts paths, file-like objects, or bytes. - TLE streaming write (write.write_tle) + tle_stats. - Databases management (client.databases): list/get/list_measurements/ create/delete. - Full enterprise surface (sync + async): rbac, governance, audit, queries (query management), cluster, compaction, tiering, backup, mqtt. - zstd write compression (now the default; gzip/none still supported) via a Compression enum; no Content-Encoding header (server auto-detects from magic bytes). - Automatic retries with exponential backoff + jitter for connection errors, 429 (honoring Retry-After), and 503. - New exceptions: ArcNotSupportedError (501), ArcLicenseError (403 for a missing enterprise-license feature). Fixes: - Polars datetime->columnar used a fragile private attribute that raised at runtime; now casts via the public polars API and normalizes time_unit. - Pandas datetime timestamp scaling respected only nanoseconds; now honors the column resolution, including tz-naive numpy datetime64 columns whose dtype has no .unit (read via numpy.datetime_data) — these previously wrote timestamps ~1000x too small. - Buffered writer now runs a background flush timer (thread for sync, task for async) so idle buffers still flush on flush_interval. - show_tables validates the database identifier before SQL interpolation. - User-Agent derived from the package version (single source of truth); removed the dead dataframe package stub. Testing: - First real HTTP round-trip coverage via pytest-httpx (headers, error mapping, on-the-wire compression, retries, msgpack decode, imports, databases, enterprise wrappers) for both sync and async. - Opt-in live integration tests under tests/integration (gated on ARC_TEST_URL). 167 unit tests passing.
|
Parity and merge analysis: https://github.com/Basekick-Labs/ops/issues/40. Source inventory finds 120 SDK operations here versus 182 explicit server routes on pinned main; 56 remaining public/operational routes need scope decisions after excluding internal protocols and equivalent aliases. Method presence is not a contract pass. PR #3 is stacked on this branch; local composition with #5 conflicts in Fresh exact-module fixtures preserve merged PR #1's sparse-column padding in sync/async across all heads, and fail on the pre-fix control. Add that oracle to the permanent regression suite. Binary query row-cap handling and nanosecond precision need explicit acceptance coverage; no full SDK/server runtime pass is claimed. |
Summary
Brings the Python SDK (
arc-tsdb-client) up to date with the Arc server after a long gap, fixes several confirmed bugs, and adds the first real HTTP-layer test coverage. Ships as v0.2.0. Sync and async paths are both covered throughout.New capabilities
query_msgpack()— experimental columnar MessagePack query response with native timestamp decoding; returns the sameQueryResponseshape asquery()(plus per-column Arrowtypes). RaisesArcNotSupportedErroron 501 (server built without theduckdb_arrowtag).client.import_) — CSV, Parquet, line-protocol file, and TLE. Accepts paths, file-like objects, or bytes.write.write_tle) +tle_stats.client.databases) — list / get / list_measurements / create / delete.rbac,governance,audit,queries(query management),cluster,compaction,tiering,backup,mqtt.Compressionenum. NoContent-Encodingheader (server auto-detects from magic bytes).Retry-After), and 503.ArcNotSupportedError(501),ArcLicenseError(403 for a missing enterprise-license feature).Fixes
time_unit.datetime64columns whose dtype has no.unit(read vianumpy.datetime_data). These previously wrote timestamps ~1000× too small (landing in ~1970). Caught by adversarial review after an initial incomplete fix.flush_interval.show_tablesvalidates the database identifier before SQL interpolation.dataframepackage stub.Testing
pytest-httpx: header assembly, error mapping (incl. 501/license), on-the-wire gzip/zstd magic bytes, retry/backoff, msgpack columnar decode, imports, databases, and enterprise wrappers — sync and async.tests/integration(gated onARC_TEST_URL).Review
Four independent adversarial reviewers ran over the diff (async/sync twin parity, wire-protocol fidelity vs the Go server, concurrency/retry correctness, algorithmic edge-cases). Three came back clean; the algorithmic pass surfaced the tz-naive timestamp corruption bug above, which is now fixed and covered by a
4 resolutions × {tz-naive, tz-aware}regression matrix.🤖 Generated with Claude Code