Feat/go table model api - #949
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
This PR introduces a table-model-only Go API backed by the public C ABI, adding schema-bound writers, unified querying with composable options, and Arrow batch interoperability while removing tree-model Go symbols/examples.
Changes:
- Add schema-bound
NewWriter(path, schema, ...), targetlessTablet, and Arrow batch write support. - Replace multiple query entry points with
Reader.Query(...)+QueryOptions (time range, tag filters, pagination, batch mode). - Add table schema lookup/enumeration, one-based
ResultSetcolumn indexing, BLOB support, and strengthen C ABI validation/ownership.
Reviewed changes
Copilot reviewed 31 out of 32 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| pom.xml | Excludes go/go.sum from Maven-related checks. |
| go/tsfile/writer.go | Schema-bound writer; adds Tablet/Arrow batch write paths and Arrow schema validation. |
| go/tsfile/tag_filter.go | Introduces composable TAG filter AST + validation. |
| go/tsfile/tablet.go | Makes Tablets targetless; adds BLOB setter; tracks row/timestamp presence. |
| go/tsfile/table_api_contract_test.go | Adds contract tests for schema/queries/tag filters/Arrow schema validation. |
| go/tsfile/schema.go | Removes tree-model schema types; adds table schema normalization/validation/copy. |
| go/tsfile/result_set.go | Switches to one-based columns; adds batch-mode Arrow reading + BLOB getter. |
| go/tsfile/reader.go | Adds unified Query API with composable options; adds schema lookup APIs. |
| go/tsfile/integration_test.go | Reworks integration coverage for new table-only API + Arrow round trips. |
| go/tsfile/errors.go | Removes tree-model sentinels; adds ErrWrongResultMode. |
| go/tsfile/doc.go | Updates package docs to table-only API and indexing rules. |
| go/tsfile/constants_test.go | Re-pins constants to new table-model-focused ABI surface. |
| go/tsfile/constants.go | Removes tree-model constants/types (encoding/compression/etc.); keeps table types. |
| go/tsfile/cgo_bridge_test.go | Updates bridge tests for new tablet/writer constructors. |
| go/tsfile/cgo_bridge.go | Refactors cgo bridge to table writer + schema binding; adds tag-filter lifetime mgmt; adds Arrow/BLOB plumbing. |
| go/tsfile/api_test.go | Updates public API tests for new NewWriter/NewTablet contracts. |
| go/go.sum | Adds Go module checksums for Arrow and dependencies. |
| go/go.mod | Adds Arrow dependency set; updates Go version directive format. |
| go/examples/tree_read_write/main.go | Removes tree-model example. |
| go/examples/table_read_write/main_test.go | Adds runnable example test harness for new table example. |
| go/examples/table_read_write/main.go | Updates example to write with Tablet + Arrow batch, then query rows. |
| go/README.md | Updates README to table-only API semantics and Arrow ownership. |
| cpp/test/file/utf8_path_test.cc | Updates expectations: writer/file creation truncates existing UTF-8 paths. |
| cpp/test/cwrapper/cwrapper_public_writer_test.cc | Adds C ABI tests for table writer schema validation and null args. |
| cpp/test/cwrapper/c_release_test.cc | Updates C release tests to reflect truncate-on-create behavior. |
| cpp/src/writer/tsfile_table_writer.h | Adds init error propagation + schema/table introspection methods. |
| cpp/src/writer/tsfile_table_writer.cc | Allows targetless tablet binding per write and exposes bound schema. |
| cpp/src/reader/tsfile_reader.h | Adds table query overload with offset/limit. |
| cpp/src/reader/tsfile_reader.cc | Implements pagination validation and safe schema lookup. |
| cpp/src/cwrapper/tsfile_cwrapper.h | Adds flush, Arrow write, unified table query, binary getter, checked schema APIs. |
| cpp/src/cwrapper/tsfile_cwrapper.cc | Strengthens validation/ownership; implements new C ABI entry points. |
| cpp/src/cwrapper/arrow_c.cc | Strengthens Arrow C Data Interface validation and schema matching. |
Suppressed comments (1)
go/tsfile/writer.go:1
cdata.ExportArrowRecordBatchtypically returns an error in Arrow Go; ignoring it can either fail compilation (if the signature returnserror) or allow an uninitialized/invalid exported C struct to be passed into the C ABI. Capture and handle the export error and return a wrappedErrInvalidArgument(or similar) before calling intowriteArrow.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| var nativeArray cdata.CArrowArray | ||
| var nativeSchema cdata.CArrowSchema | ||
| defer cdata.ReleaseCArrowArray(&nativeArray) | ||
| defer cdata.ReleaseCArrowSchema(&nativeSchema) |
| } | ||
| return nil, err | ||
| } | ||
| record, err := cdata.ImportCRecordBatch(&nativeArray, &nativeSchema) |
| h := C.tablet_new(&names[0], &types[0], C.uint32_t(len(dataTypes)), C.uint32_t(maxRows)) | ||
| if h == nil { | ||
| return nil, newError("new tablet", C.RET_OOM) | ||
| } |
| if nativeError, ok := err.(*Error); ok && nativeError.Code == 21 { | ||
| return nil, io.EOF | ||
| } |
| return copy, nil | ||
| } | ||
|
|
||
| func normalizeIdentifier(value string) string { return strings.ToLower(value) } |
| module github.com/apache/tsfile/go | ||
|
|
||
| go 1.22 | ||
| go 1.22.0 |
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Pull request overview
Copilot reviewed 31 out of 32 changed files in this pull request and generated 5 comments.
Suppressed comments (1)
go/tsfile/writer.go:1
array.TableReadercan stop iteration due to an underlying error as well as normal exhaustion. The current code ignoresreader.Err()after the loop, which can silently drop failures. Checkreader.Err()afterfor reader.Next()and return it when non-nil.
| module github.com/apache/tsfile/go | ||
|
|
||
| go 1.22 | ||
| go 1.22.0 |
| normalizedColumns := make([]string, len(columns)) | ||
| for i, column := range columns { | ||
| normalizedColumns[i] = normalizeIdentifier(column) |
| if nativeError, ok := err.(*Error); ok && nativeError.Code == 21 { | ||
| return nil, io.EOF | ||
| } |
| if err := validateCString("validate table schema", "table name", schema.Table); err != nil { | ||
| return TableSchema{}, fmt.Errorf("%w: %v", ErrInvalidSchema, err) | ||
| } |
| return copy, nil | ||
| } | ||
|
|
||
| func normalizeIdentifier(value string) string { return strings.ToLower(value) } |
Summary
Add a table-model-only Go API backed by the public C ABI.
The Go writer now binds and retains a single
TableSchemaduring construction. Callers can write targetless Tablets or Arrow Record/Table batches, then query table data through one configurableReader.Queryentry point.Changes
NewWriter(path, schema, options...)with schema ownership and validation.NewTabletto accept only columns and row capacity.io.EOFhandling.Validation
go test ./...go test -race ./tsfilego vet ./..../mvnw -P with-cpp -DskipTests -Dxml-format.skip=true package