This is the core Go component of the DataSketches library. It contains some of the sketching algorithms and can be accessed directly from user applications.
This project is currently under development. Breaking changes may occur before a stable release.
Note that we have parallel core library components for C++, Java, Python, and Rust implementations of many of the same sketch algorithms:
Please visit the main DataSketches website for more information.
If you are interested in making contributions to this site, please see our Community page for information on how to contact us.
| Type | Implementation | Status |
|---|---|---|
| Cardinality | ||
| CpcSketch | ✅ | |
| HllSketch | ✅ | |
| ThetaSketch | ✅ | |
| TupleSketch |
✅ | |
| Quantiles | ||
| CormodeDoublesSketch | ❌ | |
| CormodeItemsSketch | ❌ | |
| KllDoublesSketch | ✅ | |
| KllFloatsSketch | ✅ | |
| KllSketch | ✅ | |
| ReqFloatsSketch | ||
| TDigestDouble | ✅ | |
| TDigestFloat | ❌ | |
| Frequencies | ️ | |
| FreqLongsSketch | ✅ | |
| FreqItemsSketch | ✅ | |
| CountMinSketch | ✅ | |
| Sampling | ||
| ReservoirLongsSketch | ✅ | |
| ReserviorItemsSketch | ✅ | |
| VarOptItemsSketch | ✅ | |
| EbppsSketch | ❌ | |
| Membership | ||
| BloomFilter | ✅ | |
| Density | ||
| DensitySketch | ❌ |
| Type | Interface Name | Status |
|---|---|---|
| Cardinality/FM85 | UniqueCountMap | ❌ |
| Cardinality/Tuple | ||
| FdtSketch | ❌ | |
| ArrayOfDoublesSketch | ✅ | |
| DoubleSketch | ✅ | |
| IntegerSketch | ✅ | |
| ArrayOfStringsSketch | ✅ | |
| EngagementTest3 | ❌ |
✅ = Released in v0.2.0
❌ = Not yet implemented
🚧 = In progress
This code requires Go 1.24
The cross-language compatibility tests read serialization snapshots that are not committed to this repository. Download the C++ and Java snapshots from datasketches-tck once, then run the tests:
make download-snapshots
make test
make download-snapshots fetches the TCK revision pinned in
scripts/download_serialization_test_data.sh into serialization_test_data/.
make test generates the Go snapshots (make generate-go-snapshots) before running
go test ./.... If a snapshot is missing, the test that needs it fails and says which
command to run.
A Dockerfile is also provided with the necessary env to build and test the project.
./build/Dockerfile
./build/run-docker-test.sh
Lint runs in CI via the Lint workflow, which uses
golangci-lint v2.13.2 with the configuration in
.golangci.yml.
Install that same version locally. The upstream project recommends the binary
install and discourages go install and tool directives, because a source build
compiles against your local Go version and can report different findings than CI:
curl -sSfL https://golangci-lint.run/install.sh | sh -s -- -b $(go env GOPATH)/bin v2.13.2
Prebuilt binaries are also available on the releases page, and package managers such as Homebrew work as well. Pin v2.13.2 either way, so local results match CI.
Then:
make lint # report findings
make format # gofmt, goimports, and golangci-lint --fix