Activity
- Tag v5.0.0 created by Caio on commit 9f4547b08d 2 months ago
-
Prepare for /v5 release by Caio 2 months ago
-
Fix docs link, remove silly badges by Caio 2 months ago
-
Breaking: add Clone method to RNG interface by Caio 2 months ago
-
Optimize Merge() when the receiver is empty 💬 by Caio 2 months ago
This makes reusing an existing TDigest (through Reset()) a reasonable alternative to Clone() goos: linux goarch: amd64 pkg: github.com/caio/go-tdigest/v4 cpu: AMD Ryzen 7 PRO 4750U with Radeon Graphics │ old.txt │ new.txt │ │ sec/op │ sec/op vs base │ MergeEmpty-16 98.035µ ± 2% 3.189µ ± 6% -96.75% (p=0.000 n=10) - Tag v4.1.0 created by Caio on commit a45b433de6 6 months ago
-
Fix: Init rng in TDigest.FromBytes by Nova 6 months ago
-
Add TDigest.Reset by Nova 6 months ago
-
New lint fixes by Caio 9 months ago
-
lint fixes by Caio 2 years ago
T-Digest
A fast map-reduce and parallel streaming friendly data-structure for accurate quantile approximation.
This package provides an implementation of Ted Dunning's t-digest data structure in Go.
Project Status
This project is actively maintained. We are happy to collaborate on features and issues if/when they arrive.
Installation
This package uses go modules. Our releases are tagged and signed following the Semantic Versioning scheme.
go get github.com/caio/go-tdigest/v5
Example Usage
package main
import (
"fmt"
"math/rand"
"github.com/caio/go-tdigest/v5"
)
func main() {
// Analogue to tdigest.New(tdigest.Compression(100))
t, _ := tdigest.New()
for i := 0; i < 10000; i++ {
// Analogue to t.AddWeighted(rand.Float64(), 1)
t.Add(rand.Float64())
}
fmt.Printf("p(.5) = %.6f\n", t.Quantile(0.5))
fmt.Printf("CDF(Quantile(.5)) = %.6f\n", t.CDF(t.Quantile(0.5)))
}
Configuration
You can configure your digest upon creation with options documented at options.go. Example:
// Construct a digest with compression=200 and its own
// (thread-unsafe) RNG seeded with 0xCA10:
digest, _ := tdigest.New(
tdigest.Compression(200),
tdigest.LocalRandomNumberGenerator(0xCA10),
)
References
This is a port of the reference implementation with some ideas borrowed from the python version. If you wanna get a quick grasp of how it works and why it's useful, this video and companion article is pretty helpful.