Log for serialization.go

• Make AsBytes a convenience wrapper around ToBytes 💬 by Caio 6 years ago

  Before:
  
  > BenchmarkAsBytes-4  90274	    13426 ns/op	   1952 B/op	    210 allocs/op
  
  After:
  
  > BenchmarkAsBytes-4  877605	     1348 ns/op	   1536 B/op	      1 allocs/op
  

• merge upstream by Ian Wilkes 6 years ago
• Merge remote-tracking branch 'upstream/master' into ian.uint64 by Ian Wilkes 6 years ago
• Make FromBytes accept options for constructing the digest by Caio 7 years ago
• Fix nil rng by Vladimir Mihailenco 7 years ago
• More verbose documentation for FromBytes by Caio 7 years ago
• Change tdigest.FromBytes to decode directly into summary by Vladimir Mihailenco 7 years ago
• Add TDigest.FromBytes and cleanup RNG interface by Vladimir Mihailenco 7 years ago
• add optimized serialization methods by Ian Wilkes 7 years ago
• Use uint64 instead of uint32 to avoid overflow issues by Eben Freeman 7 years ago
• Get rid of the centroid abstraction 💬 by Caio 8 years ago

  This was only being used to pack {float64,uint32}, all the
  other functionality was skipped or became unused over
  time for performance reasons. Away it goes.
  

• Make Add take only one parameter, introduce AddWeighted 💬 by Caio 8 years ago

  This patch renames the previous Add(float64,uint32) to AddWeighted
  and introduces a method Add(float64) which is simply an alias
  to AddWeighted(float64,1).
  

• Make New() return an error instead of panic()ing 💬 by Caio 8 years ago

  This patch now makes New() return a (*TDigest,error) tuple,
  which makes deserialization safe without having to trap
  for panic()s.
  
  The only remaining panic() is for bad input in a public
  function (`Quantile(float64)`). I'm keen on keeping it.
  

• Introduce a parameter-less New() 💬 by Caio 8 years ago

  Now `tdigest.New()` gives a sane ready-to-use-in-most-cases digest.
  
  Configuration should be done via self referential functions. Ex:
  
      // create a digest with compression of 200
      tdigest.New(tdigest.Compression(200))
  
  Notice that New() can still panic, which means that deserialization
  if still more dangerous than it should.
  

• Revert "gometalinter: Use interfaces when it makes sense" 💬 by Caio 9 years ago

  This reverts commit 2686af5cd15f0a8f4650e2a4a9bbf44c05b87e10.
  
  I still don't like this even after sleeping on it :-)
  

• gometalinter: Use interfaces when it makes sense 💬 by Caio 9 years ago

  There's now another suggestion:
  
      $ gometalinter ./... --disable=gocyclo
      serialization.go:63:16:warning: buf can be jpeg.Reader (interfacer)
  
  But that's just too silly.
  
  To be honest, I'm not so sure this is a sensical change even:
  encodeUint and encodeUint are "private". bytes.{Buffer,Reader} is
  precisely what I need and use throughout the implementation. So,
  hashtag revertMeMaybe.
  

• Add missing error checks 💬 by Caio 9 years ago

  This patch fixes some missing error checks as pointed out by
  `gometalinter` (thanks @dgryski!).
  
  There is a signature change on Merge() and Compress(), from returning
  nothing to now returning an error type.
  

• Merge sortedSlice and Summary (i.e.: move away from interface{}) 💬 by Caio 10 years ago

  A lot of tedious work, but this allowed me to drastically speed up
  the computeCentroidQuantile() which was the most expensive function
  by far.
  
  Current timings are looking beter than ever:
  
  > $ go test -run XXX -bench .
  > PASS
  > BenchmarkAdd1-4          3000000               399 ns/op
  > BenchmarkAdd10-4         2000000              1048 ns/op
  > BenchmarkAdd100-4        1000000              2588 ns/op
  > ok      github.com/caio/go-tdigest      7.334s
  

• Make serialization compatible with the java version 💬 by Caio 10 years ago

  This patch adjusts how deltas are encoded so it behaves exactly the same
  as the reference implementation. A test is added to make sure this
  doesn't break in the future.
  

• Use a sorted slice instead of a tree for centroids 💬 by Caio 10 years ago

  Got massive improvements already simply by glueing the sortedSlice
  struct without exploiting its advantages (ceil/floor and surroundings
  are cheaper operations, relatively speaking)
  
  > $ go test -v ./... -run XXX -bench .
  > PASS
  > BenchmarkAdd1-4          3000000               511 ns/op
  > BenchmarkAdd10-4         1000000              4304 ns/op
  > BenchmarkAdd100-4         100000             22212 ns/op
  > ok      github.com/caio/go-tdigest      8.875s
  
  To be done:
  
   * Adapt the code to properly use the data structure
  
   * Estimate properly the initial capacity
  
   * Maybe get rid of the `interface{}` abstraction if I care enough (so
     instead of TDigest.summary.tree use TDigest.tree directly, for
     example). This is probably not worth the effort, performance-wise.
  

• Rename `Update()` to `Add()` 💬 by Caio 10 years ago

  Trying to keep the interface closer to the reference implementation.
  Arguably it's a better name anyway since we're adding a new datapoint to
  the summary.
  

• Make sure numCentroids value is sane 💬 by Damian Gryski 10 years ago

  Fixes a crash if numCentroids was negative, and potential out-of-memory
  condition if we try to allocate to huge a digest.
  
  And 4 million centroids ought to be enough for anybody...
  
  Found by fuzzing.
  

• Stop copying/creating centroids all the time 💬 by Caio 10 years ago

  This patch makes all internal calls handle *centroid instead of the
  struct directy. With some crappy profiling in osx, a lot of time was
  spent allocating and moving memory around so this seemed like a very
  easy improvement to make.
  
  > $ go test -v ./... -bench . -run XXX
  > PASS
  > BenchmarkUpdate1-4       1000000              1364 ns/op
  > BenchmarkUpdate10-4       200000             11540 ns/op
  > BenchmarkUpdate100-4       30000             58091 ns/op
  > ok      github.com/caio/go-tdigest      6.026s
  
  This is roughly an unscientific 300% improvement in the compression=100
  case (which is what should generally be used, according to the original
  paper) - Ref: c6047295a555a07c80a203c9b64e20a423cb6fed.
  

• Lint: Remove `= 0` from declarations 💬 by Caio 10 years ago

  Originally removed on c29cee2ee58306cce780d53be2e0b7ac2666403f, and lost
  during the merge resolution of other pull requests.
  

• Move everything over to a closure-based iteration model 💬 by Damian Gryski 10 years ago

  Cherry-pick dgryski/no-goroutines and resolve the conflicts
  

• use varint routines from encoding/binary 💬 by Damian Gryski 10 years ago

  Cherry pick of `dgryski/varint` fixing the merge conflicts
  

• golint fixes by Damian Gryski 10 years ago
• Tidy up the serialization "constants" by Caio 10 years ago
• Unexpose the Centroid struct (rename it to centroid) by Caio 10 years ago
• Add some docs 💬 by Caio 10 years ago

  Because why not.