Class StatCounter

Object
org.apache.spark.util.StatCounter
All Implemented Interfaces:
Serializable, scala.Serializable

public class StatCounter extends Object implements scala.Serializable
A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way. Includes support for merging two StatCounters. Based on Welford and Chan's algorithms for running variance.

See Also:
  • Constructor Summary

    Constructors
    Constructor
    Description
    Initialize the StatCounter with no values.
    StatCounter(scala.collection.TraversableOnce<Object> values)
     
  • Method Summary

    Modifier and Type
    Method
    Description
    apply(scala.collection.Seq<Object> values)
    Build a StatCounter from a list of values passed as variable-length arguments.
    apply(scala.collection.TraversableOnce<Object> values)
    Build a StatCounter from a list of values.
    Clone this StatCounter
    long
     
    double
    max()
     
    double
     
    merge(double value)
    Add a value into this StatCounter, updating the internal statistics.
    Merge another StatCounter into this one, adding up the internal statistics.
    merge(scala.collection.TraversableOnce<Object> values)
    Add multiple values into this StatCounter, updating the internal statistics.
    double
    min()
     
    double
    Return the population standard deviation of the values.
    double
    Return the population variance of the values.
    double
    Return the sample standard deviation of the values, which corrects for bias in estimating the variance by dividing by N-1 instead of N.
    double
    Return the sample variance, which corrects for bias in estimating the variance by dividing by N-1 instead of N.
    double
    Return the population standard deviation of the values.
    double
    sum()
     
     
    double
    Return the population variance of the values.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, wait, wait, wait
  • Constructor Details

    • StatCounter

      public StatCounter(scala.collection.TraversableOnce<Object> values)
    • StatCounter

      public StatCounter()
      Initialize the StatCounter with no values.
  • Method Details

    • apply

      public static StatCounter apply(scala.collection.TraversableOnce<Object> values)
      Build a StatCounter from a list of values.
    • apply

      public static StatCounter apply(scala.collection.Seq<Object> values)
      Build a StatCounter from a list of values passed as variable-length arguments.
    • merge

      public StatCounter merge(double value)
      Add a value into this StatCounter, updating the internal statistics.
    • merge

      public StatCounter merge(scala.collection.TraversableOnce<Object> values)
      Add multiple values into this StatCounter, updating the internal statistics.
    • merge

      public StatCounter merge(StatCounter other)
      Merge another StatCounter into this one, adding up the internal statistics.
    • copy

      public StatCounter copy()
      Clone this StatCounter
    • count

      public long count()
    • mean

      public double mean()
    • sum

      public double sum()
    • max

      public double max()
    • min

      public double min()
    • variance

      public double variance()
      Return the population variance of the values.
    • popVariance

      public double popVariance()
      Return the population variance of the values.
      Returns:
      (undocumented)
    • sampleVariance

      public double sampleVariance()
      Return the sample variance, which corrects for bias in estimating the variance by dividing by N-1 instead of N.
      Returns:
      (undocumented)
    • stdev

      public double stdev()
      Return the population standard deviation of the values.
    • popStdev

      public double popStdev()
      Return the population standard deviation of the values.
      Returns:
      (undocumented)
    • sampleStdev

      public double sampleStdev()
      Return the sample standard deviation of the values, which corrects for bias in estimating the variance by dividing by N-1 instead of N.
      Returns:
      (undocumented)
    • toString

      public String toString()
      Overrides:
      toString in class Object