The base class for accumulators, that can accumulate inputs of type IN
, and produce output of
type OUT
.
The base class for accumulators, that can accumulate inputs of type IN
, and produce output of
type OUT
.
OUT
should be a type that can be read atomically (e.g., Int, Long), or thread-safely
(e.g., synchronized collections) because it will be read from other threads.
An accumulator for collecting a list of elements.
An accumulator for collecting a list of elements.
2.0.0
An accumulator for computing sum, count, and averages for double precision floating numbers.
An accumulator for computing sum, count, and averages for double precision floating numbers.
2.0.0
An accumulator for computing sum, count, and average of 64-bit integers.
An accumulator for computing sum, count, and average of 64-bit integers.
2.0.0
:: DeveloperApi :: A tuple of 2 elements.
:: DeveloperApi :: A tuple of 2 elements. This can be used as an alternative to Scala's Tuple2 when we want to minimize object allocation.
Element 1 of this MutablePair
Element 2 of this MutablePair
A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way.
A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way. Includes support for merging two StatCounters. Based on Welford and Chan's algorithms for running variance.
:: DeveloperApi ::
:: DeveloperApi ::
Listener providing a callback function to invoke when a task's execution completes.
:: DeveloperApi ::
:: DeveloperApi ::
Listener providing a callback function to invoke when a task's execution encounters an error.
Operations defined here must be idempotent, as onTaskFailure
can be called multiple times.
:: DeveloperApi :: Estimates the sizes of Java objects (number of bytes of memory they occupy), for use in memory-aware caches.
:: DeveloperApi :: Estimates the sizes of Java objects (number of bytes of memory they occupy), for use in memory-aware caches.
Based on the following JavaWorld article: http://www.javaworld.com/javaworld/javaqa/2003-12/02-qa-1226-sizeof.html
Utilities for random number generation.
Spark utilities.