Packages

c

org.apache.spark

Aggregator

case class Aggregator[K, V, C](createCombiner: (V) ⇒ C, mergeValue: (C, V) ⇒ C, mergeCombiners: (C, C) ⇒ C) extends Product with Serializable

Developer API

A set of functions used to aggregate data.

createCombiner

function to create the initial value of the aggregation.

mergeValue

function to merge a new value into the aggregation result.

mergeCombiners

function to merge outputs from multiple mergeValue function.

Annotations
@DeveloperApi()
Source
Aggregator.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Aggregator
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Aggregator(createCombiner: (V) ⇒ C, mergeValue: (C, V) ⇒ C, mergeCombiners: (C, C) ⇒ C)

    createCombiner

    function to create the initial value of the aggregation.

    mergeValue

    function to merge a new value into the aggregation result.

    mergeCombiners

    function to merge outputs from multiple mergeValue function.

Value Members

  1. def combineCombinersByKey(iter: Iterator[_ <: Product2[K, C]], context: TaskContext): Iterator[(K, C)]
  2. def combineValuesByKey(iter: Iterator[_ <: Product2[K, V]], context: TaskContext): Iterator[(K, C)]
  3. val createCombiner: (V) ⇒ C
  4. val mergeCombiners: (C, C) ⇒ C
  5. val mergeValue: (C, V) ⇒ C