Packages

c

org.apache.spark.sql.util

MapperRowCounter

class MapperRowCounter extends AccumulatorV2[Long, List[(Integer, Long)]]

An AccumulatorV2 counter for collecting a list of (mapper index, row count).

Source
MapperRowCounter.scala
Since

3.4.0

Linear Supertypes
AccumulatorV2[Long, List[(Integer, Long)]], Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MapperRowCounter
  2. AccumulatorV2
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new MapperRowCounter()

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def add(v: Long): Unit

    Takes the inputs and accumulates.

    Takes the inputs and accumulates.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  7. def copy(): MapperRowCounter

    Creates a new copy of this accumulator.

    Creates a new copy of this accumulator.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  8. def copyAndReset(): MapperRowCounter

    Creates a new copy of this accumulator, which is zero value.

    Creates a new copy of this accumulator, which is zero value. i.e. call isZero on the copy must return true.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  11. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  12. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  13. final def id: Long

    Returns the id of this accumulator, can only be called after registration.

    Returns the id of this accumulator, can only be called after registration.

    Definition Classes
    AccumulatorV2
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def isRegistered: Boolean

    Returns true if this accumulator has been registered.

    Returns true if this accumulator has been registered.

    Definition Classes
    AccumulatorV2
    Note

    All accumulators must be registered before use, or it will throw exception.

  16. def isZero: Boolean

    Returns false if this accumulator has had any values added to it or the sum is non-zero.

    Returns false if this accumulator has had any values added to it or the sum is non-zero.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  17. def merge(other: AccumulatorV2[Long, List[(Integer, Long)]]): Unit

    Merges another same-type accumulator into this one and update its state, i.e.

    Merges another same-type accumulator into this one and update its state, i.e. this should be merge-in-place.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  18. final def name: Option[String]

    Returns the name of this accumulator, can only be called after registration.

    Returns the name of this accumulator, can only be called after registration.

    Definition Classes
    AccumulatorV2
  19. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  21. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  22. def reset(): Unit

    Resets this accumulator, which is zero value.

    Resets this accumulator, which is zero value. i.e. call isZero must return true.

    Definition Classes
    MapperRowCounter → AccumulatorV2
  23. def setPartitionId(id: Integer): Unit
  24. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  25. def toString(): String
    Definition Classes
    AccumulatorV2 → AnyRef → Any
  26. def value: List[(Integer, Long)]

    Defines the current value of this accumulator

    Defines the current value of this accumulator

    Definition Classes
    MapperRowCounter → AccumulatorV2
  27. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  28. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  29. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  30. def withBufferSerialized(): AccumulatorV2[Long, List[(Integer, Long)]]
    Attributes
    protected
    Definition Classes
    AccumulatorV2
  31. final def writeReplace(): Any
    Attributes
    protected
    Definition Classes
    AccumulatorV2

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from AccumulatorV2[Long, List[(Integer, Long)]]

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped