Packages

o

org.apache.spark.rdd

DeterministicLevel

object DeterministicLevel extends Enumeration

Developer API

The deterministic level of RDD's output (i.e. what RDD#compute returns). This explains how the output will diff when Spark reruns the tasks for the RDD. There are 3 deterministic levels: 1. DETERMINATE: The RDD output is always the same data set in the same order after a rerun. 2. UNORDERED: The RDD output is always the same data set but the order can be different after a rerun. 3. INDETERMINATE. The RDD output can be different after a rerun.

Note that, the output of an RDD usually relies on the parent RDDs. When the parent RDD's output is INDETERMINATE, it's very likely the RDD's output is also INDETERMINATE.

Annotations
@DeveloperApi()
Source
RDD.scala
Linear Supertypes
Enumeration, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeterministicLevel
  2. Enumeration
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Type Members

  1. abstract class Value extends Ordered[Value] with Serializable
    Definition Classes
    Enumeration
    Annotations
    @SerialVersionUID()
  2. class ValueSet extends AbstractSet[Value] with SortedSet[Value] with SortedSetOps[Value, SortedSet, ValueSet] with StrictOptimizedIterableOps[Value, Set, ValueSet] with Serializable
    Definition Classes
    Enumeration
    Annotations
    @SerialVersionUID()

Value Members

  1. val DETERMINATE: Value
  2. val INDETERMINATE: Value
  3. val UNORDERED: Value
  4. final def apply(x: Int): Value
    Definition Classes
    Enumeration
  5. final def maxId: Int
    Definition Classes
    Enumeration
  6. def toString(): String
    Definition Classes
    Enumeration → AnyRef → Any
  7. def values: ValueSet
    Definition Classes
    Enumeration
  8. final def withName(s: String): Value
    Definition Classes
    Enumeration
  9. implicit object ValueOrdering extends Ordering[Value]
    Definition Classes
    Enumeration