Class NarrowDependency<T>

Object
org.apache.spark.Dependency<T>
org.apache.spark.NarrowDependency<T>
All Implemented Interfaces:
Serializable, scala.Serializable
Direct Known Subclasses:
OneToOneDependency, RangeDependency

public abstract class NarrowDependency<T> extends Dependency<T>
:: DeveloperApi :: Base class for dependencies where each partition of the child RDD depends on a small number of partitions of the parent RDD. Narrow dependencies allow for pipelined execution.
See Also:
  • Constructor Details

    • NarrowDependency

      public NarrowDependency(RDD<T> _rdd)
  • Method Details

    • getParents

      public abstract scala.collection.Seq<Object> getParents(int partitionId)
      Get the parent partitions for a child partition.
      Parameters:
      partitionId - a partition of the child RDD
      Returns:
      the partitions of the parent RDD that the child partition depends upon
    • rdd

      public RDD<T> rdd()
      Specified by:
      rdd in class Dependency<T>