Packages

t

org.apache.spark.rdd

PartitionCoalescer

trait PartitionCoalescer extends AnyRef

Developer API

::DeveloperApi:: A PartitionCoalescer defines how to coalesce the partitions of a given RDD.

Annotations
@DeveloperApi()
Source
coalesce-public.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PartitionCoalescer
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def coalesce(maxPartitions: Int, parent: RDD[_]): Array[PartitionGroup]

    Coalesce the partitions of the given RDD.

    Coalesce the partitions of the given RDD.

    maxPartitions

    the maximum number of partitions to have after coalescing

    parent

    the parent RDD whose partitions to coalesce

    returns

    an array of PartitionGroups, where each element is itself an array of Partitions and represents a partition after coalescing is performed.