public class RangePartitioner<K,V> extends Partitioner
Partitioner
that partitions sortable records by range into roughly
equal ranges. The ranges are determined by sampling the content of the RDD passed in.
partitions
parameter, in the case where the number of sampled records is less than
the value of partitions
.Constructor and Description |
---|
RangePartitioner(int partitions,
RDD<? extends scala.Product2<K,V>> rdd,
boolean ascending,
int samplePointsPerPartitionHint,
scala.math.Ordering<K> evidence$1,
scala.reflect.ClassTag<K> evidence$2) |
RangePartitioner(int partitions,
RDD<? extends scala.Product2<K,V>> rdd,
boolean ascending,
scala.math.Ordering<K> evidence$3,
scala.reflect.ClassTag<K> evidence$4) |
Modifier and Type | Method and Description |
---|---|
static <K> Object |
determineBounds(scala.collection.mutable.ArrayBuffer<scala.Tuple2<K,Object>> candidates,
int partitions,
scala.math.Ordering<K> evidence$6,
scala.reflect.ClassTag<K> evidence$7)
Determines the bounds for range partitioning from candidates with weights indicating how many
items each represents.
|
boolean |
equals(Object other) |
int |
getPartition(Object key) |
int |
hashCode() |
int |
numPartitions() |
int |
samplePointsPerPartitionHint() |
static <K> scala.Tuple2<Object,scala.Tuple3<Object,Object,Object>[]> |
sketch(RDD<K> rdd,
int sampleSizePerPartition,
scala.reflect.ClassTag<K> evidence$5)
Sketches the input RDD via reservoir sampling on each partition.
|
defaultPartitioner
public RangePartitioner(int partitions, RDD<? extends scala.Product2<K,V>> rdd, boolean ascending, int samplePointsPerPartitionHint, scala.math.Ordering<K> evidence$1, scala.reflect.ClassTag<K> evidence$2)
public static <K> scala.Tuple2<Object,scala.Tuple3<Object,Object,Object>[]> sketch(RDD<K> rdd, int sampleSizePerPartition, scala.reflect.ClassTag<K> evidence$5)
rdd
- the input RDD to sketchsampleSizePerPartition
- max sample size per partitionevidence$5
- (undocumented)public static <K> Object determineBounds(scala.collection.mutable.ArrayBuffer<scala.Tuple2<K,Object>> candidates, int partitions, scala.math.Ordering<K> evidence$6, scala.reflect.ClassTag<K> evidence$7)
candidates
- unordered candidates with weightspartitions
- number of partitionsevidence$6
- (undocumented)evidence$7
- (undocumented)public int samplePointsPerPartitionHint()
public int numPartitions()
numPartitions
in class Partitioner
public int getPartition(Object key)
getPartition
in class Partitioner
public boolean equals(Object other)
equals
in class Object
public int hashCode()
hashCode
in class Object