Package org.apache.spark.api.java

Class JavaHadoopRDD<K,V>

Object

org.apache.spark.api.java.JavaPairRDD<K,V>

org.apache.spark.api.java.JavaHadoopRDD<K,V>

All Implemented Interfaces:

Serializable, JavaRDDLike<scala.Tuple2<K,V>,JavaPairRDD<K,V>>

public class JavaHadoopRDD<K,V>
extends JavaPairRDD<K,V>

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
JavaHadoopRDD(HadoopRDD<K,V> rdd, scala.reflect.ClassTag<K> kClassTag, scala.reflect.ClassTag<V> vClassTag)

Method Summary

Modifier and Type	Method	Description
scala.reflect.ClassTag<K>	kClassTag()
<R> JavaRDD<R>	mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,Iterator<scala.Tuple2<K,V>>,Iterator<R>> f, boolean preservesPartitioning)	Maps over a partition, providing the InputSplit that was used as the base of the partition.
scala.reflect.ClassTag<V>	vClassTag()

Methods inherited from class org.apache.spark.api.java.JavaPairRDD

aggregateByKey, aggregateByKey, aggregateByKey, cache, classTag, coalesce, coalesce, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, collectAsMap, combineByKey, combineByKey, combineByKey, combineByKey, countApproxDistinctByKey, countApproxDistinctByKey, countApproxDistinctByKey, countByKey, countByKeyApprox, countByKeyApprox, distinct, distinct, filter, filterByRange, filterByRange, first, flatMapValues, foldByKey, foldByKey, foldByKey, fromJavaRDD, fromRDD, fullOuterJoin, fullOuterJoin, fullOuterJoin, groupByKey, groupByKey, groupByKey, groupWith, groupWith, groupWith, intersection, join, join, join, keys, leftOuterJoin, leftOuterJoin, leftOuterJoin, lookup, mapValues, partitionBy, persist, rdd, reduceByKey, reduceByKey, reduceByKey, reduceByKeyLocally, repartition, repartitionAndSortWithinPartitions, repartitionAndSortWithinPartitions, rightOuterJoin, rightOuterJoin, rightOuterJoin, sample, sample, sampleByKey, sampleByKey, sampleByKeyExact, sampleByKeyExact, saveAsHadoopDataset, saveAsHadoopFile, saveAsHadoopFile, saveAsHadoopFile, saveAsNewAPIHadoopDataset, saveAsNewAPIHadoopFile, saveAsNewAPIHadoopFile, setName, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, subtract, subtract, subtract, subtractByKey, subtractByKey, subtractByKey, toRDD, union, unpersist, unpersist, values, wrapRDD

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.api.java.JavaRDDLike

aggregate, cartesian, checkpoint, collect, collectAsync, collectPartitions, context, count, countApprox, countApprox, countApproxDistinct, countAsync, countByValue, countByValueApprox, countByValueApprox, flatMap, flatMapToDouble, flatMapToPair, fold, foreach, foreachAsync, foreachPartition, foreachPartitionAsync, getCheckpointFile, getNumPartitions, getStorageLevel, glom, groupBy, groupBy, id, isCheckpointed, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitions, mapPartitionsToDouble, mapPartitionsToDouble, mapPartitionsToPair, mapPartitionsToPair, mapPartitionsWithIndex, mapToDouble, mapToPair, max, min, name, partitioner, partitions, pipe, pipe, pipe, pipe, pipe, reduce, saveAsObjectFile, saveAsTextFile, saveAsTextFile, take, takeAsync, takeOrdered, takeOrdered, takeSample, takeSample, toDebugString, toLocalIterator, top, top, treeAggregate, treeAggregate, treeAggregate, treeReduce, treeReduce, zip, zipPartitions, zipWithIndex, zipWithUniqueId

Constructor Details

JavaHadoopRDD

public JavaHadoopRDD(HadoopRDD<K,V> rdd,
 scala.reflect.ClassTag<K> kClassTag,
 scala.reflect.ClassTag<V> vClassTag)

Method Details

kClassTag

public scala.reflect.ClassTag<K> kClassTag()

Overrides:

kClassTag in class JavaPairRDD<K,V>

mapPartitionsWithInputSplit

public <R> JavaRDD<R> mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,Iterator<scala.Tuple2<K,V>>,Iterator<R>> f,
 boolean preservesPartitioning)

Maps over a partition, providing the InputSplit that was used as the base of the partition.

vClassTag

public scala.reflect.ClassTag<V> vClassTag()

Overrides:

vClassTag in class JavaPairRDD<K,V>