org.apache.spark.api.java
Class JavaHadoopRDD<K,V>

Object
  org.apache.spark.api.java.JavaPairRDD<K,V>
      org.apache.spark.api.java.JavaHadoopRDD<K,V>

All Implemented Interfaces:

java.io.Serializable, JavaRDDLike<scala.Tuple2<K,V>,JavaPairRDD<K,V>>

public class JavaHadoopRDD<K,V>
extends JavaPairRDD<K,V>

See Also:

Serialized Form

Constructor Summary

JavaHadoopRDD(HadoopRDD<K,V> rdd, scala.reflect.ClassTag<K> kClassTag, scala.reflect.ClassTag<V> vClassTag)

Method Summary

scala.reflect.ClassTag<K>	kClassTag()
<R> JavaRDD<R>	mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,java.util.Iterator<scala.Tuple2<K,V>>,java.util.Iterator<R>> f, boolean preservesPartitioning) Maps over a partition, providing the InputSplit that was used as the base of the partition.
scala.reflect.ClassTag<V>	vClassTag()

Methods inherited from class org.apache.spark.api.java.JavaPairRDD

aggregateByKey, aggregateByKey, aggregateByKey, cache, classTag, coalesce, coalesce, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, cogroup, collectAsMap, combineByKey, combineByKey, combineByKey, combineByKey, countApproxDistinctByKey, countApproxDistinctByKey, countApproxDistinctByKey, countByKey, countByKeyApprox, countByKeyApprox, distinct, distinct, filter, first, flatMapValues, foldByKey, foldByKey, foldByKey, fromJavaRDD, fromRDD, fullOuterJoin, fullOuterJoin, fullOuterJoin, groupByKey, groupByKey, groupByKey, groupWith, groupWith, groupWith, intersection, join, join, join, keys, leftOuterJoin, leftOuterJoin, leftOuterJoin, lookup, mapValues, partitionBy, persist, rdd, reduceByKey, reduceByKey, reduceByKey, reduceByKeyLocally, repartition, repartitionAndSortWithinPartitions, repartitionAndSortWithinPartitions, rightOuterJoin, rightOuterJoin, rightOuterJoin, sample, sample, sampleByKey, sampleByKey, sampleByKeyExact, sampleByKeyExact, saveAsHadoopDataset, saveAsHadoopFile, saveAsHadoopFile, saveAsHadoopFile, saveAsNewAPIHadoopDataset, saveAsNewAPIHadoopFile, saveAsNewAPIHadoopFile, setName, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, sortByKey, subtract, subtract, subtract, subtractByKey, subtractByKey, subtractByKey, toRDD, union, unpersist, unpersist, values, wrapRDD

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.api.java.JavaRDDLike

aggregate, cartesian, checkpoint, collect, collectAsync, collectPartitions, context, count, countApprox, countApprox, countApproxDistinct, countAsync, countByValue, countByValueApprox, countByValueApprox, flatMap, flatMapToDouble, flatMapToPair, fold, foreach, foreachAsync, foreachPartition, foreachPartitionAsync, getCheckpointFile, getStorageLevel, glom, groupBy, groupBy, id, isCheckpointed, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitions, mapPartitionsToDouble, mapPartitionsToDouble, mapPartitionsToPair, mapPartitionsToPair, mapPartitionsWithIndex, mapToDouble, mapToPair, max, min, name, partitions, pipe, pipe, pipe, reduce, saveAsObjectFile, saveAsTextFile, saveAsTextFile, splits, take, takeAsync, takeOrdered, takeOrdered, takeSample, takeSample, toArray, toDebugString, toLocalIterator, top, top, treeAggregate, treeAggregate, treeReduce, treeReduce, zip, zipPartitions, zipWithIndex, zipWithUniqueId

Constructor Detail

JavaHadoopRDD

public JavaHadoopRDD(HadoopRDD<K,V> rdd,
                     scala.reflect.ClassTag<K> kClassTag,
                     scala.reflect.ClassTag<V> vClassTag)

Method Detail

kClassTag

public scala.reflect.ClassTag<K> kClassTag()

Overrides:

kClassTag in class JavaPairRDD<K,V>

vClassTag

public scala.reflect.ClassTag<V> vClassTag()

Overrides:

vClassTag in class JavaPairRDD<K,V>

mapPartitionsWithInputSplit

public <R> JavaRDD<R> mapPartitionsWithInputSplit(Function2<org.apache.hadoop.mapred.InputSplit,java.util.Iterator<scala.Tuple2<K,V>>,java.util.Iterator<R>> f,
                                                  boolean preservesPartitioning)

Maps over a partition, providing the InputSplit that was used as the base of the partition.