Package org.apache.spark.rdd

package org.apache.spark.rdd
Provides implementation's of various RDDs.
  • Class
    Description
    AsyncRDDActions<T>
    A set of asynchronous RDD actions available through an implicit conversion.
    CheckpointState
    Enumeration to manage state transitions of an RDD through checkpointing
    CoGroupedRDD<K>
    :: DeveloperApi :: An RDD that cogroups its parents.
    DefaultPartitionCoalescer
    Coalesce the partitions of a parent RDD (prev) into fewer partitions, so that each partition of this RDD computes one or more of the parent ones.
    DeterministicLevel
    The deterministic level of RDD's output (i.e.
    DoubleRDDFunctions
    Extra functions available on RDDs of Doubles through an implicit conversion.
    HadoopRDD<K,V>
    :: DeveloperApi :: An RDD that provides core functionality for reading data stored in Hadoop (e.g., files in HDFS, sources in HBase, or S3), using the older MapReduce API (org.apache.hadoop.mapred).
    HadoopRDD.HadoopMapPartitionsWithSplitRDD$
     
    InputFileBlockHolder
    This holds file names of the current Spark task.
    JdbcRDD<T>
    An RDD that executes a SQL query on a JDBC connection and reads results.
    JdbcRDD.ConnectionFactory
     
    NewHadoopRDD<K,V>
    :: DeveloperApi :: An RDD that provides core functionality for reading data stored in Hadoop (e.g., files in HDFS, sources in HBase, or S3), using the new MapReduce API (org.apache.hadoop.mapreduce).
    NewHadoopRDD.NewHadoopMapPartitionsWithSplitRDD$
     
    OrderedRDDFunctions<K,V,P extends scala.Product2<K,V>>
    Extra functions available on RDDs of (key, value) pairs where the key is sortable through an implicit conversion.
    PairRDDFunctions<K,V>
    Extra functions available on RDDs of (key, value) pairs through an implicit conversion.
    PartitionCoalescer
    ::DeveloperApi:: A PartitionCoalescer defines how to coalesce the partitions of a given RDD.
    PartitionGroup
    ::DeveloperApi:: A group of Partitions param: prefLoc preferred location for the partition group
    PartitionPruningRDD<T>
    :: DeveloperApi :: An RDD used to prune RDD partitions/partitions so we can avoid launching tasks on all partitions.
    RDD<T>
    A Resilient Distributed Dataset (RDD), the basic abstraction in Spark.
    RDDBarrier<T>
    :: Experimental :: Wraps an RDD in a barrier stage, which forces Spark to launch tasks of this stage together.
    SequenceFileRDDFunctions<K,V>
    Extra functions available on RDDs of (key, value) pairs to create a Hadoop SequenceFile, through an implicit conversion.
    ShuffledRDD<K,V,C>
    :: DeveloperApi :: The resulting RDD from a shuffle (e.g.
    UnionRDD<T>