Package org.apache.spark.rdd

Class UnionRDD<T>

Object

org.apache.spark.rdd.RDD<T>

org.apache.spark.rdd.UnionRDD<T>

All Implemented Interfaces:

Serializable, org.apache.spark.internal.Logging

public class UnionRDD<T>
extends RDD<T>

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

Constructor Summary

Constructors

Constructor	Description
UnionRDD(SparkContext sc, scala.collection.immutable.Seq<RDD<T>> rdds, scala.reflect.ClassTag<T> evidence$2)

Method Summary

Modifier and Type	Method	Description
void	clearDependencies()
scala.collection.Iterator<T>	compute(Partition s, TaskContext context)	:: DeveloperApi :: Implemented by subclasses to compute a given partition.
scala.collection.immutable.Seq<Dependency<?>>	getDependencies()
Partition[]	getPartitions()
scala.collection.immutable.Seq<String>	getPreferredLocations(Partition s)
scala.collection.immutable.Seq<RDD<T>>	rdds()

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, barrier, cache, cartesian, checkpoint, cleanShuffleDependencies, coalesce, collect, collect, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, dependencies, distinct, distinct, doubleRDDToDoubleRDDFunctions, filter, first, flatMap, fold, foreach, foreachPartition, getCheckpointFile, getNumPartitions, getResourceProfile, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, isEmpty, iterator, keyBy, localCheckpoint, map, mapPartitions, mapPartitionsWithEvaluator, mapPartitionsWithIndex, max, min, name, numericRDDToDoubleRDDFunctions, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeAggregate, treeReduce, union, unpersist, withResources, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitionsWithEvaluator, zipWithIndex, zipWithUniqueId

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

Constructor Details

UnionRDD

public UnionRDD(SparkContext sc,
 scala.collection.immutable.Seq<RDD<T>> rdds,
 scala.reflect.ClassTag<T> evidence$2)

Method Details

rdds

public scala.collection.immutable.Seq<RDD<T>> rdds()

getPartitions

public Partition[] getPartitions()

getDependencies

public scala.collection.immutable.Seq<Dependency<?>> getDependencies()

compute

public scala.collection.Iterator<T> compute(Partition s,
 TaskContext context)

Description copied from class: RDD

:: DeveloperApi :: Implemented by subclasses to compute a given partition.

Specified by:

compute in class RDD<T>

Parameters:

s - (undocumented)

context - (undocumented)

Returns:

(undocumented)

getPreferredLocations

public scala.collection.immutable.Seq<String> getPreferredLocations(Partition s)

clearDependencies

public void clearDependencies()