org.apache.spark.graphx

Class EdgeRDD<ED>

Object

org.apache.spark.rdd.RDD<Edge<ED>>

org.apache.spark.graphx.EdgeRDD<ED>

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.internal.Logging

Direct Known Subclasses:

EdgeRDDImpl

public abstract class EdgeRDD<ED>
extends RDD<Edge<ED>>

EdgeRDD[ED, VD] extends RDD[Edge[ED} by storing the edges in columnar format on each partition for performance. It may additionally store the vertex attributes associated with each edge to provide the triplet view. Shipping of the vertex attributes is managed by impl.ReplicatedVertexView.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.SparkShellLoggingFilter

Constructor Summary

Constructors

Constructor and Description
EdgeRDD(SparkContext sc, scala.collection.Seq<Dependency<?>> deps)

Method Summary

Modifier and Type	Method and Description
scala.collection.Iterator<Edge<ED>>	compute(Partition part, TaskContext context) :: DeveloperApi :: Implemented by subclasses to compute a given partition.
static <ED,VD> EdgeRDDImpl<ED,VD>	fromEdges(RDD<Edge<ED>> edges, scala.reflect.ClassTag<ED> evidence$4, scala.reflect.ClassTag<VD> evidence$5) Creates an EdgeRDD from a set of edges.
abstract <ED2,ED3> EdgeRDD<ED3>	innerJoin(EdgeRDD<ED2> other, scala.Function4<Object,Object,ED,ED2,ED3> f, scala.reflect.ClassTag<ED2> evidence$2, scala.reflect.ClassTag<ED3> evidence$3) Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.
abstract <ED2> EdgeRDD<ED2>	mapValues(scala.Function1<Edge<ED>,ED2> f, scala.reflect.ClassTag<ED2> evidence$1) Map the values in an edge partitioning preserving the structure but changing the values.
abstract EdgeRDD<ED>	reverse() Reverse all the edges in this RDD.

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, barrier, cache, cartesian, checkpoint, cleanShuffleDependencies, coalesce, collect, collect, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, dependencies, distinct, distinct, doubleRDDToDoubleRDDFunctions, filter, first, flatMap, fold, foreach, foreachPartition, getCheckpointFile, getNumPartitions, getResourceProfile, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, isEmpty, iterator, keyBy, localCheckpoint, map, mapPartitions, mapPartitionsWithIndex, max, min, name, numericRDDToDoubleRDDFunctions, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeAggregate, treeReduce, union, unpersist, withResources, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize

Constructor Detail

EdgeRDD

public EdgeRDD(SparkContext sc,
               scala.collection.Seq<Dependency<?>> deps)

Method Detail

fromEdges

public static <ED,VD> EdgeRDDImpl<ED,VD> fromEdges(RDD<Edge<ED>> edges,
                                                   scala.reflect.ClassTag<ED> evidence$4,
                                                   scala.reflect.ClassTag<VD> evidence$5)

Creates an EdgeRDD from a set of edges.

Parameters:

edges - (undocumented)

evidence$4 - (undocumented)

evidence$5 - (undocumented)

Returns:

(undocumented)

compute

public scala.collection.Iterator<Edge<ED>> compute(Partition part,
                                                   TaskContext context)

Description copied from class: RDD

:: DeveloperApi :: Implemented by subclasses to compute a given partition.

Specified by:

compute in class RDD<Edge<ED>>

Parameters:

part - (undocumented)

context - (undocumented)

Returns:

(undocumented)

mapValues

public abstract <ED2> EdgeRDD<ED2> mapValues(scala.Function1<Edge<ED>,ED2> f,
                                             scala.reflect.ClassTag<ED2> evidence$1)

Map the values in an edge partitioning preserving the structure but changing the values.

Parameters:

f - the function from an edge to a new edge value

evidence$1 - (undocumented)

Returns:

a new EdgeRDD containing the new edge values

reverse

public abstract EdgeRDD<ED> reverse()

Reverse all the edges in this RDD.

Returns:

a new EdgeRDD containing all the edges reversed

innerJoin

public abstract <ED2,ED3> EdgeRDD<ED3> innerJoin(EdgeRDD<ED2> other,
                                                 scala.Function4<Object,Object,ED,ED2,ED3> f,
                                                 scala.reflect.ClassTag<ED2> evidence$2,
                                                 scala.reflect.ClassTag<ED3> evidence$3)

Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same PartitionStrategy.

Parameters:

other - the EdgeRDD to join with

f - the join function applied to corresponding values of this and other

evidence$2 - (undocumented)

evidence$3 - (undocumented)

Returns:

a new EdgeRDD containing only edges that appear in both this and other, with values supplied by f