public class VertexRDDImpl<VD> extends VertexRDD<VD>
Constructor and Description |
---|
VertexRDDImpl(RDD<ShippableVertexPartition<VD>> partitionsRDD,
StorageLevel targetStorageLevel,
scala.reflect.ClassTag<VD> vdTag) |
Modifier and Type | Method and Description |
---|---|
<VD2> VertexRDD<VD2> |
aggregateUsingIndex(RDD<scala.Tuple2<Object,VD2>> messages,
scala.Function2<VD2,VD2,VD2> reduceFunc,
scala.reflect.ClassTag<VD2> evidence$12)
Aggregates vertices in
messages that have the same ids using reduceFunc , returning a
VertexRDD co-indexed with this . |
VertexRDDImpl<VD> |
cache()
Persists the vertex partitions at `targetStorageLevel`, which defaults to MEMORY_ONLY.
|
void |
checkpoint()
Mark this RDD for checkpointing.
|
long |
count()
The number of vertices in the RDD.
|
VertexRDD<VD> |
diff(VertexRDD<VD> other)
Hides vertices that are the same between
this and other ; for vertices that are different,
keeps the values from other . |
<U,VD2> VertexRDD<VD2> |
innerJoin(RDD<scala.Tuple2<Object,U>> other,
scala.Function3<Object,VD,U,VD2> f,
scala.reflect.ClassTag<U> evidence$10,
scala.reflect.ClassTag<VD2> evidence$11)
Inner joins this VertexRDD with an RDD containing vertex attribute pairs.
|
<U,VD2> VertexRDD<VD2> |
innerZipJoin(VertexRDD<U> other,
scala.Function3<Object,VD,U,VD2> f,
scala.reflect.ClassTag<U> evidence$8,
scala.reflect.ClassTag<VD2> evidence$9)
Efficiently inner joins this VertexRDD with another VertexRDD sharing the same index.
|
<VD2,VD3> VertexRDD<VD3> |
leftJoin(RDD<scala.Tuple2<Object,VD2>> other,
scala.Function3<Object,VD,scala.Option<VD2>,VD3> f,
scala.reflect.ClassTag<VD2> evidence$6,
scala.reflect.ClassTag<VD3> evidence$7)
Left joins this VertexRDD with an RDD containing vertex attribute pairs.
|
<VD2,VD3> VertexRDD<VD3> |
leftZipJoin(VertexRDD<VD2> other,
scala.Function3<Object,VD,scala.Option<VD2>,VD3> f,
scala.reflect.ClassTag<VD2> evidence$4,
scala.reflect.ClassTag<VD3> evidence$5)
Left joins this RDD with another VertexRDD with the same index.
|
<VD2> VertexRDD<VD2> |
mapValues(scala.Function1<VD,VD2> f,
scala.reflect.ClassTag<VD2> evidence$2)
Maps each vertex attribute, preserving the index.
|
<VD2> VertexRDD<VD2> |
mapValues(scala.Function2<Object,VD,VD2> f,
scala.reflect.ClassTag<VD2> evidence$3)
Maps each vertex attribute, additionally supplying the vertex ID.
|
<VD2> VertexRDD<VD2> |
mapVertexPartitions(scala.Function1<ShippableVertexPartition<VD>,ShippableVertexPartition<VD2>> f,
scala.reflect.ClassTag<VD2> evidence$1)
Applies a function to each
VertexPartition of this RDD and returns a new VertexRDD. |
scala.Option<Partitioner> |
partitioner()
Optionally overridden by subclasses to specify how they are partitioned.
|
RDD<ShippableVertexPartition<VD>> |
partitionsRDD() |
VertexRDDImpl<VD> |
persist(StorageLevel newLevel)
Persists the vertex partitions at the specified storage level, ignoring any existing target
storage level.
|
VertexRDD<VD> |
reindex()
Construct a new VertexRDD that is indexed by only the visible vertices.
|
VertexRDD<VD> |
reverseRoutingTables()
Returns a new
VertexRDD reflecting a reversal of all edge directions in the corresponding
EdgeRDD . |
VertexRDDImpl<VD> |
setName(String _name)
Assign a name to this RDD
|
RDD<scala.Tuple2<Object,VertexAttributeBlock<VD>>> |
shipVertexAttributes(boolean shipSrc,
boolean shipDst)
Generates an RDD of vertex attributes suitable for shipping to the edge partitions.
|
RDD<scala.Tuple2<Object,long[]>> |
shipVertexIds()
Generates an RDD of vertex IDs suitable for shipping to the edge partitions.
|
StorageLevel |
targetStorageLevel() |
VertexRDDImpl<VD> |
unpersist(boolean blocking)
Mark the RDD as non-persistent, and remove all blocks for it from memory and disk.
|
VertexRDD<VD> |
withEdges(EdgeRDD<?> edges)
Prepares this VertexRDD for efficient joins with the given EdgeRDD.
|
<VD2> VertexRDD<VD2> |
withPartitionsRDD(RDD<ShippableVertexPartition<VD2>> partitionsRDD,
scala.reflect.ClassTag<VD2> evidence$13)
Replaces the vertex partitions while preserving all other properties of the VertexRDD.
|
VertexRDD<VD> |
withTargetStorageLevel(StorageLevel targetStorageLevel)
Changes the target storage level while preserving all other properties of the
VertexRDD.
|
apply, apply, apply, compute, createRoutingTables, filter, fromEdges
aggregate, cartesian, checkpointData, coalesce, collect, collect, collectPartitions, computeOrReadCheckpoint, conf, context, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doCheckpoint, elementClassTag, filterWith, first, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCheckpointFile, getCreationSite, getNarrowAncestors, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, iterator, keyBy, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, markCheckpointed, max, min, name, partitions, persist, pipe, pipe, pipe, preferredLocations, randomSplit, reduce, repartition, retag, retag, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, union, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public VertexRDDImpl(RDD<ShippableVertexPartition<VD>> partitionsRDD, StorageLevel targetStorageLevel, scala.reflect.ClassTag<VD> vdTag)
public RDD<ShippableVertexPartition<VD>> partitionsRDD()
partitionsRDD
in class VertexRDD<VD>
public StorageLevel targetStorageLevel()
public VertexRDD<VD> reindex()
VertexRDD
public scala.Option<Partitioner> partitioner()
RDD
partitioner
in class RDD<scala.Tuple2<Object,VD>>
public VertexRDDImpl<VD> setName(String _name)
RDD
public VertexRDDImpl<VD> persist(StorageLevel newLevel)
public VertexRDDImpl<VD> unpersist(boolean blocking)
RDD
public VertexRDDImpl<VD> cache()
public void checkpoint()
RDD
checkpoint
in class RDD<scala.Tuple2<Object,VD>>
public long count()
public <VD2> VertexRDD<VD2> mapVertexPartitions(scala.Function1<ShippableVertexPartition<VD>,ShippableVertexPartition<VD2>> f, scala.reflect.ClassTag<VD2> evidence$1)
VertexRDD
VertexPartition
of this RDD and returns a new VertexRDD.mapVertexPartitions
in class VertexRDD<VD>
public <VD2> VertexRDD<VD2> mapValues(scala.Function1<VD,VD2> f, scala.reflect.ClassTag<VD2> evidence$2)
VertexRDD
public <VD2> VertexRDD<VD2> mapValues(scala.Function2<Object,VD,VD2> f, scala.reflect.ClassTag<VD2> evidence$3)
VertexRDD
public VertexRDD<VD> diff(VertexRDD<VD> other)
VertexRDD
this
and other
; for vertices that are different,
keeps the values from other
.public <VD2,VD3> VertexRDD<VD3> leftZipJoin(VertexRDD<VD2> other, scala.Function3<Object,VD,scala.Option<VD2>,VD3> f, scala.reflect.ClassTag<VD2> evidence$4, scala.reflect.ClassTag<VD3> evidence$5)
VertexRDD
this
.
If other
is missing any vertex in this VertexRDD, f
is passed None
.
leftZipJoin
in class VertexRDD<VD>
other
- the other VertexRDD with which to join.f
- the function mapping a vertex id and its attributes in this and the other vertex set
to a new vertex attribute.f
public <VD2,VD3> VertexRDD<VD3> leftJoin(RDD<scala.Tuple2<Object,VD2>> other, scala.Function3<Object,VD,scala.Option<VD2>,VD3> f, scala.reflect.ClassTag<VD2> evidence$6, scala.reflect.ClassTag<VD3> evidence$7)
VertexRDD
leftZipJoin
implementation is
used. The resulting VertexRDD contains an entry for each vertex in this
. If other
is
missing any vertex in this VertexRDD, f
is passed None
. If there are duplicates,
the vertex is picked arbitrarily.
leftJoin
in class VertexRDD<VD>
other
- the other VertexRDD with which to joinf
- the function mapping a vertex id and its attributes in this and the other vertex set
to a new vertex attribute.f
.public <U,VD2> VertexRDD<VD2> innerZipJoin(VertexRDD<U> other, scala.Function3<Object,VD,U,VD2> f, scala.reflect.ClassTag<U> evidence$8, scala.reflect.ClassTag<VD2> evidence$9)
VertexRDD
innerJoin
for the behavior of the join.innerZipJoin
in class VertexRDD<VD>
public <U,VD2> VertexRDD<VD2> innerJoin(RDD<scala.Tuple2<Object,U>> other, scala.Function3<Object,VD,U,VD2> f, scala.reflect.ClassTag<U> evidence$10, scala.reflect.ClassTag<VD2> evidence$11)
VertexRDD
innerZipJoin
implementation
is used.
innerJoin
in class VertexRDD<VD>
other
- an RDD containing vertices to join. If there are multiple entries for the same
vertex, one is picked arbitrarily. Use aggregateUsingIndex
to merge multiple entries.f
- the join function applied to corresponding values of this
and other
this
, containing only vertices that appear in both
this
and other
, with values supplied by f
public <VD2> VertexRDD<VD2> aggregateUsingIndex(RDD<scala.Tuple2<Object,VD2>> messages, scala.Function2<VD2,VD2,VD2> reduceFunc, scala.reflect.ClassTag<VD2> evidence$12)
VertexRDD
messages
that have the same ids using reduceFunc
, returning a
VertexRDD co-indexed with this
.
aggregateUsingIndex
in class VertexRDD<VD>
messages
- an RDD containing messages to aggregate, where each message is a pair of its
target vertex ID and the message datareduceFunc
- the associative aggregation function for merging messages to the same vertexthis
, containing only vertices that received messages.
For those vertices, their values are the result of applying reduceFunc
to all received
messages.public VertexRDD<VD> reverseRoutingTables()
VertexRDD
VertexRDD
reflecting a reversal of all edge directions in the corresponding
EdgeRDD
.reverseRoutingTables
in class VertexRDD<VD>
public VertexRDD<VD> withEdges(EdgeRDD<?> edges)
VertexRDD
public <VD2> VertexRDD<VD2> withPartitionsRDD(RDD<ShippableVertexPartition<VD2>> partitionsRDD, scala.reflect.ClassTag<VD2> evidence$13)
VertexRDD
withPartitionsRDD
in class VertexRDD<VD>
public VertexRDD<VD> withTargetStorageLevel(StorageLevel targetStorageLevel)
VertexRDD
This does not actually trigger a cache; to do this, call
RDD.cache()
on the returned VertexRDD.
withTargetStorageLevel
in class VertexRDD<VD>
public RDD<scala.Tuple2<Object,VertexAttributeBlock<VD>>> shipVertexAttributes(boolean shipSrc, boolean shipDst)
VertexRDD
shipVertexAttributes
in class VertexRDD<VD>
public RDD<scala.Tuple2<Object,long[]>> shipVertexIds()
VertexRDD
shipVertexIds
in class VertexRDD<VD>