public class EdgeRDDImpl<ED,VD> extends EdgeRDD<ED>
Constructor and Description |
---|
EdgeRDDImpl(RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> partitionsRDD,
StorageLevel targetStorageLevel,
scala.reflect.ClassTag<ED> evidence$1,
scala.reflect.ClassTag<VD> evidence$2) |
Modifier and Type | Method and Description |
---|---|
EdgeRDDImpl<ED,VD> |
cache()
Persists the edge partitions using `targetStorageLevel`, which defaults to MEMORY_ONLY.
|
void |
checkpoint()
Mark this RDD for checkpointing.
|
Edge<ED>[] |
collect()
Return an array that contains all of the elements in this RDD.
|
long |
count()
The number of edges in the RDD.
|
EdgeRDDImpl<ED,VD> |
filter(scala.Function1<EdgeTriplet<VD,ED>,Object> epred,
scala.Function2<Object,VD,Object> vpred) |
scala.Option<String> |
getCheckpointFile()
Gets the name of the file to which this RDD was checkpointed
|
StorageLevel |
getStorageLevel()
Get the RDD's current storage level, or StorageLevel.NONE if none is set.
|
<ED2,ED3> EdgeRDDImpl<ED3,VD> |
innerJoin(EdgeRDD<ED2> other,
scala.Function4<Object,Object,ED,ED2,ED3> f,
scala.reflect.ClassTag<ED2> evidence$4,
scala.reflect.ClassTag<ED3> evidence$5)
Inner joins this EdgeRDD with another EdgeRDD, assuming both are partitioned using the same
PartitionStrategy . |
boolean |
isCheckpointed()
Return whether this RDD has been checkpointed or not
|
<ED2,VD2> EdgeRDDImpl<ED2,VD2> |
mapEdgePartitions(scala.Function2<Object,EdgePartition<ED,VD>,EdgePartition<ED2,VD2>> f,
scala.reflect.ClassTag<ED2> evidence$6,
scala.reflect.ClassTag<VD2> evidence$7) |
<ED2> EdgeRDDImpl<ED2,VD> |
mapValues(scala.Function1<Edge<ED>,ED2> f,
scala.reflect.ClassTag<ED2> evidence$3)
Map the values in an edge partitioning preserving the structure but changing the values.
|
scala.Option<Partitioner> |
partitioner()
If
partitionsRDD already has a partitioner, use it. |
RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> |
partitionsRDD() |
EdgeRDDImpl<ED,VD> |
persist(StorageLevel newLevel)
Persists the edge partitions at the specified storage level, ignoring any existing target
storage level.
|
EdgeRDDImpl<ED,VD> |
reverse()
Reverse all the edges in this RDD.
|
EdgeRDDImpl<ED,VD> |
setName(String _name)
Assign a name to this RDD
|
StorageLevel |
targetStorageLevel() |
EdgeRDDImpl<ED,VD> |
unpersist(boolean blocking)
Mark the RDD as non-persistent, and remove all blocks for it from memory and disk.
|
<ED2,VD2> EdgeRDDImpl<ED2,VD2> |
withPartitionsRDD(RDD<scala.Tuple2<Object,EdgePartition<ED2,VD2>>> partitionsRDD,
scala.reflect.ClassTag<ED2> evidence$8,
scala.reflect.ClassTag<VD2> evidence$9) |
EdgeRDDImpl<ED,VD> |
withTargetStorageLevel(StorageLevel targetStorageLevel)
Changes the target storage level while preserving all other properties of the
EdgeRDD.
|
compute, fromEdgePartitions, fromEdges
aggregate, cartesian, checkpointData, coalesce, collect, collectPartitions, computeOrReadCheckpoint, conf, context, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doCheckpoint, doubleRDDToDoubleRDDFunctions, elementClassTag, filter, filterWith, first, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCreationSite, getNarrowAncestors, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, markCheckpointed, max, min, name, numericRDDToDoubleRDDFunctions, partitions, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, retag, retag, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeReduce, union, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public EdgeRDDImpl(RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> partitionsRDD, StorageLevel targetStorageLevel, scala.reflect.ClassTag<ED> evidence$1, scala.reflect.ClassTag<VD> evidence$2)
public RDD<scala.Tuple2<Object,EdgePartition<ED,VD>>> partitionsRDD()
public StorageLevel targetStorageLevel()
public EdgeRDDImpl<ED,VD> setName(String _name)
RDD
public scala.Option<Partitioner> partitioner()
partitionsRDD
already has a partitioner, use it. Otherwise assume that the
PartitionID
s in partitionsRDD
correspond to the actual partitions and create a new
partitioner that allows co-partitioning with partitionsRDD
.partitioner
in class RDD<Edge<ED>>
public Edge<ED>[] collect()
RDD
public EdgeRDDImpl<ED,VD> persist(StorageLevel newLevel)
public EdgeRDDImpl<ED,VD> unpersist(boolean blocking)
RDD
public EdgeRDDImpl<ED,VD> cache()
public StorageLevel getStorageLevel()
RDD
getStorageLevel
in class RDD<Edge<ED>>
public void checkpoint()
RDD
checkpoint
in class RDD<Edge<ED>>
public boolean isCheckpointed()
RDD
isCheckpointed
in class RDD<Edge<ED>>
public scala.Option<String> getCheckpointFile()
RDD
getCheckpointFile
in class RDD<Edge<ED>>
public <ED2> EdgeRDDImpl<ED2,VD> mapValues(scala.Function1<Edge<ED>,ED2> f, scala.reflect.ClassTag<ED2> evidence$3)
EdgeRDD
public EdgeRDDImpl<ED,VD> reverse()
EdgeRDD
public EdgeRDDImpl<ED,VD> filter(scala.Function1<EdgeTriplet<VD,ED>,Object> epred, scala.Function2<Object,VD,Object> vpred)
public <ED2,ED3> EdgeRDDImpl<ED3,VD> innerJoin(EdgeRDD<ED2> other, scala.Function4<Object,Object,ED,ED2,ED3> f, scala.reflect.ClassTag<ED2> evidence$4, scala.reflect.ClassTag<ED3> evidence$5)
EdgeRDD
PartitionStrategy
.
public <ED2,VD2> EdgeRDDImpl<ED2,VD2> mapEdgePartitions(scala.Function2<Object,EdgePartition<ED,VD>,EdgePartition<ED2,VD2>> f, scala.reflect.ClassTag<ED2> evidence$6, scala.reflect.ClassTag<VD2> evidence$7)
public <ED2,VD2> EdgeRDDImpl<ED2,VD2> withPartitionsRDD(RDD<scala.Tuple2<Object,EdgePartition<ED2,VD2>>> partitionsRDD, scala.reflect.ClassTag<ED2> evidence$8, scala.reflect.ClassTag<VD2> evidence$9)
public EdgeRDDImpl<ED,VD> withTargetStorageLevel(StorageLevel targetStorageLevel)
EdgeRDD
This does not actually trigger a cache; to do this, call
RDD.cache()
on the returned EdgeRDD.
withTargetStorageLevel
in class EdgeRDD<ED>