org.apache.spark.mllib.rdd
Class RDDFunctions<T>

Object
  org.apache.spark.mllib.rdd.RDDFunctions<T>

All Implemented Interfaces:

java.io.Serializable

public class RDDFunctions<T>
extends Object
implements scala.Serializable

Machine learning specific RDD functions.

See Also:

Serialized Form

Constructor Summary

RDDFunctions(RDD<T> self, scala.reflect.ClassTag<T> evidence$1)

Method Summary

static <T> RDDFunctions<T>	fromRDD(RDD<T> rdd, scala.reflect.ClassTag<T> evidence$3) Implicit conversion from an RDD to RDDFunctions.
RDD<Object>	sliding(int windowSize) Returns a RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them.
<U> U	treeAggregate(U zeroValue, scala.Function2<U,T,U> seqOp, scala.Function2<U,U,U> combOp, int depth, scala.reflect.ClassTag<U> evidence$2) Deprecated. Use RDD.treeAggregate(U, scala.Function2, scala.Function2, int, scala.reflect.ClassTag) instead.
T	treeReduce(scala.Function2<T,T,T> f, int depth) Deprecated. Use RDD.treeReduce(scala.Function2, int) instead.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RDDFunctions

public RDDFunctions(RDD<T> self,
                    scala.reflect.ClassTag<T> evidence$1)

Method Detail

fromRDD

public static <T> RDDFunctions<T> fromRDD(RDD<T> rdd,
                                          scala.reflect.ClassTag<T> evidence$3)

Implicit conversion from an RDD to RDDFunctions.

sliding

public RDD<Object> sliding(int windowSize)

Returns a RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them. The ordering is first based on the partition index and then the ordering of items within each partition. This is similar to sliding in Scala collections, except that it becomes an empty RDD if the window size is greater than the total number of items. It needs to trigger a Spark job if the parent RDD has more than one partitions and the window size is greater than 1.

Parameters:

windowSize - (undocumented)

Returns:

(undocumented)

treeReduce

public T treeReduce(scala.Function2<T,T,T> f,
                    int depth)

Deprecated. Use RDD.treeReduce(scala.Function2, int) instead.

Reduces the elements of this RDD in a multi-level tree pattern.

Parameters:

depth - suggested depth of the tree (default: 2)

f - (undocumented)

Returns:

(undocumented)

See Also:

RDD.treeReduce(scala.Function2, int)

treeAggregate

public <U> U treeAggregate(U zeroValue,
                           scala.Function2<U,T,U> seqOp,
                           scala.Function2<U,U,U> combOp,
                           int depth,
                           scala.reflect.ClassTag<U> evidence$2)

Deprecated. Use RDD.treeAggregate(U, scala.Function2, scala.Function2, int, scala.reflect.ClassTag) instead.

Aggregates the elements of this RDD in a multi-level tree pattern.

Parameters:

depth - suggested depth of the tree (default: 2)

zeroValue - (undocumented)

seqOp - (undocumented)

combOp - (undocumented)

evidence$2 - (undocumented)

Returns:

(undocumented)

See Also:

RDD.treeAggregate(U, scala.Function2, scala.Function2, int, scala.reflect.ClassTag)