Package org.apache.spark.mllib.rdd

Class RDDFunctions<T>

Object

org.apache.spark.mllib.rdd.RDDFunctions<T>

All Implemented Interfaces:

Serializable

public class RDDFunctions<T>
extends Object
implements Serializable

Machine learning specific RDD functions.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
RDDFunctions(RDD<T> self, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Modifier and Type	Method	Description
static <T> RDDFunctions<T>	fromRDD(RDD<T> rdd, scala.reflect.ClassTag<T> evidence$2)	Implicit conversion from an RDD to RDDFunctions.
RDD<Object>	sliding(int windowSize)	sliding(Int, Int)* with step = 1.
RDD<Object>	sliding(int windowSize, int step)	Returns an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RDDFunctions

public RDDFunctions(RDD<T> self,
 scala.reflect.ClassTag<T> evidence$1)

Method Details

fromRDD

public static <T> RDDFunctions<T> fromRDD(RDD<T> rdd,
 scala.reflect.ClassTag<T> evidence$2)

Implicit conversion from an RDD to RDDFunctions.

sliding

public RDD<Object> sliding(int windowSize,
 int step)

Returns an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them. The ordering is first based on the partition index and then the ordering of items within each partition. This is similar to sliding in Scala collections, except that it becomes an empty RDD if the window size is greater than the total number of items. It needs to trigger a Spark job if the parent RDD has more than one partitions and the window size is greater than 1.

Parameters:

windowSize - (undocumented)

step - (undocumented)

Returns:

(undocumented)

sliding

public RDD<Object> sliding(int windowSize)

sliding(Int, Int)* with step = 1.

Parameters:

windowSize - (undocumented)

Returns:

(undocumented)