org.apache.spark.api.r
Class RRDD<T>

Object
  org.apache.spark.rdd.RDD<U>
      org.apache.spark.api.r.BaseRRDD<T,byte[]>
          org.apache.spark.api.r.RRDD<T>

All Implemented Interfaces:

java.io.Serializable, Logging

public class RRDD<T>
extends BaseRRDD<T,byte[]>

An RDD that stores serialized R objects as Array[Byte].

See Also:

Serialized Form

Constructor Summary

RRDD(RDD<T> parent, byte[] func, String deserializer, String serializer, byte[] packageNames, String rLibDir, Object[] broadcastVars, scala.reflect.ClassTag<T> evidence$4)

Method Summary

JavaRDD<byte[]>	asJavaRDD()
static JavaRDD<byte[]>	createRDDFromArray(JavaSparkContext jsc, byte[][] arr) Create an RRDD given a sequence of byte arrays.
static org.apache.spark.api.r.BufferedStreamThread	createRWorker(String rLibDir, int port) ProcessBuilder used to launch worker R processes.
static JavaSparkContext	createSparkContext(String master, String appName, String sparkHome, String[] jars, java.util.Map<Object,Object> sparkEnvirMap, java.util.Map<Object,Object> sparkExecutorEnvMap)

Methods inherited from class org.apache.spark.api.r.BaseRRDD

compute, getPartitions

Methods inherited from class org.apache.spark.rdd.RDD

aggregate, cache, cartesian, checkpoint, checkpointData, coalesce, collect, collect, context, count, countApprox, countApproxDistinct, countApproxDistinct, countByValue, countByValueApprox, creationSite, dependencies, distinct, distinct, doubleRDDToDoubleRDDFunctions, filter, filterWith, first, flatMap, flatMapWith, fold, foreach, foreachPartition, foreachWith, getCheckpointFile, getStorageLevel, glom, groupBy, groupBy, groupBy, id, intersection, intersection, intersection, isCheckpointed, isEmpty, iterator, keyBy, map, mapPartitions, mapPartitionsWithContext, mapPartitionsWithIndex, mapPartitionsWithSplit, mapWith, max, min, name, numericRDDToDoubleRDDFunctions, partitioner, partitions, persist, persist, pipe, pipe, pipe, preferredLocations, randomSplit, rddToAsyncRDDActions, rddToOrderedRDDFunctions, rddToPairRDDFunctions, rddToSequenceFileRDDFunctions, reduce, repartition, sample, saveAsObjectFile, saveAsTextFile, saveAsTextFile, scope, setName, sortBy, sparkContext, subtract, subtract, subtract, take, takeOrdered, takeSample, toArray, toDebugString, toJavaRDD, toLocalIterator, top, toString, treeAggregate, treeReduce, union, unpersist, zip, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipPartitions, zipWithIndex, zipWithUniqueId

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

RRDD

public RRDD(RDD<T> parent,
            byte[] func,
            String deserializer,
            String serializer,
            byte[] packageNames,
            String rLibDir,
            Object[] broadcastVars,
            scala.reflect.ClassTag<T> evidence$4)

Method Detail

createSparkContext

public static JavaSparkContext createSparkContext(String master,
                                                  String appName,
                                                  String sparkHome,
                                                  String[] jars,
                                                  java.util.Map<Object,Object> sparkEnvirMap,
                                                  java.util.Map<Object,Object> sparkExecutorEnvMap)

createRWorker

public static org.apache.spark.api.r.BufferedStreamThread createRWorker(String rLibDir,
                                                                        int port)

ProcessBuilder used to launch worker R processes.

Parameters:

rLibDir - (undocumented)

port - (undocumented)

Returns:

(undocumented)

createRDDFromArray

public static JavaRDD<byte[]> createRDDFromArray(JavaSparkContext jsc,
                                                 byte[][] arr)

Create an RRDD given a sequence of byte arrays. Used to create RRDD when parallelize is called from R.

Parameters:

jsc - (undocumented)

arr - (undocumented)

Returns:

(undocumented)

asJavaRDD

public JavaRDD<byte[]> asJavaRDD()