org.apache.spark.streaming.dstream

Class StateDStream<K,V,S>

Object

org.apache.spark.streaming.dstream.DStream<scala.Tuple2<K,S>>

org.apache.spark.streaming.dstream.StateDStream<K,V,S>

All Implemented Interfaces:

java.io.Serializable, Logging

public class StateDStream<K,V,S>
extends DStream<scala.Tuple2<K,S>>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StateDStream(DStream<scala.Tuple2<K,V>> parent, scala.Function1<scala.collection.Iterator<scala.Tuple3<K,scala.collection.Seq<V>,scala.Option<S>>>,scala.collection.Iterator<scala.Tuple2<K,S>>> updateFunc, Partitioner partitioner, boolean preservePartitioning, scala.reflect.ClassTag<K> evidence$1, scala.reflect.ClassTag<V> evidence$2, scala.reflect.ClassTag<S> evidence$3)

Method Summary

Methods

Modifier and Type	Method and Description
scala.Option<RDD<scala.Tuple2<K,S>>>	compute(Time validTime) Method that generates a RDD for the given time
scala.collection.immutable.List<DStream<scala.Tuple2<K,V>>>	dependencies()
boolean	mustCheckpoint()
Duration	slideDuration() Time interval after which the DStream generates a RDD

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, checkpointData, checkpointDuration, clearCheckpointData, clearMetadata, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, generateJob, getCreationSite, getOrCompute, glom, graph, initialize, isInitialized, isTimeValid, map, mapPartitions, parentRememberDuration, persist, persist, print, reduce, reduceByWindow, reduceByWindow, register, remember, rememberDuration, repartition, restoreCheckpointData, saveAsObjectFiles, saveAsTextFiles, setContext, setGraph, slice, slice, ssc, storageLevel, transform, transform, transformWith, transformWith, union, updateCheckpointData, validate, window, window, zeroTime

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

StateDStream

public StateDStream(DStream<scala.Tuple2<K,V>> parent,
            scala.Function1<scala.collection.Iterator<scala.Tuple3<K,scala.collection.Seq<V>,scala.Option<S>>>,scala.collection.Iterator<scala.Tuple2<K,S>>> updateFunc,
            Partitioner partitioner,
            boolean preservePartitioning,
            scala.reflect.ClassTag<K> evidence$1,
            scala.reflect.ClassTag<V> evidence$2,
            scala.reflect.ClassTag<S> evidence$3)

Method Detail

dependencies

public scala.collection.immutable.List<DStream<scala.Tuple2<K,V>>> dependencies()

slideDuration

public Duration slideDuration()

Description copied from class: DStream

Time interval after which the DStream generates a RDD

Specified by:

slideDuration in class DStream<scala.Tuple2<K,S>>

mustCheckpoint

public boolean mustCheckpoint()

Overrides:

mustCheckpoint in class DStream<scala.Tuple2<K,S>>

compute

public scala.Option<RDD<scala.Tuple2<K,S>>> compute(Time validTime)

Description copied from class: DStream

Method that generates a RDD for the given time

Specified by:

compute in class DStream<scala.Tuple2<K,S>>