Package org.apache.spark.streaming.dstream

Class ReceiverInputDStream<T>

Object

org.apache.spark.streaming.dstream.DStream<T>

org.apache.spark.streaming.dstream.InputDStream<T>

org.apache.spark.streaming.dstream.ReceiverInputDStream<T>

Type Parameters:

T - Class type of the object of this stream

All Implemented Interfaces:

Serializable, org.apache.spark.internal.Logging

public abstract class ReceiverInputDStream<T>
extends InputDStream<T>

Abstract class for defining any InputDStream that has to start a receiver on worker nodes to receive external data. Specific implementations of ReceiverInputDStream must define getReceiver() function that gets the receiver object of type Receiver that will be sent to the workers to receive data. param: _ssc Streaming context that will execute this input stream

See Also:

• Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

Constructor Summary

Constructors

Constructor	Description
ReceiverInputDStream(StreamingContext _ssc, scala.reflect.ClassTag<T> evidence$1)

Method Summary

Modifier and Type	Method	Description
scala.Option<RDD<T>>	compute(Time validTime)	Method that generates an RDD for the given time
abstract Receiver<T>	getReceiver()	Gets the receiver object that will be sent to the worker nodes to receive data.
void	start()	Method called to start receiving data.
void	stop()	Method called to stop receiving data.

Methods inherited from class org.apache.spark.streaming.dstream.InputDStream

dependencies, id, slideDuration

Methods inherited from class org.apache.spark.streaming.dstream.DStream

cache, checkpoint, context, count, countByValue, countByValueAndWindow, countByWindow, filter, flatMap, foreachRDD, foreachRDD, glom, map, mapPartitions, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, repartition, saveAsObjectFiles, saveAsTextFiles, slice, slice, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, window, window

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

Constructor Details

ReceiverInputDStream

public ReceiverInputDStream(StreamingContext _ssc,
 scala.reflect.ClassTag<T> evidence$1)

Method Details

compute

public scala.Option<RDD<T>> compute(Time validTime)

Description copied from class: DStream

Method that generates an RDD for the given time

Specified by:

compute in class DStream<T>

Parameters:

validTime - (undocumented)

Returns:

(undocumented) Generates RDDs with blocks received by the receiver of this stream.

getReceiver

public abstract Receiver<T> getReceiver()

Gets the receiver object that will be sent to the worker nodes to receive data. This method needs to defined by any specific implementation of a ReceiverInputDStream.

Returns:

(undocumented)

start

public void start()

Description copied from class: InputDStream

Method called to start receiving data. Subclasses must implement this method.

Specified by:

start in class InputDStream<T>

stop

public void stop()

Description copied from class: InputDStream

Method called to stop receiving data. Subclasses must implement this method.

Specified by:

stop in class InputDStream<T>