org.apache.spark.streaming.zeromq

Class ZeroMQUtils

Object

org.apache.spark.streaming.zeromq.ZeroMQUtils

public class ZeroMQUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
ZeroMQUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects, StorageLevel storageLevel) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects, StorageLevel storageLevel, akka.actor.SupervisorStrategy supervisorStrategy) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> ReceiverInputDStream<T>	createStream(StreamingContext ssc, String publisherUrl, akka.zeromq.Subscribe subscribe, scala.Function1<scala.collection.Seq<akka.util.ByteString>,scala.collection.Iterator<T>> bytesToObjects, StorageLevel storageLevel, akka.actor.SupervisorStrategy supervisorStrategy, scala.reflect.ClassTag<T> evidence$1) Create an input stream that receives messages pushed by a zeromq publisher.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZeroMQUtils

public ZeroMQUtils()

Method Detail

createStream

public static <T> ReceiverInputDStream<T> createStream(StreamingContext ssc,
                                       String publisherUrl,
                                       akka.zeromq.Subscribe subscribe,
                                       scala.Function1<scala.collection.Seq<akka.util.ByteString>,scala.collection.Iterator<T>> bytesToObjects,
                                       StorageLevel storageLevel,
                                       akka.actor.SupervisorStrategy supervisorStrategy,
                                       scala.reflect.ClassTag<T> evidence$1)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

ssc - StreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter (which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - RDD storage level. Defaults to StorageLevel.MEMORY_AND_DISK_SER_2.

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                           String publisherUrl,
                                           akka.zeromq.Subscribe subscribe,
                                           Function<byte[][],Iterable<T>> bytesToObjects,
                                           StorageLevel storageLevel,
                                           akka.actor.SupervisorStrategy supervisorStrategy)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote ZeroMQ publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - Storage level to use for storing the received objects

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                           String publisherUrl,
                                           akka.zeromq.Subscribe subscribe,
                                           Function<byte[][],Iterable<T>> bytesToObjects,
                                           StorageLevel storageLevel)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - RDD storage level.

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                           String publisherUrl,
                                           akka.zeromq.Subscribe subscribe,
                                           Function<byte[][],Iterable<T>> bytesToObjects)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.