org.apache.spark.streaming.zeromq
Class ZeroMQUtils

Object
  org.apache.spark.streaming.zeromq.ZeroMQUtils

public class ZeroMQUtils
extends Object

Constructor Summary

ZeroMQUtils()

Method Summary

static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects, StorageLevel storageLevel) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> JavaReceiverInputDStream<T>	createStream(JavaStreamingContext jssc, String publisherUrl, akka.zeromq.Subscribe subscribe, Function<byte[][],Iterable<T>> bytesToObjects, StorageLevel storageLevel, akka.actor.SupervisorStrategy supervisorStrategy) Create an input stream that receives messages pushed by a zeromq publisher.
static <T> ReceiverInputDStream<T>	createStream(StreamingContext ssc, String publisherUrl, akka.zeromq.Subscribe subscribe, scala.Function1<scala.collection.Seq<akka.util.ByteString>,scala.collection.Iterator<T>> bytesToObjects, StorageLevel storageLevel, akka.actor.SupervisorStrategy supervisorStrategy, scala.reflect.ClassTag<T> evidence$1) Create an input stream that receives messages pushed by a zeromq publisher.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZeroMQUtils

public ZeroMQUtils()

Method Detail

createStream

public static <T> ReceiverInputDStream<T> createStream(StreamingContext ssc,
                                                       String publisherUrl,
                                                       akka.zeromq.Subscribe subscribe,
                                                       scala.Function1<scala.collection.Seq<akka.util.ByteString>,scala.collection.Iterator<T>> bytesToObjects,
                                                       StorageLevel storageLevel,
                                                       akka.actor.SupervisorStrategy supervisorStrategy,
                                                       scala.reflect.ClassTag<T> evidence$1)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

ssc - StreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter (which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - RDD storage level. Defaults to StorageLevel.MEMORY_AND_DISK_SER_2.

supervisorStrategy - (undocumented)

evidence$1 - (undocumented)

Returns:

(undocumented)

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                                           String publisherUrl,
                                                           akka.zeromq.Subscribe subscribe,
                                                           Function<byte[][],Iterable<T>> bytesToObjects,
                                                           StorageLevel storageLevel,
                                                           akka.actor.SupervisorStrategy supervisorStrategy)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote ZeroMQ publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - Storage level to use for storing the received objects

supervisorStrategy - (undocumented)

Returns:

(undocumented)

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                                           String publisherUrl,
                                                           akka.zeromq.Subscribe subscribe,
                                                           Function<byte[][],Iterable<T>> bytesToObjects,
                                                           StorageLevel storageLevel)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

storageLevel - RDD storage level.

Returns:

(undocumented)

createStream

public static <T> JavaReceiverInputDStream<T> createStream(JavaStreamingContext jssc,
                                                           String publisherUrl,
                                                           akka.zeromq.Subscribe subscribe,
                                                           Function<byte[][],Iterable<T>> bytesToObjects)

Create an input stream that receives messages pushed by a zeromq publisher.

Parameters:

jssc - JavaStreamingContext object

publisherUrl - Url of remote zeromq publisher

subscribe - Topic to subscribe to

bytesToObjects - A zeroMQ stream publishes sequence of frames for each topic and each frame has sequence of byte thus it needs the converter(which might be deserializer of bytes) to translate from sequence of sequence of bytes, where sequence refer to a frame and sub sequence refer to its payload.

Returns:

(undocumented)