org.apache.spark

streaming

package streaming

Spark Streaming functionality. org.apache.spark.streaming.StreamingContext serves as the main entry point to Spark Streaming, while org.apache.spark.streaming.dstream.DStream is the data type representing a continuous sequence of RDDs, representing a continuous stream of data.

In addition, org.apache.spark.streaming.dstream.PairDStreamFunctions contains operations available only on DStreams of key-value pairs, such as groupByKey and reduceByKey. These operations are automatically available on any DStream of the right type (e.g. DStream[(Int, Int)] through implicit conversions.

For the Java API of Spark Streaming, take a look at the org.apache.spark.streaming.api.java.JavaStreamingContext which serves as the entry point, and the org.apache.spark.streaming.api.java.JavaDStream and the org.apache.spark.streaming.api.java.JavaPairDStream which have the DStream functionality.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. streaming
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. case class Duration(millis: Long) extends Product with Serializable

  2. class StreamingContext extends Logging

    Main entry point for Spark Streaming functionality.

  3. class StreamingContextState extends Enum[StreamingContextState]

  4. case class Time(millis: Long) extends Product with Serializable

    This is a simple class that represents an absolute instant of time.

Value Members

  1. object Durations

  2. object Milliseconds

    Helper object that creates instance of org.apache.spark.streaming.Duration representing a given number of milliseconds.

  3. object Minutes

    Helper object that creates instance of org.apache.spark.streaming.Duration representing a given number of minutes.

  4. object Seconds

    Helper object that creates instance of org.apache.spark.streaming.Duration representing a given number of seconds.

  5. object StreamingContext extends Logging

    StreamingContext object contains a number of utility functions related to the StreamingContext class.

  6. object Time extends Serializable

  7. package api

  8. package dstream

    Various implementations of DStream's.

  9. package flume

    Spark streaming receiver for Flume.

  10. package kafka

    Kafka receiver for spark streaming,

  11. package kinesis

  12. package mqtt

    MQTT receiver for Spark Streaming.

  13. package receiver

  14. package scheduler

  15. package twitter

    Twitter feed receiver for spark streaming.

  16. package util

  17. package zeromq

    Zeromq receiver for spark streaming.

Inherited from AnyRef

Inherited from Any

Ungrouped