Packages

package java

Spark streaming's Java API.

Source
package.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. java
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class JavaDStream[T] extends AbstractJavaDStreamLike[T, JavaDStream[T], JavaRDD[T]]

    A Java-friendly interface to org.apache.spark.streaming.dstream.DStream, the basic abstraction in Spark Streaming that represents a continuous stream of data.

    A Java-friendly interface to org.apache.spark.streaming.dstream.DStream, the basic abstraction in Spark Streaming that represents a continuous stream of data. DStreams can either be created from live data (such as, data from TCP sockets, Kafka, etc.) or it can be generated by transforming existing DStreams using operations such as map, window. For operations applicable to key-value pair DStreams, see org.apache.spark.streaming.api.java.JavaPairDStream.

  2. trait JavaDStreamLike[T, This <: JavaDStreamLike[T, This, R], R <: JavaRDDLike[T, R]] extends Serializable
  3. class JavaInputDStream[T] extends JavaDStream[T]

    A Java-friendly interface to org.apache.spark.streaming.dstream.InputDStream.

  4. class JavaMapWithStateDStream[KeyType, ValueType, StateType, MappedType] extends JavaDStream[MappedType]

    DStream representing the stream of data generated by mapWithState operation on a JavaPairDStream.

    DStream representing the stream of data generated by mapWithState operation on a JavaPairDStream. Additionally, it also gives access to the stream of state snapshots, that is, the state data of all keys after a batch has updated them.

    KeyType

    Class of the keys

    ValueType

    Class of the values

    StateType

    Class of the state data

    MappedType

    Class of the mapped data

  5. class JavaPairDStream[K, V] extends AbstractJavaDStreamLike[(K, V), JavaPairDStream[K, V], JavaPairRDD[K, V]]

    A Java-friendly interface to a DStream of key-value pairs, which provides extra methods like reduceByKey and join.

  6. class JavaPairInputDStream[K, V] extends JavaPairDStream[K, V]

    A Java-friendly interface to org.apache.spark.streaming.dstream.InputDStream of key-value pairs.

  7. class JavaPairReceiverInputDStream[K, V] extends JavaPairInputDStream[K, V]

    A Java-friendly interface to org.apache.spark.streaming.dstream.ReceiverInputDStream, the abstract class for defining any input stream that receives data over the network.

  8. class JavaReceiverInputDStream[T] extends JavaInputDStream[T]

    A Java-friendly interface to org.apache.spark.streaming.dstream.ReceiverInputDStream, the abstract class for defining any input stream that receives data over the network.

  9. class JavaStreamingContext extends Closeable

    A Java-friendly version of org.apache.spark.streaming.StreamingContext which is the main entry point for Spark Streaming functionality.

    A Java-friendly version of org.apache.spark.streaming.StreamingContext which is the main entry point for Spark Streaming functionality. It provides methods to create org.apache.spark.streaming.api.java.JavaDStream and org.apache.spark.streaming.api.java.JavaPairDStream from input sources. The internal org.apache.spark.api.java.JavaSparkContext (see core Spark documentation) can be accessed using context.sparkContext. After creating and transforming DStreams, the streaming computation can be started and stopped using context.start() and context.stop(), respectively. context.awaitTermination() allows the current thread to wait for the termination of a context by stop() or by an exception.

    Annotations
    @deprecated
    Deprecated

    (Since version Spark 3.4.0) DStream is deprecated. Migrate to Structured Streaming.

Value Members

  1. object JavaDStream extends Serializable
  2. object JavaInputDStream extends Serializable
  3. object JavaPairDStream extends Serializable
  4. object JavaPairInputDStream extends Serializable
  5. object JavaPairReceiverInputDStream extends Serializable
  6. object JavaReceiverInputDStream extends Serializable
  7. object JavaStreamingContext

    JavaStreamingContext object contains a number of utility functions.

Inherited from AnyRef

Inherited from Any

Ungrouped