Class MapWithStateDStream<KeyType,ValueType,StateType,MappedType>

Object
org.apache.spark.streaming.dstream.DStream<MappedType>
org.apache.spark.streaming.dstream.MapWithStateDStream<KeyType,ValueType,StateType,MappedType>
Type Parameters:
KeyType - Class of the key
ValueType - Class of the value
StateType - Class of the state data
MappedType - Class of the mapped data
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, scala.Serializable

public abstract class MapWithStateDStream<KeyType,ValueType,StateType,MappedType> extends DStream<MappedType>
DStream representing the stream of data generated by mapWithState operation on a pair DStream. Additionally, it also gives access to the stream of state snapshots, that is, the state data of all keys after a batch has updated them.

See Also:
  • Constructor Details

  • Method Details

    • stateSnapshots

      public abstract DStream<scala.Tuple2<KeyType,StateType>> stateSnapshots()
      Return a pair DStream where each RDD is the snapshot of the state of all the keys.