Package org.apache.spark.sql.streaming

Interface StatefulProcessorHandle

All Superinterfaces:
Serializable
public interface StatefulProcessorHandle extends Serializable
Represents the operation handle provided to the stateful processor used in the arbitrary state API v2.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    deleteIfExists(String stateName)
    Function to delete and purge state variable if defined previously
    void
    deleteTimer(long expiryTimestampMs)
    Function to delete a processing/event time based timer for given implicit grouping key and provided timestamp
    <T> ListState<T>
    getListState(String stateName, Encoder<T> valEncoder, TTLConfig ttlConfig)
    Function to create new or return existing list state variable of given type with ttl.
    <T> ListState<T>
    getListState(String stateName, TTLConfig ttlConfig, Encoder<T> evidence$2)
    (Scala-specific) Function to create new or return existing list state variable of given type with ttl.
    <K, V> MapState<K,V>
    getMapState(String stateName, Encoder<K> userKeyEnc, Encoder<V> valEncoder, TTLConfig ttlConfig)
    Function to create new or return existing map state variable of given type with ttl.
    <K, V> MapState<K,V>
    getMapState(String stateName, TTLConfig ttlConfig, Encoder<K> evidence$3, Encoder<V> evidence$4)
    (Scala-specific) Function to create new or return existing map state variable of given type with ttl.
    QueryInfo
    getQueryInfo()
    Function to return query info for the current query
    <T> ValueState<T>
    getValueState(String stateName, Encoder<T> valEncoder, TTLConfig ttlConfig)
    Function to create new or return existing single value state variable of given type with ttl.
    <T> ValueState<T>
    getValueState(String stateName, TTLConfig ttlConfig, Encoder<T> evidence$1)
    (Scala-specific) Function to create new or return existing single value state variable of given type with ttl.
    scala.collection.Iterator<Object>
    listTimers()
    Function to list all the timers registered for given implicit grouping key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.
    void
    registerTimer(long expiryTimestampMs)
    Function to register a processing/event time based timer for given implicit grouping key and provided timestamp

  • Method Details

    • deleteIfExists

      void deleteIfExists(String stateName)
      Function to delete and purge state variable if defined previously

      Parameters:
      stateName - \- name of the state variable

    • deleteTimer

      void deleteTimer(long expiryTimestampMs)
      Function to delete a processing/event time based timer for given implicit grouping key and provided timestamp

      Parameters:
      expiryTimestampMs - \- timer expiry timestamp in milliseconds

    • getListState

      <T> ListState<T> getListState(String stateName, Encoder<T> valEncoder, TTLConfig ttlConfig)
      Function to create new or return existing list state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in listState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor.

      Parameters:
      stateName - \- name of the state variable
      valEncoder - \- SQL encoder for state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      Returns:
      \- instance of ListState of type T that can be used to store state persistently

    • getListState

      <T> ListState<T> getListState(String stateName, TTLConfig ttlConfig, Encoder<T> evidence$2)
      (Scala-specific) Function to create new or return existing list state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in listState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

      Parameters:
      stateName - \- name of the state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      evidence$2 - (undocumented)
      Returns:
      \- instance of ListState of type T that can be used to store state persistently

    • getMapState

      <K, V> MapState<K,V> getMapState(String stateName, Encoder<K> userKeyEnc, Encoder<V> valEncoder, TTLConfig ttlConfig)
      Function to create new or return existing map state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in mapState which have expired after ttlDuration will not returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor.

      Parameters:
      stateName - \- name of the state variable
      userKeyEnc - \- spark sql encoder for the map key
      valEncoder - \- SQL encoder for state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      Returns:
      \- instance of MapState of type [K,V] that can be used to store state persistently

    • getMapState

      <K, V> MapState<K,V> getMapState(String stateName, TTLConfig ttlConfig, Encoder<K> evidence$3, Encoder<V> evidence$4)
      (Scala-specific) Function to create new or return existing map state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in mapState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

      Parameters:
      stateName - \- name of the state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      evidence$3 - (undocumented)
      evidence$4 - (undocumented)
      Returns:
      \- instance of MapState of type [K,V] that can be used to store state persistently

    • getQueryInfo

      QueryInfo getQueryInfo()
      Function to return query info for the current query

      Returns:
      - QueryInfo object with access to streaming query metadata

    • getValueState

      <T> ValueState<T> getValueState(String stateName, Encoder<T> valEncoder, TTLConfig ttlConfig)
      Function to create new or return existing single value state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any state update resets the ttl to current processing time plus ttlDuration. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor.

      Parameters:
      stateName - \- name of the state variable
      valEncoder - \- SQL encoder for state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      Returns:
      \- instance of ValueState of type T that can be used to store state persistently

    • getValueState

      <T> ValueState<T> getValueState(String stateName, TTLConfig ttlConfig, Encoder<T> evidence$1)
      (Scala-specific) Function to create new or return existing single value state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any state update resets the ttl to current processing time plus ttlDuration. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

      The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

      Parameters:
      stateName - \- name of the state variable
      ttlConfig - \- the ttl configuration (time to live duration etc.)
      evidence$1 - (undocumented)
      Returns:
      \- instance of ValueState of type T that can be used to store state persistently

    • listTimers

      scala.collection.Iterator<Object> listTimers()
      Function to list all the timers registered for given implicit grouping key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.

      Returns:
      \- list of all the registered timers for given implicit grouping key

    • registerTimer

      void registerTimer(long expiryTimestampMs)
      Function to register a processing/event time based timer for given implicit grouping key and provided timestamp

      Parameters:
      expiryTimestampMs - \- timer expiry timestamp in milliseconds