Packages

t

org.apache.spark.sql.streaming

StatefulProcessorHandle

trait StatefulProcessorHandle extends Serializable

Represents the operation handle provided to the stateful processor used in the arbitrary state API v2.

Annotations
@Evolving()
Source
StatefulProcessorHandle.scala
Linear Supertypes
Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StatefulProcessorHandle
  2. Serializable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Abstract Value Members

  1. abstract def deleteIfExists(stateName: String): Unit

    Function to delete and purge state variable if defined previously

    Function to delete and purge state variable if defined previously

    stateName

    \- name of the state variable

  2. abstract def deleteTimer(expiryTimestampMs: Long): Unit

    Function to delete a processing/event time based timer for given implicit grouping key and provided timestamp

    Function to delete a processing/event time based timer for given implicit grouping key and provided timestamp

    expiryTimestampMs

    \- timer expiry timestamp in milliseconds

  3. abstract def getListState[T](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[T]): ListState[T]

    (Scala-specific) Function to create new or return existing list state variable of given type with ttl.

    (Scala-specific) Function to create new or return existing list state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in listState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

    T

    \- type of state variable

    stateName

    \- name of the state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of ListState of type T that can be used to store state persistently

  4. abstract def getListState[T](stateName: String, valEncoder: Encoder[T], ttlConfig: TTLConfig): ListState[T]

    Function to create new or return existing list state variable of given type with ttl.

    Function to create new or return existing list state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in listState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor.

    T

    \- type of state variable

    stateName

    \- name of the state variable

    valEncoder

    \- SQL encoder for state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of ListState of type T that can be used to store state persistently

  5. abstract def getMapState[K, V](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[K], arg1: Encoder[V]): MapState[K, V]

    (Scala-specific) Function to create new or return existing map state variable of given type with ttl.

    (Scala-specific) Function to create new or return existing map state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in mapState which have expired after ttlDuration will not be returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

    K

    \- type of key for map state variable

    V

    \- type of value for map state variable

    stateName

    \- name of the state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of MapState of type [K,V] that can be used to store state persistently

  6. abstract def getMapState[K, V](stateName: String, userKeyEnc: Encoder[K], valEncoder: Encoder[V], ttlConfig: TTLConfig): MapState[K, V]

    Function to create new or return existing map state variable of given type with ttl.

    Function to create new or return existing map state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any values in mapState which have expired after ttlDuration will not returned on get() and will be eventually removed from the state. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor.

    K

    \- type of key for map state variable

    V

    \- type of value for map state variable

    stateName

    \- name of the state variable

    userKeyEnc

    \- spark sql encoder for the map key

    valEncoder

    \- SQL encoder for state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of MapState of type [K,V] that can be used to store state persistently

  7. abstract def getQueryInfo(): QueryInfo

    Function to return query info for the current query

    Function to return query info for the current query

    returns

    - QueryInfo object with access to streaming query metadata

  8. abstract def getValueState[T](stateName: String, ttlConfig: TTLConfig)(implicit arg0: Encoder[T]): ValueState[T]

    (Scala-specific) Function to create new or return existing single value state variable of given type with ttl.

    (Scala-specific) Function to create new or return existing single value state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any state update resets the ttl to current processing time plus ttlDuration. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor. Note that this API uses the implicit SQL encoder in Scala.

    T

    \- type of state variable

    stateName

    \- name of the state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of ValueState of type T that can be used to store state persistently

  9. abstract def getValueState[T](stateName: String, valEncoder: Encoder[T], ttlConfig: TTLConfig): ValueState[T]

    Function to create new or return existing single value state variable of given type with ttl.

    Function to create new or return existing single value state variable of given type with ttl. State values will not be returned past ttlDuration, and will be eventually removed from the state store. Any state update resets the ttl to current processing time plus ttlDuration. Users can use the helper method TTLConfig.NONE in Scala or TTLConfig.NONE() in Java for the TTLConfig parameter to disable TTL for the state variable.

    The user must ensure to call this function only within the init() method of the StatefulProcessor.

    T

    \- type of state variable

    stateName

    \- name of the state variable

    valEncoder

    \- SQL encoder for state variable

    ttlConfig

    \- the ttl configuration (time to live duration etc.)

    returns

    \- instance of ValueState of type T that can be used to store state persistently

  10. abstract def listTimers(): Iterator[Long]

    Function to list all the timers registered for given implicit grouping key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.

    Function to list all the timers registered for given implicit grouping key Note: calling listTimers() within the handleInputRows method of the StatefulProcessor will return all the unprocessed registered timers, including the one being fired within the invocation of handleInputRows.

    returns

    \- list of all the registered timers for given implicit grouping key

  11. abstract def registerTimer(expiryTimestampMs: Long): Unit

    Function to register a processing/event time based timer for given implicit grouping key and provided timestamp

    Function to register a processing/event time based timer for given implicit grouping key and provided timestamp

    expiryTimestampMs

    \- timer expiry timestamp in milliseconds

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  8. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  9. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  10. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  11. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  13. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  14. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  15. def toString(): String
    Definition Classes
    AnyRef → Any
  16. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  17. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  18. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped