org.apache.spark.sql.streaming

Class GroupStateTimeout

Object

org.apache.spark.sql.streaming.GroupStateTimeout

@Experimental
 @Evolving
public class GroupStateTimeout
extends Object

Represents the type of timeouts possible for the Dataset operations mapGroupsWithState and flatMapGroupsWithState.

See documentation on GroupState for more details.

Since:

2.2.0

Constructor Summary

Constructors

Constructor and Description
GroupStateTimeout()

Method Summary

Modifier and Type	Method and Description
static GroupStateTimeout	EventTimeTimeout() Timeout based on event-time.
static GroupStateTimeout	NoTimeout() No timeout.
static GroupStateTimeout	ProcessingTimeTimeout() Timeout based on processing time.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GroupStateTimeout

public GroupStateTimeout()

Method Detail

ProcessingTimeTimeout

public static GroupStateTimeout ProcessingTimeTimeout()

Timeout based on processing time.

The duration of timeout can be set for each group in map/flatMapGroupsWithState by calling GroupState.setTimeoutDuration().

See documentation on GroupState for more details.

EventTimeTimeout

public static GroupStateTimeout EventTimeTimeout()

Timeout based on event-time.

The event-time timestamp for timeout can be set for each group in map/flatMapGroupsWithState by calling GroupState.setTimeoutTimestamp(). In addition, you have to define the watermark in the query using Dataset.withWatermark. When the watermark advances beyond the set timestamp of a group and the group has not received any data, then the group times out.

See documentation on GroupState for more details.

NoTimeout

public static GroupStateTimeout NoTimeout()

No timeout.