Package org.apache.spark.sql.streaming

Class GroupStateTimeout

Object

org.apache.spark.sql.streaming.GroupStateTimeout

@Evolving
public class GroupStateTimeout
extends Object

Represents the type of timeouts possible for the Dataset operations mapGroupsWithState and flatMapGroupsWithState.

See documentation on GroupState for more details.

Since:

2.2.0

Constructor Summary

Constructors

Constructor	Description
GroupStateTimeout()

Method Summary

Modifier and Type	Method	Description
static GroupStateTimeout	EventTimeTimeout()	Timeout based on event-time.
static GroupStateTimeout	NoTimeout()	No timeout.
static GroupStateTimeout	ProcessingTimeTimeout()	Timeout based on processing time.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

GroupStateTimeout

public GroupStateTimeout()

Method Details

ProcessingTimeTimeout

public static GroupStateTimeout ProcessingTimeTimeout()

Timeout based on processing time.

The duration of timeout can be set for each group in map/flatMapGroupsWithState by calling GroupState.setTimeoutDuration().

See documentation on GroupState for more details.

EventTimeTimeout

public static GroupStateTimeout EventTimeTimeout()

Timeout based on event-time.

The event-time timestamp for timeout can be set for each group in map/flatMapGroupsWithState by calling GroupState.setTimeoutTimestamp(). In addition, you have to define the watermark in the query using Dataset.withWatermark. When the watermark advances beyond the set timestamp of a group and the group has not received any data, then the group times out.

See documentation on GroupState for more details.

NoTimeout

public static GroupStateTimeout NoTimeout()

No timeout.