org.apache.spark.sql.connector.read.streaming

Class Offset

Object

org.apache.spark.sql.connector.read.streaming.Offset

@Evolving
public abstract class Offset
extends Object

An abstract representation of progress through a MicroBatchStream or ContinuousStream.

During execution, offsets provided by the data source implementation will be logged and used as restart checkpoints. Each source should provide an offset implementation which the source can use to reconstruct a position in the stream up to which data has been seen/processed.

Since:

3.0.0

Constructor Summary

Constructors

Constructor and Description
Offset()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object obj) Equality based on JSON string representation.
int	hashCode()
abstract String	json() A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.
String	toString()

Methods inherited from class Object

getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

Offset

public Offset()

Method Detail

json

public abstract String json()

A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.

Note: We assume that equivalent/equal offsets serialize to identical JSON strings.

Returns:

JSON string encoding

equals

public boolean equals(Object obj)

Equality based on JSON string representation. We leverage the JSON representation for normalization between the Offset's in deserialized and serialized representations.

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object