Package org.apache.spark.sql.connector.read.streaming

Class Offset

Object
org.apache.spark.sql.connector.read.streaming.Offset
@Evolving public abstract class Offset extends Object
An abstract representation of progress through a MicroBatchStream or ContinuousStream.

During execution, offsets provided by the data source implementation will be logged and used as restart checkpoints. Each source should provide an offset implementation which the source can use to reconstruct a position in the stream up to which data has been seen/processed.

Since:
3.0.0

  • Constructor Summary

    Constructors
    Constructor
    Description
    Offset()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    equals(Object obj)
    Equality based on JSON string representation.
    int
    hashCode()
     
    abstract String
    json()
    A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.
    String
    toString()
     

    Methods inherited from class java.lang.Object

    getClass, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • Offset

      public Offset()

  • Method Details

    • json

      public abstract String json()
      A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.

      Note: We assume that equivalent/equal offsets serialize to identical JSON strings.

      Returns:
      JSON string encoding

    • equals

      public boolean equals(Object obj)
      Equality based on JSON string representation. We leverage the JSON representation for normalization between the Offset's in deserialized and serialized representations.
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object