Object
org.apache.spark.sql.connector.read.streaming.Offset

@Evolving public abstract class Offset extends Object
An abstract representation of progress through a MicroBatchStream or ContinuousStream.

During execution, offsets provided by the data source implementation will be logged and used as restart checkpoints. Each source should provide an offset implementation which the source can use to reconstruct a position in the stream up to which data has been seen/processed.

Since:
3.0.0
  • Constructor Summary

    Constructors
    Constructor
    Description
     
  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    Equality based on JSON string representation.
    int
     
    abstract String
    A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.
     

    Methods inherited from class java.lang.Object

    getClass, notify, notifyAll, wait, wait, wait
  • Constructor Details

    • Offset

      public Offset()
  • Method Details

    • json

      public abstract String json()
      A JSON-serialized representation of an Offset that is used for saving offsets to the offset log.

      Note: We assume that equivalent/equal offsets serialize to identical JSON strings.

      Returns:
      JSON string encoding
    • equals

      public boolean equals(Object obj)
      Equality based on JSON string representation. We leverage the JSON representation for normalization between the Offset's in deserialized and serialized representations.
      Overrides:
      equals in class Object
    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object
    • toString

      public String toString()
      Overrides:
      toString in class Object