@Evolving public interface SupportsAdmissionControl extends SparkDataStream
SparkDataStreamstreaming sources to signal that they can control the rate of data ingested into the system. These rate limits can come implicitly from the contract of triggers, e.g. Trigger.Once() requires that a micro-batch process all data available to the system at the start of the micro-batch. Alternatively, sources can decide to limit ingest through data source options.
Through this interface, a MicroBatchStream should be able to return the next offset that it will
process until given a
|Modifier and Type||Method and Description|
Returns the read limits potentially passed to the data source through options when creating the data source.
Returns the most recent offset available given a read limit.
Returns the most recent offset available.
commit, deserializeOffset, initialOffset, stop
default ReadLimit getDefaultReadLimit()
Offset latestOffset(Offset startOffset, ReadLimit limit)
When this method is called on a `Source`, the source can return `null` if there is no data to process. In addition, for the very first micro-batch, the `startOffset` will be null as well.
When this method is called on a MicroBatchStream, the `startOffset` will be `initialOffset` for the very first micro-batch. The source can return `null` if there is no data to process.
default Offset reportLatestOffset()
The source can return `null`, if there is no data to process or the source does not support to this method.