public interface RateEstimator
extends scala.Serializable
InputDStream
should ingest
records, based on updates at every batch completion.
Please see org.apache.spark.streaming.scheduler.RateController
for more details.
Modifier and Type | Method and Description |
---|---|
scala.Option<Object> |
compute(long time,
long elements,
long processingDelay,
long schedulingDelay)
Computes the number of records the stream attached to this
RateEstimator
should ingest per second, given an update on the size and completion
times of the latest batch. |
scala.Option<Object> compute(long time, long elements, long processingDelay, long schedulingDelay)
RateEstimator
should ingest per second, given an update on the size and completion
times of the latest batch.
time
- The timestamp of the current batch interval that just finishedelements
- The number of records that were processed in this batchprocessingDelay
- The time in ms that took for the job to completeschedulingDelay
- The time in ms that the job spent in the scheduling queue