public interface RateEstimator extends scala.Serializable
InputDStreamshould ingest records, based on updates at every batch completion.
org.apache.spark.streaming.scheduler.RateController for more details.
|Modifier and Type||Method and Description|
Computes the number of records the stream attached to this
scala.Option<Object> compute(long time, long elements, long processingDelay, long schedulingDelay)
RateEstimatorshould ingest per second, given an update on the size and completion times of the latest batch.
time- The timestamp of the current batch interval that just finished
elements- The number of records that were processed in this batch
processingDelay- The time in ms that took for the job to complete
schedulingDelay- The time in ms that the job spent in the scheduling queue