Class BatchInfo

Object
org.apache.spark.streaming.scheduler.BatchInfo
All Implemented Interfaces:
Serializable, scala.Equals, scala.Product

public class BatchInfo extends Object implements scala.Product, Serializable
:: DeveloperApi :: Class having information on completed batches. param: batchTime Time of the batch param: streamIdToInputInfo A map of input stream id to its input info param: submissionTime Clock time of when jobs of this batch was submitted to the streaming scheduler queue param: processingStartTime Clock time of when the first job of this batch started processing param: processingEndTime Clock time of when the last job of this batch finished processing param: outputOperationInfos The output operations in this batch
See Also:
  • Constructor Details

    • BatchInfo

      public BatchInfo(Time batchTime, scala.collection.immutable.Map<Object,StreamInputInfo> streamIdToInputInfo, long submissionTime, scala.Option<Object> processingStartTime, scala.Option<Object> processingEndTime, scala.collection.immutable.Map<Object,OutputOperationInfo> outputOperationInfos)
  • Method Details

    • apply

      public abstract static R apply(T1 v1, T2 v2, T3 v3, T4 v4, T5 v5, T6 v6)
    • toString

      public static String toString()
    • batchTime

      public Time batchTime()
    • streamIdToInputInfo

      public scala.collection.immutable.Map<Object,StreamInputInfo> streamIdToInputInfo()
    • submissionTime

      public long submissionTime()
    • processingStartTime

      public scala.Option<Object> processingStartTime()
    • processingEndTime

      public scala.Option<Object> processingEndTime()
    • outputOperationInfos

      public scala.collection.immutable.Map<Object,OutputOperationInfo> outputOperationInfos()
    • schedulingDelay

      public scala.Option<Object> schedulingDelay()
      Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler. Essentially, it is processingStartTime - submissionTime.
      Returns:
      (undocumented)
    • processingDelay

      public scala.Option<Object> processingDelay()
      Time taken for the all jobs of this batch to finish processing from the time they started processing. Essentially, it is processingEndTime - processingStartTime.
      Returns:
      (undocumented)
    • totalDelay

      public scala.Option<Object> totalDelay()
      Time taken for all the jobs of this batch to finish processing from the time they were submitted. Essentially, it is processingDelay + schedulingDelay.
      Returns:
      (undocumented)
    • numRecords

      public long numRecords()
      The number of recorders received by the receivers in this batch.
      Returns:
      (undocumented)