Packages

case class BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo]) extends Product with Serializable

Developer API

Class having information on completed batches.

batchTime

Time of the batch

streamIdToInputInfo

A map of input stream id to its input info

submissionTime

Clock time of when jobs of this batch was submitted to the streaming scheduler queue

processingStartTime

Clock time of when the first job of this batch started processing

processingEndTime

Clock time of when the last job of this batch finished processing

outputOperationInfos

The output operations in this batch

Annotations
@DeveloperApi()
Source
BatchInfo.scala
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. BatchInfo
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo])

    batchTime

    Time of the batch

    streamIdToInputInfo

    A map of input stream id to its input info

    submissionTime

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

    processingStartTime

    Clock time of when the first job of this batch started processing

    processingEndTime

    Clock time of when the last job of this batch finished processing

    outputOperationInfos

    The output operations in this batch

Value Members

  1. val batchTime: Time
  2. def numRecords: Long

    The number of recorders received by the receivers in this batch.

  3. val outputOperationInfos: Map[Int, OutputOperationInfo]
  4. def processingDelay: Option[Long]

    Time taken for the all jobs of this batch to finish processing from the time they started processing.

    Time taken for the all jobs of this batch to finish processing from the time they started processing. Essentially, it is processingEndTime - processingStartTime.

  5. val processingEndTime: Option[Long]
  6. val processingStartTime: Option[Long]
  7. def schedulingDelay: Option[Long]

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler.

    Time taken for the first job of this batch to start processing from the time this batch was submitted to the streaming scheduler. Essentially, it is processingStartTime - submissionTime.

  8. val streamIdToInputInfo: Map[Int, StreamInputInfo]
  9. val submissionTime: Long
  10. def totalDelay: Option[Long]

    Time taken for all the jobs of this batch to finish processing from the time they were submitted.

    Time taken for all the jobs of this batch to finish processing from the time they were submitted. Essentially, it is processingDelay + schedulingDelay.