Class StatsReportListener

Object
org.apache.spark.streaming.scheduler.StatsReportListener
All Implemented Interfaces:
StreamingListener

public class StatsReportListener extends Object implements StreamingListener
:: DeveloperApi :: A simple StreamingListener that logs summary statistics across Spark Streaming batches param: numBatchInfos Number of last batches to consider for generating statistics (default: 10)
  • Constructor Details

    • StatsReportListener

      public StatsReportListener(int numBatchInfos)
  • Method Details

    • batchInfos

      public scala.collection.mutable.Queue<BatchInfo> batchInfos()
    • onBatchCompleted

      public void onBatchCompleted(StreamingListenerBatchCompleted batchStarted)
      Description copied from interface: StreamingListener
      Called when processing of a batch of jobs has completed.
      Specified by:
      onBatchCompleted in interface StreamingListener
    • printStats

      public void printStats()
    • showMillisDistribution

      public void showMillisDistribution(String heading, scala.Function1<BatchInfo,scala.Option<Object>> getMetric)
    • extractDistribution

      public scala.Option<org.apache.spark.util.Distribution> extractDistribution(scala.Function1<BatchInfo,scala.Option<Object>> getMetric)