Package org.apache.spark.sql.streaming

Class SourceProgress

Object
org.apache.spark.sql.streaming.SourceProgress
All Implemented Interfaces:
Serializable
public class SourceProgress extends Object implements Serializable
Information about progress made for a source in the execution of a StreamingQuery during a trigger. See StreamingQueryProgress for more information.

param: description Description of the source. param: startOffset The starting offset for data being read. param: endOffset The ending offset for data being read. param: latestOffset The latest offset from this source. param: numInputRows The number of records read from this source. param: inputRowsPerSecond The rate at which data is arriving from this source. param: processedRowsPerSecond The rate at which data from this source is being processed by Spark.

Since:
2.1.0
See Also:

  • Method Details

    • description

      public String description()

    • startOffset

      public String startOffset()

    • endOffset

      public String endOffset()

    • latestOffset

      public String latestOffset()

    • numInputRows

      public long numInputRows()

    • inputRowsPerSecond

      public double inputRowsPerSecond()

    • processedRowsPerSecond

      public double processedRowsPerSecond()

    • metrics

      public Map<String,String> metrics()

    • json

      public String json()
      The compact JSON representation of this progress.

    • prettyJson

      public String prettyJson()
      The pretty (i.e. indented) JSON representation of this progress.

    • toString

      public String toString()
      Overrides:
      toString in class Object