@Evolving public interface CustomTaskMetric
PartitionReaderand update internal metrics based on collected metric values. For streaming query, Spark will collect and combine metrics for a final result per micro batch.
The metrics will be gathered during query execution back to the driver and then combined. How
the task metrics are combined is defined by corresponding
CustomMetric with same metric
name. The final result will be shown up in the data source scan operator in Spark UI.
There are a few special metric names: "bytesWritten" and "recordsWritten". If the data source defines custom metrics with the same names, the metric values will also be updated to corresponding task metrics.
|Modifier and Type||Method and Description|
Returns the name of custom task metric.
Returns the long value of custom task metric.