Package org.apache.spark.sql.connector.read

Interface HasPartitionStatistics

All Superinterfaces:
InputPartition, Serializable
public interface HasPartitionStatistics extends InputPartition
A mix-in for input partitions whose records are clustered on the same set of partition keys (provided via SupportsReportPartitioning, see below). Data sources can opt-in to implement this interface for the partitions they report to Spark, which will use the info to decide whether partition grouping should be applied or not.
Since:
4.0.0
See Also:

  • Method Summary

    Modifier and Type
    Method
    Description
    OptionalLong
    filesCount()
    Returns the count of files in the partition statistics associated to this partition.
    OptionalLong
    numRows()
    Returns the number of rows in the partition statistics associated to this partition.
    OptionalLong
    sizeInBytes()
    Returns the size in bytes of the partition statistics associated to this partition.

    Methods inherited from interface org.apache.spark.sql.connector.read.InputPartition

    preferredLocations

  • Method Details

    • sizeInBytes

      OptionalLong sizeInBytes()
      Returns the size in bytes of the partition statistics associated to this partition.

    • numRows

      OptionalLong numRows()
      Returns the number of rows in the partition statistics associated to this partition.

    • filesCount

      OptionalLong filesCount()
      Returns the count of files in the partition statistics associated to this partition.