Packages

t

org.apache.spark.sql.columnar

SimpleMetricsCachedBatch

trait SimpleMetricsCachedBatch extends CachedBatch

Developer API

A CachedBatch that stores some simple metrics that can be used for filtering of batches with the SimpleMetricsCachedBatchSerializer. The metrics are returned by the stats value. For each column in the batch 5 columns of metadata are needed in the row.

Annotations
@DeveloperApi() @Since( "3.1.0" )
Source
CachedBatchSerializer.scala
Linear Supertypes
CachedBatch, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SimpleMetricsCachedBatch
  2. CachedBatch
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def numRows: Int
    Definition Classes
    CachedBatch
  2. abstract val stats: InternalRow

    Holds stats for each cached column.

    Holds stats for each cached column. The optional upperBound and lowerBound should be of the same type as the original column. If they are null, then it is assumed that they are not provided, and will not be used for filtering.

    • upperBound (optional)
    • lowerBound (Optional)
    • nullCount: Int
    • rowCount: Int
    • sizeInBytes: Long

    These are repeated for each column in the original cached data.

Concrete Value Members

  1. def sizeInBytes: Long
    Definition Classes
    SimpleMetricsCachedBatch → CachedBatch