trait SupportsPushDownRequiredColumns extends ScanBuilder
A mix-in interface for ScanBuilder. Data sources can implement this
interface to push down required columns to the data source and only read these columns during
scan to reduce the size of the data to be read.
- Annotations
- @Evolving()
- Source
- SupportsPushDownRequiredColumns.java
- Since
3.0.0
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- SupportsPushDownRequiredColumns
- ScanBuilder
- AnyRef
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Abstract Value Members
- abstract def build(): Scan
- Definition Classes
- ScanBuilder
- abstract def pruneColumns(requiredSchema: StructType): Unit
Applies column pruning w.r.t.
Applies column pruning w.r.t. the given requiredSchema.
Implementation should try its best to prune the unnecessary columns or nested fields, but it's also OK to do the pruning partially, e.g., a data source may not be able to prune nested fields, and only prune top-level columns.
Note that,
Scan#readSchema()implementation should take care of the column pruning applied here.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)