trait PartitionReaderFactory extends Serializable
A factory used to create PartitionReader instances.
If Spark fails to execute any methods in the implementations of this interface or in the returned
PartitionReader (by throwing an exception), corresponding Spark task would fail and
get retried until hitting the maximum retry times.
- Annotations
- @Evolving()
- Source
- PartitionReaderFactory.java
- Since
3.0.0
- Alphabetic
- By Inheritance
- PartitionReaderFactory
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def createReader(partition: InputPartition): PartitionReader[InternalRow]
Returns a row-based partition reader to read data from the given
InputPartition.Returns a row-based partition reader to read data from the given
InputPartition.Implementations probably need to cast the input partition to the concrete
InputPartitionclass defined for the data source.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def createColumnarReader(partition: InputPartition): PartitionReader[ColumnarBatch]
Returns a columnar partition reader to read data from the given
InputPartition.Returns a columnar partition reader to read data from the given
InputPartition.Implementations probably need to cast the input partition to the concrete
InputPartitionclass defined for the data source. - final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def supportColumnarReads(partition: InputPartition): Boolean
Returns true if the given
InputPartitionshould be read by Spark in a columnar way.Returns true if the given
InputPartitionshould be read by Spark in a columnar way. This means, implementations must also implement#createColumnarReader(InputPartition)for the input partitions that this method returns true.As of Spark 2.4, Spark can only read all input partition in a columnar way, or none of them. Data source can't mix columnar and row-based partitions. This may be relaxed in future versions.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)