@InterfaceStability.Unstable public interface SupportsPushDownCatalystFilters extends DataSourceReader
DataSourceReader
. Data source readers can implement this
interface to push down arbitrary expressions as predicates to the data source.
This is an experimental and unstable interface as Expression
is not public and may get
changed in the future Spark versions.
Note that, if data source readers implement both this interface and
SupportsPushDownFilters
, Spark will ignore SupportsPushDownFilters
and only
process this interface.Modifier and Type | Method and Description |
---|---|
org.apache.spark.sql.catalyst.expressions.Expression[] |
pushCatalystFilters(org.apache.spark.sql.catalyst.expressions.Expression[] filters)
Pushes down filters, and returns unsupported filters.
|
org.apache.spark.sql.catalyst.expressions.Expression[] |
pushedCatalystFilters()
Returns the catalyst filters that are pushed in
pushCatalystFilters(Expression[]) . |
createDataReaderFactories, readSchema
org.apache.spark.sql.catalyst.expressions.Expression[] pushCatalystFilters(org.apache.spark.sql.catalyst.expressions.Expression[] filters)
org.apache.spark.sql.catalyst.expressions.Expression[] pushedCatalystFilters()
pushCatalystFilters(Expression[])
.
It's possible that there is no filters in the query and
pushCatalystFilters(Expression[])
is never called, empty array should be returned for
this case.