public class ParquetFilters
extends Object
Constructor and Description |
---|
ParquetFilters() |
Modifier and Type | Method and Description |
---|---|
static scala.Option<parquet.filter2.predicate.FilterPredicate> |
createFilter(org.apache.spark.sql.catalyst.expressions.Expression predicate) |
static scala.Option<parquet.filter2.compat.FilterCompat.Filter> |
createRecordFilter(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> filterExpressions) |
static scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> |
deserializeFilterExpressions(org.apache.hadoop.conf.Configuration conf)
Note: Inside the Hadoop API we only have access to
Configuration , not to
SparkContext , so we cannot use broadcasts to convey
the actual filter predicate. |
static String |
PARQUET_FILTER_DATA() |
static void |
serializeFilterExpressions(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> filters,
org.apache.hadoop.conf.Configuration conf)
Note: Inside the Hadoop API we only have access to
Configuration , not to
SparkContext , so we cannot use broadcasts to convey
the actual filter predicate. |
public static String PARQUET_FILTER_DATA()
public static scala.Option<parquet.filter2.compat.FilterCompat.Filter> createRecordFilter(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> filterExpressions)
public static scala.Option<parquet.filter2.predicate.FilterPredicate> createFilter(org.apache.spark.sql.catalyst.expressions.Expression predicate)
public static void serializeFilterExpressions(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> filters, org.apache.hadoop.conf.Configuration conf)
Configuration
, not to
SparkContext
, so we cannot use broadcasts to convey
the actual filter predicate.public static scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> deserializeFilterExpressions(org.apache.hadoop.conf.Configuration conf)
Configuration
, not to
SparkContext
, so we cannot use broadcasts to convey
the actual filter predicate.