Package org.apache.spark.sql.util
Class PartitioningUtils
Object
org.apache.spark.sql.util.PartitioningUtils
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionstatic org.apache.spark.sql.catalyst.expressions.Expression
castPartitionSpec
(String value, DataType dt, org.apache.spark.sql.internal.SQLConf conf) static <T> scala.collection.immutable.Map<String,
T> normalizePartitionSpec
(scala.collection.immutable.Map<String, T> partitionSpec, StructType partCols, String tblName, scala.Function2<String, String, Object> resolver) Normalize the column names in partition specification, w.r.t.static void
requireExactMatchedPartitionSpec
(String tableName, scala.collection.immutable.Map<String, String> spec, scala.collection.immutable.Seq<String> partitionColumnNames) Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.
-
Constructor Details
-
PartitioningUtils
public PartitioningUtils()
-
-
Method Details
-
castPartitionSpec
-
normalizePartitionSpec
public static <T> scala.collection.immutable.Map<String,T> normalizePartitionSpec(scala.collection.immutable.Map<String, T> partitionSpec, StructType partCols, String tblName, scala.Function2<String, String, Object> resolver) Normalize the column names in partition specification, w.r.t. the real partition column names and case sensitivity. e.g., if the partition spec has a column namedmonTh
, and there is a partition column namedmonth
, and it's case insensitive, we will normalizemonTh
tomonth
.- Parameters:
partitionSpec
- (undocumented)partCols
- (undocumented)tblName
- (undocumented)resolver
- (undocumented)- Returns:
- (undocumented)
-
requireExactMatchedPartitionSpec
public static void requireExactMatchedPartitionSpec(String tableName, scala.collection.immutable.Map<String, String> spec, scala.collection.immutable.Seq<String> partitionColumnNames) Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.- Parameters:
tableName
- (undocumented)spec
- (undocumented)partitionColumnNames
- (undocumented)
-