org.apache.spark.sql.util

Class PartitioningUtils

Object

org.apache.spark.sql.util.PartitioningUtils

public class PartitioningUtils
extends Object

Constructor Summary

Constructors

Constructor and Description
PartitioningUtils()

Method Summary

Modifier and Type	Method and Description
static org.apache.spark.sql.catalyst.expressions.Expression	castPartitionSpec(String value, DataType dt, org.apache.spark.sql.internal.SQLConf conf)
static <T> scala.collection.immutable.Map<String,T>	normalizePartitionSpec(scala.collection.immutable.Map<String,T> partitionSpec, StructType partCols, String tblName, scala.Function2<String,String,Object> resolver) Normalize the column names in partition specification, w.r.t.
static void	requireExactMatchedPartitionSpec(String tableName, scala.collection.immutable.Map<String,String> spec, scala.collection.Seq<String> partitionColumnNames) Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitioningUtils

public PartitioningUtils()

Method Detail

castPartitionSpec

public static org.apache.spark.sql.catalyst.expressions.Expression castPartitionSpec(String value,
                                                                                     DataType dt,
                                                                                     org.apache.spark.sql.internal.SQLConf conf)

normalizePartitionSpec

public static <T> scala.collection.immutable.Map<String,T> normalizePartitionSpec(scala.collection.immutable.Map<String,T> partitionSpec,
                                                                                  StructType partCols,
                                                                                  String tblName,
                                                                                  scala.Function2<String,String,Object> resolver)

Normalize the column names in partition specification, w.r.t. the real partition column names and case sensitivity. e.g., if the partition spec has a column named monTh, and there is a partition column named month, and it's case insensitive, we will normalize monTh to month.

Parameters:

partitionSpec - (undocumented)

partCols - (undocumented)

tblName - (undocumented)

resolver - (undocumented)

Returns:

(undocumented)

requireExactMatchedPartitionSpec

public static void requireExactMatchedPartitionSpec(String tableName,
                                                    scala.collection.immutable.Map<String,String> spec,
                                                    scala.collection.Seq<String> partitionColumnNames)

Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.

Parameters:

tableName - (undocumented)

spec - (undocumented)

partitionColumnNames - (undocumented)