Class PartitioningUtils

Object
org.apache.spark.sql.util.PartitioningUtils

public class PartitioningUtils extends Object
  • Constructor Details

    • PartitioningUtils

      public PartitioningUtils()
  • Method Details

    • castPartitionSpec

      public static org.apache.spark.sql.catalyst.expressions.Expression castPartitionSpec(String value, DataType dt, org.apache.spark.sql.internal.SQLConf conf)
    • normalizePartitionSpec

      public static <T> scala.collection.immutable.Map<String,T> normalizePartitionSpec(scala.collection.immutable.Map<String,T> partitionSpec, StructType partCols, String tblName, scala.Function2<String,String,Object> resolver)
      Normalize the column names in partition specification, w.r.t. the real partition column names and case sensitivity. e.g., if the partition spec has a column named monTh, and there is a partition column named month, and it's case insensitive, we will normalize monTh to month.
      Parameters:
      partitionSpec - (undocumented)
      partCols - (undocumented)
      tblName - (undocumented)
      resolver - (undocumented)
      Returns:
      (undocumented)
    • requireExactMatchedPartitionSpec

      public static void requireExactMatchedPartitionSpec(String tableName, scala.collection.immutable.Map<String,String> spec, scala.collection.immutable.Seq<String> partitionColumnNames)
      Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.
      Parameters:
      tableName - (undocumented)
      spec - (undocumented)
      partitionColumnNames - (undocumented)