Package org.apache.spark.sql.util

Class PartitioningUtils

Object
org.apache.spark.sql.util.PartitioningUtils
public class PartitioningUtils extends Object

  • Constructor Summary

    Constructors
    Constructor
    Description
    PartitioningUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static org.apache.spark.sql.catalyst.expressions.Expression
    castPartitionSpec(String value, DataType dt, org.apache.spark.sql.internal.SQLConf conf)
     
    static <T> scala.collection.immutable.Map<String,T>
    normalizePartitionSpec(scala.collection.immutable.Map<String,T> partitionSpec, StructType partCols, String tblName, scala.Function2<String,String,Object> resolver)
    Normalize the column names in partition specification, w.r.t. the real partition column names and case sensitivity. e.g., if the partition spec has a column named monTh, and there is a partition column named month, and it's case insensitive, we will normalize monTh to month.
    static void
    requireExactMatchedPartitionSpec(String tableName, scala.collection.immutable.Map<String,String> spec, scala.collection.Seq<String> partitionColumnNames)
    Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • PartitioningUtils

      public PartitioningUtils()

  • Method Details

    • castPartitionSpec

      public static org.apache.spark.sql.catalyst.expressions.Expression castPartitionSpec(String value, DataType dt, org.apache.spark.sql.internal.SQLConf conf)

    • normalizePartitionSpec

      public static <T> scala.collection.immutable.Map<String,T> normalizePartitionSpec(scala.collection.immutable.Map<String,T> partitionSpec, StructType partCols, String tblName, scala.Function2<String,String,Object> resolver)
      Normalize the column names in partition specification, w.r.t. the real partition column names and case sensitivity. e.g., if the partition spec has a column named monTh, and there is a partition column named month, and it's case insensitive, we will normalize monTh to month.
      Parameters:
      partitionSpec - (undocumented)
      partCols - (undocumented)
      tblName - (undocumented)
      resolver - (undocumented)
      Returns:
      (undocumented)

    • requireExactMatchedPartitionSpec

      public static void requireExactMatchedPartitionSpec(String tableName, scala.collection.immutable.Map<String,String> spec, scala.collection.Seq<String> partitionColumnNames)
      Verify if the input partition spec exactly matches the existing defined partition spec The columns must be the same but the orders could be different.
      Parameters:
      tableName - (undocumented)
      spec - (undocumented)
      partitionColumnNames - (undocumented)