Package org.apache.spark.sql.connector.catalog

Class CatalogV2Util

Object
org.apache.spark.sql.connector.catalog.CatalogV2Util
public class CatalogV2Util extends Object

  • Constructor Details

    • CatalogV2Util

      public CatalogV2Util()

  • Method Details

    • TABLE_RESERVED_PROPERTIES

      public static scala.collection.Seq<String> TABLE_RESERVED_PROPERTIES()
      The list of reserved table properties, which can not be removed or changed directly by the syntax: {{ ALTER TABLE ... SET TBLPROPERTIES ... }}

      They need specific syntax to modify

      Returns:
      (undocumented)

    • NAMESPACE_RESERVED_PROPERTIES

      public static scala.collection.Seq<String> NAMESPACE_RESERVED_PROPERTIES()
      The list of reserved namespace properties, which can not be removed or changed directly by the syntax: {{ ALTER NAMESPACE ... SET PROPERTIES ... }}

      They need specific syntax to modify

      Returns:
      (undocumented)

    • applyNamespaceChanges

      public static scala.collection.immutable.Map<String,String> applyNamespaceChanges(scala.collection.immutable.Map<String,String> properties, scala.collection.Seq<NamespaceChange> changes)
      Apply properties changes to a map and return the result.
      Parameters:
      properties - (undocumented)
      changes - (undocumented)
      Returns:
      (undocumented)

    • applyNamespaceChanges

      public static Map<String,String> applyNamespaceChanges(Map<String,String> properties, scala.collection.Seq<NamespaceChange> changes)
      Apply properties changes to a Java map and return the result.
      Parameters:
      properties - (undocumented)
      changes - (undocumented)
      Returns:
      (undocumented)

    • applyPropertiesChanges

      public static scala.collection.immutable.Map<String,String> applyPropertiesChanges(scala.collection.immutable.Map<String,String> properties, scala.collection.Seq<TableChange> changes)
      Apply properties changes to a map and return the result.
      Parameters:
      properties - (undocumented)
      changes - (undocumented)
      Returns:
      (undocumented)

    • applyPropertiesChanges

      public static Map<String,String> applyPropertiesChanges(Map<String,String> properties, scala.collection.Seq<TableChange> changes)
      Apply properties changes to a Java map and return the result.
      Parameters:
      properties - (undocumented)
      changes - (undocumented)
      Returns:
      (undocumented)

    • applySchemaChanges

      public static StructType applySchemaChanges(StructType schema, scala.collection.Seq<TableChange> changes, scala.Option<String> tableProvider, String statementType)
      Apply schema changes to a schema and return the result.
      Parameters:
      schema - (undocumented)
      changes - (undocumented)
      tableProvider - (undocumented)
      statementType - (undocumented)
      Returns:
      (undocumented)

    • loadTable

      public static scala.Option<Table> loadTable(CatalogPlugin catalog, Identifier ident, scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)

    • getTable

      public static Table getTable(CatalogPlugin catalog, Identifier ident, scala.Option<org.apache.spark.sql.catalyst.analysis.TimeTravelSpec> timeTravelSpec)

    • loadFunction

      public static scala.Option<UnboundFunction> loadFunction(CatalogPlugin catalog, Identifier ident)

    • loadRelation

      public static scala.Option<org.apache.spark.sql.catalyst.analysis.NamedRelation> loadRelation(CatalogPlugin catalog, Identifier ident)

    • isSessionCatalog

      public static boolean isSessionCatalog(CatalogPlugin catalog)

    • convertTableProperties

      public static scala.collection.immutable.Map<String,String> convertTableProperties(org.apache.spark.sql.catalyst.plans.logical.TableSpec t)

    • withDefaultOwnership

      public static scala.collection.immutable.Map<String,String> withDefaultOwnership(scala.collection.immutable.Map<String,String> properties)

    • getTableProviderCatalog

      public static TableCatalog getTableProviderCatalog(SupportsCatalogOptions provider, org.apache.spark.sql.connector.catalog.CatalogManager catalogManager, CaseInsensitiveStringMap options)

    • v2ColumnsToStructType

      public static StructType v2ColumnsToStructType(Column[] columns)
      Converts DS v2 columns to StructType, which encodes column comment and default value to StructField metadata. This is mainly used to define the schema of v2 scan, w.r.t. the columns of the v2 table.
      Parameters:
      columns - (undocumented)
      Returns:
      (undocumented)

    • structTypeToV2Columns

      public static Column[] structTypeToV2Columns(StructType schema)
      Converts a StructType to DS v2 columns, which decodes the StructField metadata to v2 column comment and default value or generation expression. This is mainly used to generate DS v2 columns from table schema in DDL commands, so that Spark can pass DS v2 columns to DS v2 createTable and related APIs.
      Parameters:
      schema - (undocumented)
      Returns:
      (undocumented)