DelegatingCatalogExtension (Spark 3.5.3 JavaDoc)

Object
- org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension

All Implemented Interfaces:

CatalogExtension, CatalogPlugin, FunctionCatalog, SupportsNamespaces, TableCatalog
```
@Evolving
public abstract class DelegatingCatalogExtension
extends Object
implements CatalogExtension
```
A simple implementation of CatalogExtension, which implements all the catalog functions by calling the built-in session catalog directly. This is created for convenience, so that users only need to override some methods where they want to apply custom logic. For example, they can override createTable, do something else before calling super.createTable.

Since:

3.0.0

Field Summary
- Fields inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog
  OPTION_PREFIX, PROP_COMMENT, PROP_EXTERNAL, PROP_IS_MANAGED_LOCATION, PROP_LOCATION, PROP_OWNER, PROP_PROVIDER
- Fields inherited from interface org.apache.spark.sql.connector.catalog.SupportsNamespaces
  PROP_COMMENT, PROP_LOCATION, PROP_OWNER

Constructor Summary

Constructors
Constructor and Description

DelegatingCatalogExtension()

Constructors
Constructor and Description
`DelegatingCatalogExtension()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`alterNamespace(String[] namespace, NamespaceChange... changes)` Apply a set of metadata changes to a namespace in the catalog.
`Table`	`alterTable(Identifier ident, TableChange... changes)` Apply a set of `changes` to a table in the catalog.
`java.util.Set<TableCatalogCapability>`	`capabilities()`
`void`	`createNamespace(String[] namespace, java.util.Map<String,String> metadata)` Create a namespace in the catalog.
`Table`	`createTable(Identifier ident, Column[] columns, Transform[] partitions, java.util.Map<String,String> properties)` Create a table in the catalog.
`Table`	`createTable(Identifier ident, StructType schema, Transform[] partitions, java.util.Map<String,String> properties)` Create a table in the catalog.
`String[]`	`defaultNamespace()` Return a default namespace for the catalog.
`boolean`	`dropNamespace(String[] namespace, boolean cascade)` Drop a namespace from the catalog with cascade mode, recursively dropping all objects within the namespace if cascade is true.
`boolean`	`dropTable(Identifier ident)` Drop a table in the catalog.
`boolean`	`functionExists(Identifier ident)` Returns true if the function exists, false otherwise.
`void`	`initialize(String name, CaseInsensitiveStringMap options)` Called to initialize configuration.
`void`	`invalidateTable(Identifier ident)` Invalidate cached table metadata for an `identifier`.
`Identifier[]`	`listFunctions(String[] namespace)` List the functions in a namespace from the catalog.
`String[][]`	`listNamespaces()` List top-level namespaces from the catalog.
`String[][]`	`listNamespaces(String[] namespace)` List namespaces in a namespace.
`Identifier[]`	`listTables(String[] namespace)` List the tables in a namespace from the catalog.
`UnboundFunction`	`loadFunction(Identifier ident)` Load a function by `identifier` from the catalog.
`java.util.Map<String,String>`	`loadNamespaceMetadata(String[] namespace)` Load metadata properties for a namespace.
`Table`	`loadTable(Identifier ident)` Load table metadata by `identifier` from the catalog.
`Table`	`loadTable(Identifier ident, long timestamp)` Load table metadata at a specific time by `identifier` from the catalog.
`Table`	`loadTable(Identifier ident, String version)` Load table metadata of a specific version by `identifier` from the catalog.
`String`	`name()` Called to get this catalog's name.
`boolean`	`namespaceExists(String[] namespace)` Test whether a namespace exists.
`boolean`	`purgeTable(Identifier ident)` Drop a table in the catalog and completely remove its data by skipping a trash even if it is supported.
`void`	`renameTable(Identifier oldIdent, Identifier newIdent)` Renames a table in the catalog.
`void`	`setDelegateCatalog(CatalogPlugin delegate)` This will be called only once by Spark to pass in the Spark built-in session catalog, after `CatalogPlugin.initialize(String, CaseInsensitiveStringMap)` is called.
`boolean`	`tableExists(Identifier ident)` Test whether a table exists using an `identifier` from the catalog.

Methods inherited from class Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog
loadTable, useNullableQuerySchema

- Constructor Detail
  - DelegatingCatalogExtension
```
public DelegatingCatalogExtension()
```
- Method Detail
  - setDelegateCatalog
```
public final void setDelegateCatalog(CatalogPlugin delegate)
```
    Description copied from interface: CatalogExtension
    
    This will be called only once by Spark to pass in the Spark built-in session catalog, after CatalogPlugin.initialize(String, CaseInsensitiveStringMap) is called.
    
    Specified by:
    
    setDelegateCatalog in interface CatalogExtension
  - name
```
public String name()
```
    Description copied from interface: CatalogPlugin
    
    Called to get this catalog's name.
    This method is only called after CatalogPlugin.initialize(String, CaseInsensitiveStringMap) is called to pass the catalog's name.
    
    Specified by:
    
    name in interface CatalogPlugin
  - initialize
```
public final void initialize(String name,
                             CaseInsensitiveStringMap options)
```
    Description copied from interface: CatalogPlugin
    
    Called to initialize configuration.
    This method is called once, just after the provider is instantiated.
    
    Specified by:
    
    initialize in interface CatalogPlugin
    
    Parameters:
    
    name - the name used to identify and load this catalog
    
    options - a case-insensitive string map of configuration
  - capabilities
```
public java.util.Set<TableCatalogCapability> capabilities()
```
    Specified by:
    
    capabilities in interface TableCatalog
    
    Returns:
    
    the set of capabilities for this TableCatalog
  - defaultNamespace
```
public String[] defaultNamespace()
```
    Description copied from interface: CatalogPlugin
    
    Return a default namespace for the catalog.
    When this catalog is set as the current catalog, the namespace returned by this method will be set as the current namespace.
    The namespace returned by this method is not required to exist.
    
    Specified by:
    
    defaultNamespace in interface CatalogPlugin
    
    Returns:
    
    a multi-part namespace
  - listTables
```
public Identifier[] listTables(String[] namespace)
                        throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: TableCatalog
    
    List the tables in a namespace from the catalog.
    If the catalog supports views, this must return identifiers for only tables and not views.
    
    Specified by:
    
    listTables in interface TableCatalog
    
    Parameters:
    
    namespace - a multi-part namespace
    
    Returns:
    
    an array of Identifiers for tables
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional).
  - loadTable
```
public Table loadTable(Identifier ident)
                throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException
```
    Description copied from interface: TableCatalog
    
    Load table metadata by identifier from the catalog.
    If the catalog supports views and contains a view for the identifier and not a table, this must throw NoSuchTableException.
    
    Specified by:
    
    loadTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    Returns:
    
    the table's metadata
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table doesn't exist or is a view
  - loadTable
```
public Table loadTable(Identifier ident,
                       long timestamp)
                throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException
```
    Description copied from interface: TableCatalog
    
    Load table metadata at a specific time by identifier from the catalog.
    If the catalog supports views and contains a view for the identifier and not a table, this must throw NoSuchTableException.
    
    Specified by:
    
    loadTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    timestamp - timestamp of the table, which is microseconds since 1970-01-01 00:00:00 UTC
    
    Returns:
    
    the table's metadata
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table doesn't exist or is a view
  - loadTable
```
public Table loadTable(Identifier ident,
                       String version)
                throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException
```
    Description copied from interface: TableCatalog
    
    Load table metadata of a specific version by identifier from the catalog.
    If the catalog supports views and contains a view for the identifier and not a table, this must throw NoSuchTableException.
    
    Specified by:
    
    loadTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    version - version of the table
    
    Returns:
    
    the table's metadata
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table doesn't exist or is a view
  - invalidateTable
```
public void invalidateTable(Identifier ident)
```
    Description copied from interface: TableCatalog
    
    Invalidate cached table metadata for an identifier.
    If the table is already loaded or cached, drop cached data. If the table does not exist or is not cached, do nothing. Calling this method should not query remote services.
    
    Specified by:
    
    invalidateTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
  - tableExists
```
public boolean tableExists(Identifier ident)
```
    Description copied from interface: TableCatalog
    
    Test whether a table exists using an identifier from the catalog.
    If the catalog supports views and contains a view for the identifier and not a table, this must return false.
    
    Specified by:
    
    tableExists in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    Returns:
    
    true if the table exists, false otherwise
  - createTable
```
public Table createTable(Identifier ident,
                         StructType schema,
                         Transform[] partitions,
                         java.util.Map<String,String> properties)
                  throws org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException,
                         org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: TableCatalog
    
    Create a table in the catalog.
    This is deprecated. Please override TableCatalog.createTable(Identifier, Column[], Transform[], Map) instead.
    
    Specified by:
    
    createTable in interface TableCatalog
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
  - createTable
```
public Table createTable(Identifier ident,
                         Column[] columns,
                         Transform[] partitions,
                         java.util.Map<String,String> properties)
                  throws org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException,
                         org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: TableCatalog
    
    Create a table in the catalog.
    
    Specified by:
    
    createTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    columns - the columns of the new table.
    
    partitions - transforms to use for partitioning data in the table
    
    properties - a string map of table properties
    
    Returns:
    
    metadata for the new table. This can be null if getting the metadata for the new table is expensive. Spark will call TableCatalog.loadTable(Identifier) if needed (e.g. CTAS).
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException - If a table or view already exists for the identifier
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the identifier namespace does not exist (optional)
  - alterTable
```
public Table alterTable(Identifier ident,
                        TableChange... changes)
                 throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException
```
    Description copied from interface: TableCatalog
    
    Apply a set of changes to a table in the catalog.
    Implementations may reject the requested changes. If any change is rejected, none of the changes should be applied to the table.
    The requested changes must be applied in the order given.
    If the catalog supports views and contains a view for the identifier and not a table, this must throw NoSuchTableException.
    
    Specified by:
    
    alterTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    changes - changes to apply to the table
    
    Returns:
    
    updated metadata for the table. This can be null if getting the metadata for the updated table is expensive. Spark always discard the returned table here.
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table doesn't exist or is a view
  - dropTable
```
public boolean dropTable(Identifier ident)
```
    Description copied from interface: TableCatalog
    
    Drop a table in the catalog.
    If the catalog supports views and contains a view for the identifier and not a table, this must not drop the view and must return false.
    
    Specified by:
    
    dropTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    Returns:
    
    true if a table was deleted, false if no table exists for the identifier
  - purgeTable
```
public boolean purgeTable(Identifier ident)
```
    Description copied from interface: TableCatalog
    
    Drop a table in the catalog and completely remove its data by skipping a trash even if it is supported.
    If the catalog supports views and contains a view for the identifier and not a table, this must not drop the view and must return false.
    If the catalog supports to purge a table, this method should be overridden. The default implementation throws UnsupportedOperationException.
    
    Specified by:
    
    purgeTable in interface TableCatalog
    
    Parameters:
    
    ident - a table identifier
    
    Returns:
    
    true if a table was deleted, false if no table exists for the identifier
  - renameTable
```
public void renameTable(Identifier oldIdent,
                        Identifier newIdent)
                 throws org.apache.spark.sql.catalyst.analysis.NoSuchTableException,
                        org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException
```
    Description copied from interface: TableCatalog
    
    Renames a table in the catalog.
    If the catalog supports views and contains a view for the old identifier and not a table, this throws NoSuchTableException. Additionally, if the new identifier is a table or a view, this throws TableAlreadyExistsException.
    If the catalog does not support table renames between namespaces, it throws UnsupportedOperationException.
    
    Specified by:
    
    renameTable in interface TableCatalog
    
    Parameters:
    
    oldIdent - the table identifier of the existing table to rename
    
    newIdent - the new table identifier of the table
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchTableException - If the table to rename doesn't exist or is a view
    
    org.apache.spark.sql.catalyst.analysis.TableAlreadyExistsException - If the new table name already exists or is a view
  - listNamespaces
```
public String[][] listNamespaces()
                          throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: SupportsNamespaces
    
    List top-level namespaces from the catalog.
    If an object such as a table, view, or function exists, its parent namespaces must also exist and must be returned by this discovery method. For example, if table a.b.t exists, this method must return ["a"] in the result array.
    
    Specified by:
    
    listNamespaces in interface SupportsNamespaces
    
    Returns:
    
    an array of multi-part namespace names
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
  - listNamespaces
```
public String[][] listNamespaces(String[] namespace)
                          throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: SupportsNamespaces
    
    List namespaces in a namespace.
    If an object such as a table, view, or function exists, its parent namespaces must also exist and must be returned by this discovery method. For example, if table a.b.t exists, this method invoked as listNamespaces(["a"]) must return ["a", "b"] in the result array.
    
    Specified by:
    
    listNamespaces in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    Returns:
    
    an array of multi-part namespace names
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional)
  - namespaceExists
```
public boolean namespaceExists(String[] namespace)
```
    Description copied from interface: SupportsNamespaces
    
    Test whether a namespace exists.
    If an object such as a table, view, or function exists, its parent namespaces must also exist. For example, if table a.b.t exists, this method invoked as namespaceExists(["a"]) or namespaceExists(["a", "b"]) must return true.
    
    Specified by:
    
    namespaceExists in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    Returns:
    
    true if the namespace exists, false otherwise
  - loadNamespaceMetadata
```
public java.util.Map<String,String> loadNamespaceMetadata(String[] namespace)
                                                   throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: SupportsNamespaces
    
    Load metadata properties for a namespace.
    
    Specified by:
    
    loadNamespaceMetadata in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    Returns:
    
    a string map of properties for the given namespace
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional)
  - createNamespace
```
public void createNamespace(String[] namespace,
                            java.util.Map<String,String> metadata)
                     throws org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException
```
    Description copied from interface: SupportsNamespaces
    
    Create a namespace in the catalog.
    
    Specified by:
    
    createNamespace in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    metadata - a string map of properties for the given namespace
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NamespaceAlreadyExistsException - If the namespace already exists
  - alterNamespace
```
public void alterNamespace(String[] namespace,
                           NamespaceChange... changes)
                    throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: SupportsNamespaces
    
    Apply a set of metadata changes to a namespace in the catalog.
    
    Specified by:
    
    alterNamespace in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    changes - a collection of changes to apply to the namespace
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional)
  - dropNamespace
```
public boolean dropNamespace(String[] namespace,
                             boolean cascade)
                      throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException,
                             org.apache.spark.sql.catalyst.analysis.NonEmptyNamespaceException
```
    Description copied from interface: SupportsNamespaces
    
    Drop a namespace from the catalog with cascade mode, recursively dropping all objects within the namespace if cascade is true.
    If the catalog implementation does not support this operation, it may throw UnsupportedOperationException.
    
    Specified by:
    
    dropNamespace in interface SupportsNamespaces
    
    Parameters:
    
    namespace - a multi-part namespace
    
    cascade - When true, deletes all objects under the namespace
    
    Returns:
    
    true if the namespace was dropped
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional)
    
    org.apache.spark.sql.catalyst.analysis.NonEmptyNamespaceException - If the namespace is non-empty and cascade is false
  - loadFunction
```
public UnboundFunction loadFunction(Identifier ident)
                             throws org.apache.spark.sql.catalyst.analysis.NoSuchFunctionException
```
    Description copied from interface: FunctionCatalog
    
    Load a function by identifier from the catalog.
    
    Specified by:
    
    loadFunction in interface FunctionCatalog
    
    Parameters:
    
    ident - a function identifier
    
    Returns:
    
    an unbound function instance
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchFunctionException - If the function doesn't exist
  - listFunctions
```
public Identifier[] listFunctions(String[] namespace)
                           throws org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException
```
    Description copied from interface: FunctionCatalog
    
    List the functions in a namespace from the catalog.
    If there are no functions in the namespace, implementations should return an empty array.
    
    Specified by:
    
    listFunctions in interface FunctionCatalog
    
    Parameters:
    
    namespace - a multi-part namespace
    
    Returns:
    
    an array of Identifiers for functions
    
    Throws:
    
    org.apache.spark.sql.catalyst.analysis.NoSuchNamespaceException - If the namespace does not exist (optional).
  - functionExists
```
public boolean functionExists(Identifier ident)
```
    Description copied from interface: FunctionCatalog
    
    Returns true if the function exists, false otherwise.
    
    Specified by:
    
    functionExists in interface FunctionCatalog

Class DelegatingCatalogExtension

Field Summary

Fields inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog

Fields inherited from interface org.apache.spark.sql.connector.catalog.SupportsNamespaces

Constructor Summary

Method Summary

Methods inherited from class Object

Methods inherited from interface org.apache.spark.sql.connector.catalog.TableCatalog

Constructor Detail

DelegatingCatalogExtension

Method Detail

setDelegateCatalog

name

initialize

capabilities

defaultNamespace

listTables

loadTable

loadTable

loadTable

invalidateTable

tableExists

createTable

createTable

alterTable

dropTable

purgeTable

renameTable

listNamespaces

listNamespaces

namespaceExists

loadNamespaceMetadata

createNamespace

alterNamespace

dropNamespace

loadFunction

listFunctions

functionExists