Class

org.apache.spark.sql.internal

CatalogImpl

Related Doc: package internal

Permalink

class CatalogImpl extends Catalog

Internal implementation of the user-facing Catalog.

Source
CatalogImpl.scala
Linear Supertypes
Catalog, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. CatalogImpl
  2. Catalog
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CatalogImpl(sparkSession: SparkSession)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def cacheTable(tableName: String): Unit

    Permalink

    Caches the specified table in-memory.

    Caches the specified table in-memory.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  6. def clearCache(): Unit

    Permalink

    Removes all cached tables from the in-memory cache.

    Removes all cached tables from the in-memory cache.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  8. def createExternalTable(tableName: String, source: String, schema: StructType, options: Map[String, String]): DataFrame

    Permalink

    :: Experimental :: (Scala-specific) Create an external table from the given path based on a data source, a schema and a set of options.

    :: Experimental :: (Scala-specific) Create an external table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  9. def createExternalTable(tableName: String, source: String, schema: StructType, options: Map[String, String]): DataFrame

    Permalink

    :: Experimental :: Create an external table from the given path based on a data source, a schema and a set of options.

    :: Experimental :: Create an external table from the given path based on a data source, a schema and a set of options. Then, returns the corresponding DataFrame.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  10. def createExternalTable(tableName: String, source: String, options: Map[String, String]): DataFrame

    Permalink

    :: Experimental :: (Scala-specific) Creates an external table from the given path based on a data source and a set of options.

    :: Experimental :: (Scala-specific) Creates an external table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  11. def createExternalTable(tableName: String, source: String, options: Map[String, String]): DataFrame

    Permalink

    :: Experimental :: Creates an external table from the given path based on a data source and a set of options.

    :: Experimental :: Creates an external table from the given path based on a data source and a set of options. Then, returns the corresponding DataFrame.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  12. def createExternalTable(tableName: String, path: String, source: String): DataFrame

    Permalink

    :: Experimental :: Creates an external table from the given path based on a data source and returns the corresponding DataFrame.

    :: Experimental :: Creates an external table from the given path based on a data source and returns the corresponding DataFrame.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  13. def createExternalTable(tableName: String, path: String): DataFrame

    Permalink

    :: Experimental :: Creates an external table from the given path and returns the corresponding DataFrame.

    :: Experimental :: Creates an external table from the given path and returns the corresponding DataFrame. It will use the default data source configured by spark.sql.sources.default.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @Experimental()
    Since

    2.0.0

  14. def currentDatabase: String

    Permalink

    Returns the current default database in this session.

    Returns the current default database in this session.

    Definition Classes
    CatalogImplCatalog
  15. def dropTempView(viewName: String): Unit

    Permalink

    Drops the temporary view with the given view name in the catalog.

    Drops the temporary view with the given view name in the catalog. If the view has been cached/persisted before, it's also unpersisted.

    viewName

    the name of the view to be dropped.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  16. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  17. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  18. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  19. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  20. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  21. def isCached(qName: Dataset[_]): Boolean

    Permalink

    Returns true if the Dataset is currently cached in-memory.

    Returns true if the Dataset is currently cached in-memory.

    Attributes
    protected[org.apache.spark.sql]
    Since

    2.0.0

  22. def isCached(tableName: String): Boolean

    Permalink

    Returns true if the table is currently cached in-memory.

    Returns true if the table is currently cached in-memory.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  23. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  24. def listColumns(dbName: String, tableName: String): Dataset[catalog.Column]

    Permalink

    Returns a list of columns for the given table in the specified database.

    Returns a list of columns for the given table in the specified database.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @throws( "database or table does not exist" )
  25. def listColumns(tableName: String): Dataset[catalog.Column]

    Permalink

    Returns a list of columns for the given table in the current database.

    Returns a list of columns for the given table in the current database.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @throws( "table does not exist" )
  26. def listDatabases(): Dataset[Database]

    Permalink

    Returns a list of databases available across all sessions.

    Returns a list of databases available across all sessions.

    Definition Classes
    CatalogImplCatalog
  27. def listFunctions(dbName: String): Dataset[Function]

    Permalink

    Returns a list of functions registered in the specified database.

    Returns a list of functions registered in the specified database. This includes all temporary functions

    Definition Classes
    CatalogImplCatalog
    Annotations
    @throws( "database does not exist" )
  28. def listFunctions(): Dataset[Function]

    Permalink

    Returns a list of functions registered in the current database.

    Returns a list of functions registered in the current database. This includes all temporary functions

    Definition Classes
    CatalogImplCatalog
  29. def listTables(dbName: String): Dataset[Table]

    Permalink

    Returns a list of tables in the specified database.

    Returns a list of tables in the specified database. This includes all temporary tables.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @throws( "database does not exist" )
  30. def listTables(): Dataset[Table]

    Permalink

    Returns a list of tables in the current database.

    Returns a list of tables in the current database. This includes all temporary tables.

    Definition Classes
    CatalogImplCatalog
  31. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  32. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  33. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  34. def refreshByPath(resourcePath: String): Unit

    Permalink

    Refresh the cache entry and the associated metadata for all dataframes (if any), that contain the given data source path.

    Refresh the cache entry and the associated metadata for all dataframes (if any), that contain the given data source path.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  35. def refreshTable(tableName: String): Unit

    Permalink

    Refresh the cache entry for a table, if any.

    Refresh the cache entry for a table, if any. For Hive metastore table, the metadata is refreshed.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  36. def setCurrentDatabase(dbName: String): Unit

    Permalink

    Sets the current default database in this session.

    Sets the current default database in this session.

    Definition Classes
    CatalogImplCatalog
    Annotations
    @throws( "database does not exist" )
  37. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  38. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  39. def uncacheTable(tableName: String): Unit

    Permalink

    Removes the specified table from the in-memory cache.

    Removes the specified table from the in-memory cache.

    Definition Classes
    CatalogImplCatalog
    Since

    2.0.0

  40. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  41. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  42. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Catalog

Inherited from AnyRef

Inherited from Any

cachemgmt

ddl_ops

Ungrouped