case class Table(identifier: TableIdentifier, specifiedSchema: Option[StructType], partitionCols: Option[Seq[String]], normalizedPath: Option[String], properties: Map[String, String] = Map.empty, comment: Option[String], origin: QueryOrigin, isStreamingTable: Boolean, format: Option[String]) extends TableInput with Dataset with Product with Serializable
A table representing a materialized dataset in a DataflowGraph.
- identifier
The identifier of this table within the graph.
- specifiedSchema
The user-specified schema for this table.
- partitionCols
What columns the table should be partitioned by when materialized.
- normalizedPath
Normalized storage location for the table based on the user-specified table path (if not defined, we will normalize a managed storage path for it).
- properties
Table Properties to set in table metadata.
- comment
User-specified comment that can be placed on the table.
- isStreamingTable
if the table is a streaming table, as defined by the source code.
- Source
- elements.scala
- Alphabetic
- By Inheritance
- Table
- Serializable
- Product
- Equals
- Dataset
- Output
- TableInput
- Input
- GraphElement
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new Table(identifier: TableIdentifier, specifiedSchema: Option[StructType], partitionCols: Option[Seq[String]], normalizedPath: Option[String], properties: Map[String, String] = Map.empty, comment: Option[String], origin: QueryOrigin, isStreamingTable: Boolean, format: Option[String])
- identifier
The identifier of this table within the graph.
- specifiedSchema
The user-specified schema for this table.
- partitionCols
What columns the table should be partitioned by when materialized.
- normalizedPath
Normalized storage location for the table based on the user-specified table path (if not defined, we will normalize a managed storage path for it).
- properties
Table Properties to set in table metadata.
- comment
User-specified comment that can be placed on the table.
- isStreamingTable
if the table is a streaming table, as defined by the source code.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- val comment: Option[String]
- def datasetType: DatasetType
Get the DatasetType of the table
- def displayName: String
Returns a user-visible name for the element.
Returns a user-visible name for the element.
- Definition Classes
- GraphElement
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- val format: Option[String]
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- val identifier: TableIdentifier
Returns the unique identifier for this GraphElement.
Returns the unique identifier for this GraphElement.
- Definition Classes
- Table → GraphElement
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- val isStreamingTable: Boolean
- def load(readOptions: InputReadOptions): classic.DataFrame
Returns a DataFrame that is a result of loading data from this Input.
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def normalized: Boolean
Return whether the storage location for this Output has been normalized.
- val normalizedPath: Option[String]
Normalized storage location used for storing materializations for this Output.
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- val origin: QueryOrigin
Contains provenance to tie back this GraphElement to the user code that defined it.
Contains provenance to tie back this GraphElement to the user code that defined it.
This must be set when a GraphElement is directly created by some user code. Subsequently, this initial origin must be propagated as is without modification. If this GraphElement is copied or converted to a different type, then this origin must be copied as is.
- Definition Classes
- Table → GraphElement
- val partitionCols: Option[Seq[String]]
- def path: String
Returns the normalized storage location to this Table.
- def productElementNames: Iterator[String]
- Definition Classes
- Product
- val properties: Map[String, String]
- def spark: classic.SparkSession
- Attributes
- protected
- Definition Classes
- GraphElement
- val specifiedSchema: Option[StructType]
The user-specified schema for this table.
The user-specified schema for this table.
- Definition Classes
- Table → TableInput
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)