object GraphIdentifierManager
Responsible for properly qualify the identifiers for datasets inside or referenced by the dataflow graph.
- Alphabetic
- By Inheritance
- GraphIdentifierManager
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- sealed trait DatasetIdentifier extends AnyRef
Represents the identifier for a dataset that is defined or referenced in a pipeline.
- case class ExternalDatasetIdentifier(identifier: TableIdentifier) extends DatasetIdentifier with Product with Serializable
Represents the identifier for a dataset that is external to the current pipeline.
- case class InternalDatasetIdentifier extends DatasetIdentifier with Product with Serializable
Represents the identifier for a dataset that is defined by the current pipeline.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def parseAndQualifyFlowIdentifier(rawFlowIdentifier: TableIdentifier, currentCatalog: Option[String], currentDatabase: Option[String]): InternalDatasetIdentifier
Parses the flow identifier from the raw flow identifier and fully qualify it.
Parses the flow identifier from the raw flow identifier and fully qualify it.
- rawFlowIdentifier
the raw flow identifier
- returns
the parsed flow identifier
- Annotations
- @throws("")
- def parseAndQualifyInputIdentifier(context: FlowAnalysisContext, rawInputName: String): DatasetIdentifier
Fully qualify (if needed) the user-specified identifier used to reference datasets, and categorizing the dataset we're referencing (i.e.
Fully qualify (if needed) the user-specified identifier used to reference datasets, and categorizing the dataset we're referencing (i.e. dataset from this pipeline or dataset that is external to this pipeline).
Returns whether the input dataset should be read as a dataset and also the qualified identifier.
- rawInputName
the user-specified name when referencing datasets.
- def parseAndQualifyTableIdentifier(rawTableIdentifier: TableIdentifier, currentCatalog: Option[String], currentDatabase: Option[String]): InternalDatasetIdentifier
Parses the table identifier from the raw table identifier and fully qualifies it.
Parses the table identifier from the raw table identifier and fully qualifies it.
- rawTableIdentifier
the raw table identifier
- returns
the parsed table identifier
- Annotations
- @throws("")
- def parseAndValidatePersistedViewIdentifier(rawViewIdentifier: TableIdentifier, currentCatalog: Option[String], currentDatabase: Option[String]): TableIdentifier
Parses and validates the view identifier from the raw view identifier for persisted views.
Parses and validates the view identifier from the raw view identifier for persisted views.
- rawViewIdentifier
the raw view identifier
- currentCatalog
the catalog
- currentDatabase
the schema
- returns
the parsed view identifier
- def parseAndValidateSinkIdentifier(rawSinkIdentifier: TableIdentifier): TableIdentifier
Parses and validates the sink identifier from the raw sink identifier.
Parses and validates the sink identifier from the raw sink identifier.
- rawSinkIdentifier
the raw view identifier
- returns
the parsed sink identifier
- Annotations
- @throws("")
- def parseAndValidateTemporaryViewIdentifier(rawViewIdentifier: TableIdentifier): TableIdentifier
Parses and validates the view identifier from the raw view identifier for temporary views.
Parses and validates the view identifier from the raw view identifier for temporary views.
- rawViewIdentifier
the raw view identifier
- returns
the parsed view identifier
- Annotations
- @throws("")
- def parseTableIdentifier(name: String, spark: classic.SparkSession): TableIdentifier
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)