Package org.apache.spark.sql.pipelines.graph

Class GraphIdentifierManager

Object

org.apache.spark.sql.pipelines.graph.GraphIdentifierManager

public class GraphIdentifierManager
extends Object

Responsible for properly qualify the identifiers for datasets inside or referenced by the dataflow graph.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	GraphIdentifierManager.DatasetIdentifier	Represents the identifier for a dataset that is defined or referenced in a pipeline.
static class	GraphIdentifierManager.ExternalDatasetIdentifier	Represents the identifier for a dataset that is external to the current pipeline.
static class	GraphIdentifierManager.ExternalDatasetIdentifier$
static class	GraphIdentifierManager.InternalDatasetIdentifier	Represents the identifier for a dataset that is defined by the current pipeline.
static class	GraphIdentifierManager.InternalDatasetIdentifier$

Constructor Summary

Constructors

Constructor	Description
GraphIdentifierManager()

Method Summary

Modifier and Type	Method	Description
static GraphIdentifierManager.InternalDatasetIdentifier	parseAndQualifyFlowIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawFlowIdentifier, scala.Option<String> currentCatalog, scala.Option<String> currentDatabase)	Parses the flow identifier from the raw flow identifier and fully qualify it.
static GraphIdentifierManager.DatasetIdentifier	parseAndQualifyInputIdentifier(org.apache.spark.sql.pipelines.graph.FlowAnalysisContext context, String rawInputName)	Fully qualify (if needed) the user-specified identifier used to reference datasets, and categorizing the dataset we're referencing (i.e.
static GraphIdentifierManager.InternalDatasetIdentifier	parseAndQualifyTableIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawTableIdentifier, scala.Option<String> currentCatalog, scala.Option<String> currentDatabase)	Parses the table identifier from the raw table identifier and fully qualifies it.
static org.apache.spark.sql.catalyst.TableIdentifier	parseAndValidatePersistedViewIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawViewIdentifier, scala.Option<String> currentCatalog, scala.Option<String> currentDatabase)	Parses and validates the view identifier from the raw view identifier for persisted views.
static org.apache.spark.sql.catalyst.TableIdentifier	parseAndValidateTemporaryViewIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawViewIdentifier)	Parses and validates the view identifier from the raw view identifier for temporary views.
static org.apache.spark.sql.catalyst.TableIdentifier	parseTableIdentifier(String name, org.apache.spark.sql.classic.SparkSession spark)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

GraphIdentifierManager

public GraphIdentifierManager()

Method Details

parseTableIdentifier

public static org.apache.spark.sql.catalyst.TableIdentifier parseTableIdentifier(String name,
 org.apache.spark.sql.classic.SparkSession spark)

parseAndQualifyInputIdentifier

public static GraphIdentifierManager.DatasetIdentifier parseAndQualifyInputIdentifier(org.apache.spark.sql.pipelines.graph.FlowAnalysisContext context,
 String rawInputName)

Fully qualify (if needed) the user-specified identifier used to reference datasets, and categorizing the dataset we're referencing (i.e. dataset from this pipeline or dataset that is external to this pipeline).

Returns whether the input dataset should be read as a dataset and also the qualified identifier.

Parameters:

rawInputName - the user-specified name when referencing datasets.

context - (undocumented)

Returns:

(undocumented)

parseAndQualifyTableIdentifier

public static GraphIdentifierManager.InternalDatasetIdentifier parseAndQualifyTableIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawTableIdentifier,
 scala.Option<String> currentCatalog,
 scala.Option<String> currentDatabase)
                                                                                       throws AnalysisException

Parses the table identifier from the raw table identifier and fully qualifies it.

Parameters:

rawTableIdentifier - the raw table identifier

currentCatalog - (undocumented)

currentDatabase - (undocumented)

Returns:

the parsed table identifier

Throws:

AnalysisException

parseAndValidateTemporaryViewIdentifier

public static org.apache.spark.sql.catalyst.TableIdentifier parseAndValidateTemporaryViewIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawViewIdentifier)
                                                                                             throws AnalysisException

Parses and validates the view identifier from the raw view identifier for temporary views.

Parameters:

rawViewIdentifier - the raw view identifier

Returns:

the parsed view identifier

Throws:

AnalysisException

parseAndValidatePersistedViewIdentifier

public static org.apache.spark.sql.catalyst.TableIdentifier parseAndValidatePersistedViewIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawViewIdentifier,
 scala.Option<String> currentCatalog,
 scala.Option<String> currentDatabase)

Parses and validates the view identifier from the raw view identifier for persisted views.

Parameters:

rawViewIdentifier - the raw view identifier

currentCatalog - the catalog

currentDatabase - the schema

Returns:

the parsed view identifier

parseAndQualifyFlowIdentifier

public static GraphIdentifierManager.InternalDatasetIdentifier parseAndQualifyFlowIdentifier(org.apache.spark.sql.catalyst.TableIdentifier rawFlowIdentifier,
 scala.Option<String> currentCatalog,
 scala.Option<String> currentDatabase)
                                                                                      throws AnalysisException

Parses the flow identifier from the raw flow identifier and fully qualify it.

Parameters:

rawFlowIdentifier - the raw flow identifier

currentCatalog - (undocumented)

currentDatabase - (undocumented)

Returns:

the parsed flow identifier

Throws:

AnalysisException