Package org.apache.spark.sql.pipelines.graph

Class SqlGraphRegistrationContext

Object

org.apache.spark.sql.pipelines.graph.SqlGraphRegistrationContext

public class SqlGraphRegistrationContext
extends Object

SQL statement processor context. At any instant, an instance of this class holds the "active" catalog/schema in use within this SQL statement processing context, and tables/views/flows that have been registered from SQL statements within this context.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	SqlGraphRegistrationContext.SqlQueryPlanWithOrigin	Class that holds the logical plan and query origin parsed from a SQL statement.
static class	SqlGraphRegistrationContext.SqlQueryPlanWithOrigin$

Constructor Summary

Constructors

Constructor	Description
SqlGraphRegistrationContext(GraphRegistrationContext graphRegistrationContext)

Method Summary

Modifier and Type	Method	Description
void	processSqlFile(String sqlText, String sqlFilePath, SparkSession spark)
static scala.collection.immutable.Seq<SqlGraphRegistrationContext.SqlQueryPlanWithOrigin>	splitSqlFileIntoQueries(SparkSession spark, String sqlFileText, String sqlFilePath)	Given a SQL file (raw text content and path), return the parsed logical plan and query origin per SQL statement in the file contents.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SqlGraphRegistrationContext

public SqlGraphRegistrationContext(GraphRegistrationContext graphRegistrationContext)

Method Details

splitSqlFileIntoQueries

public static scala.collection.immutable.Seq<SqlGraphRegistrationContext.SqlQueryPlanWithOrigin> splitSqlFileIntoQueries(SparkSession spark,
 String sqlFileText,
 String sqlFilePath)

Given a SQL file (raw text content and path), return the parsed logical plan and query origin per SQL statement in the file contents.

Note that the returned origins will not be complete - origin information like object name and type will only be determined and populate when the logical plan is inspected during SQL element registration.

Parameters:

spark - the spark session to use to parse SQL statements.

sqlFileText - the raw text content of the SQL file.

sqlFilePath - the file path to the SQL file. Only used to populate the query origin.

Returns:

a SqlGraphRegistrationContext.SqlQueryPlanWithOrigin object per SQL statement, in the same order the SQL statements were defined in the file contents.

processSqlFile

public void processSqlFile(String sqlText,
 String sqlFilePath,
 SparkSession spark)