Class VirtualTableInput
Object
org.apache.spark.sql.pipelines.graph.VirtualTableInput
- All Implemented Interfaces:
Serializable
,org.apache.spark.internal.Logging
,GraphElement
,Input
,TableInput
,scala.Equals
,scala.Product
public class VirtualTableInput
extends Object
implements TableInput, org.apache.spark.internal.Logging, scala.Product, Serializable
A type of
TableInput
that returns data from a specified schema or from the inferred
Flow
s that write to the table.- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Constructor Summary
ConstructorsConstructorDescriptionVirtualTableInput
(org.apache.spark.sql.catalyst.TableIdentifier identifier, scala.Option<StructType> specifiedSchema, scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier> incomingFlowIdentifiers, scala.collection.immutable.Seq<ResolvedFlow> availableFlows) -
Method Summary
Modifier and TypeMethodDescriptionabstract static R
apply
(T1 v1, T2 v2, T3 v3, T4 v4) scala.collection.immutable.Seq<ResolvedFlow>
org.apache.spark.sql.catalyst.TableIdentifier
Returns the unique identifier for thisGraphElement
.scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier>
org.apache.spark.sql.classic.Dataset<Row>
load
(InputReadOptions readOptions) Returns a DataFrame that is a result of loading data from thisInput
.origin()
Contains provenance to tie back this GraphElement to the user code that defined it.scala.Option<StructType>
The user-specified schema for this table.static String
toString()
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface scala.Equals
canEqual, equals
Methods inherited from interface org.apache.spark.sql.pipelines.graph.GraphElement
displayName, spark
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
Methods inherited from interface scala.Product
productArity, productElement, productElementName, productElementNames, productIterator, productPrefix
-
Constructor Details
-
VirtualTableInput
public VirtualTableInput(org.apache.spark.sql.catalyst.TableIdentifier identifier, scala.Option<StructType> specifiedSchema, scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier> incomingFlowIdentifiers, scala.collection.immutable.Seq<ResolvedFlow> availableFlows)
-
-
Method Details
-
apply
public abstract static R apply(T1 v1, T2 v2, T3 v3, T4 v4) -
toString
-
identifier
public org.apache.spark.sql.catalyst.TableIdentifier identifier()Description copied from interface:GraphElement
Returns the unique identifier for thisGraphElement
.- Specified by:
identifier
in interfaceGraphElement
-
specifiedSchema
Description copied from interface:TableInput
The user-specified schema for this table.- Specified by:
specifiedSchema
in interfaceTableInput
-
incomingFlowIdentifiers
public scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier> incomingFlowIdentifiers() -
availableFlows
-
origin
Description copied from interface:GraphElement
Contains provenance to tie back this GraphElement to the user code that defined it.This must be set when a
GraphElement
is directly created by some user code. Subsequently, this initial origin must be propagated as is without modification. If thisGraphElement
is copied or converted to a different type, then this origin must be copied as is.- Specified by:
origin
in interfaceGraphElement
- Returns:
- (undocumented)
-
load
Description copied from interface:Input
Returns a DataFrame that is a result of loading data from thisInput
.
-