Class AppendOnceFlow
Object
org.apache.spark.sql.pipelines.graph.AppendOnceFlow
- All Implemented Interfaces:
org.apache.spark.internal.Logging
,Flow
,GraphElement
,Input
,ResolutionCompletedFlow
,ResolvedFlow
A
Flow
that reads source[s] completely and appends data to the target, just once.-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionorg.apache.spark.sql.catalyst.TableIdentifier
The dataset that this Flow represents a write to.flow()
org.apache.spark.sql.catalyst.TableIdentifier
Returns the unique identifier for thisGraphElement
.boolean
once()
Whether this is a ONCE flow.Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.spark.sql.pipelines.graph.GraphElement
displayName, spark
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
Methods inherited from interface org.apache.spark.sql.pipelines.graph.ResolutionCompletedFlow
comment, func, origin, queryContext, sqlConf
Methods inherited from interface org.apache.spark.sql.pipelines.graph.ResolvedFlow
df, inputs, load, schema
-
Constructor Details
-
AppendOnceFlow
-
-
Method Details
-
destinationIdentifier
public org.apache.spark.sql.catalyst.TableIdentifier destinationIdentifier()Description copied from interface:Flow
The dataset that this Flow represents a write to.- Specified by:
destinationIdentifier
in interfaceFlow
- Specified by:
destinationIdentifier
in interfaceResolutionCompletedFlow
- Returns:
- (undocumented)
-
flow
- Specified by:
flow
in interfaceResolutionCompletedFlow
-
funcResult
- Specified by:
funcResult
in interfaceResolutionCompletedFlow
-
identifier
public org.apache.spark.sql.catalyst.TableIdentifier identifier()Description copied from interface:GraphElement
Returns the unique identifier for thisGraphElement
.- Specified by:
identifier
in interfaceFlow
- Specified by:
identifier
in interfaceGraphElement
- Specified by:
identifier
in interfaceResolutionCompletedFlow
-
once
public boolean once()Description copied from interface:Flow
Whether this is a ONCE flow. ONCE flows should run only once per full refresh.
-