Class TriggeredGraphExecution
Object
org.apache.spark.sql.pipelines.graph.GraphExecution
org.apache.spark.sql.pipelines.graph.TriggeredGraphExecution
- All Implemented Interfaces:
org.apache.spark.internal.Logging
Executes all of the flows in the given graph in topological order. Each flow processes
all available data before downstream flows are triggered.
param: graphForExecution the graph to execute. param: env the context in which the graph is executed. param: onCompletion a callback to execute after all streams are done. The boolean argument is true if the execution was successful. param: clock a clock used to determine the time of execution.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic interface
static class
Nested classes/interfaces inherited from class org.apache.spark.sql.pipelines.graph.GraphExecution
GraphExecution.FlowExecutionAction, GraphExecution.FlowExecutionStopReason, GraphExecution.RetryFlowExecution$, GraphExecution.StopFlowExecution, GraphExecution.StopFlowExecution$
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Constructor Summary
ConstructorsConstructorDescriptionTriggeredGraphExecution
(DataflowGraph graphForExecution, PipelineUpdateContext env, scala.Function1<RunTerminationReason, scala.runtime.BoxedUnit> onCompletion, Clock clock) -
Method Summary
Modifier and TypeMethodDescriptionvoid
Blocks the current thread while any flows are queued or running.Returns the reason why this flow execution has terminated.void
start()
Starts the execution of flows ingraphForExecution
.void
stop()
Stops this execution by stopping all streams and terminating any other resources.streamTrigger
(Flow flow) The `Trigger` configuration for a streaming flow.Methods inherited from class org.apache.spark.sql.pipelines.graph.GraphExecution
determineFlowExecutionActionFromError, flowExecutions, graphForExecution, LogStringContext, maxRetryAttemptsForFlow, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, planAndStartFlow, stopFlow, stopThread
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
-
Constructor Details
-
TriggeredGraphExecution
public TriggeredGraphExecution(DataflowGraph graphForExecution, PipelineUpdateContext env, scala.Function1<RunTerminationReason, scala.runtime.BoxedUnit> onCompletion, Clock clock)
-
-
Method Details
-
streamTrigger
Description copied from class:GraphExecution
The `Trigger` configuration for a streaming flow.- Specified by:
streamTrigger
in classGraphExecution
-
start
public void start()Description copied from class:GraphExecution
Starts the execution of flows ingraphForExecution
. Does not block.- Overrides:
start
in classGraphExecution
-
awaitCompletion
public void awaitCompletion()Description copied from class:GraphExecution
Blocks the current thread while any flows are queued or running. Returns when all flows that could be run have completed. When this returns, all flows are either SUCCESSFUL, TERMINATED_WITH_ERROR, SKIPPED, CANCELED, or EXCLUDED.- Specified by:
awaitCompletion
in classGraphExecution
-
stop
public void stop()Description copied from class:GraphExecution
Stops this execution by stopping all streams and terminating any other resources.This method may be called multiple times due to race conditions and must be idempotent.
- Overrides:
stop
in classGraphExecution
-
getRunTerminationReason
Description copied from class:GraphExecution
Returns the reason why this flow execution has terminated. If the function is called before the flow has not terminated yet, the behavior is undefined, and may returnUnexpectedRunFailure
.- Specified by:
getRunTerminationReason
in classGraphExecution
- Returns:
- (undocumented)
-