Class TriggeredGraphExecution

Object
org.apache.spark.sql.pipelines.graph.GraphExecution
org.apache.spark.sql.pipelines.graph.TriggeredGraphExecution
All Implemented Interfaces:
org.apache.spark.internal.Logging

public class TriggeredGraphExecution extends GraphExecution
Executes all of the flows in the given graph in topological order. Each flow processes all available data before downstream flows are triggered.

param: graphForExecution the graph to execute. param: env the context in which the graph is executed. param: onCompletion a callback to execute after all streams are done. The boolean argument is true if the execution was successful. param: clock a clock used to determine the time of execution.

  • Constructor Details

  • Method Details

    • streamTrigger

      public Trigger streamTrigger(Flow flow)
      Description copied from class: GraphExecution
      The `Trigger` configuration for a streaming flow.
      Specified by:
      streamTrigger in class GraphExecution
    • start

      public void start()
      Description copied from class: GraphExecution
      Starts the execution of flows in graphForExecution. Does not block.
      Overrides:
      start in class GraphExecution
    • awaitCompletion

      public void awaitCompletion()
      Description copied from class: GraphExecution
      Blocks the current thread while any flows are queued or running. Returns when all flows that could be run have completed. When this returns, all flows are either SUCCESSFUL, TERMINATED_WITH_ERROR, SKIPPED, CANCELED, or EXCLUDED.
      Specified by:
      awaitCompletion in class GraphExecution
    • stop

      public void stop()
      Description copied from class: GraphExecution
      Stops this execution by stopping all streams and terminating any other resources.

      This method may be called multiple times due to race conditions and must be idempotent.

      Overrides:
      stop in class GraphExecution
    • getRunTerminationReason

      public RunTerminationReason getRunTerminationReason()
      Description copied from class: GraphExecution
      Returns the reason why this flow execution has terminated. If the function is called before the flow has not terminated yet, the behavior is undefined, and may return UnexpectedRunFailure.
      Specified by:
      getRunTerminationReason in class GraphExecution
      Returns:
      (undocumented)