trait FlowExecution extends AnyRef
A FlowExecution specifies how to execute a flow and manages its execution.
- Source
- FlowExecution.scala
- Alphabetic
- By Inheritance
- FlowExecution
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def destination: Output
The destination that this
FlowExecutionis writing to. - abstract def executeInternal(): Future[Unit]
Executes this FlowExecution synchronously to perform its intended update.
Executes this FlowExecution synchronously to perform its intended update. This method should be overridden by subclasses to provide the actual execution logic.
- returns
a Future that completes when the execution is finished or stopped.
- abstract def getOrigin: QueryOrigin
Origin to use when recording events for this flow.
- abstract def identifier: TableIdentifier
Identifier of this physical flow
- abstract def isStreaming: Boolean
Returns true iff this
FlowExecutionexecutes using Spark Structured Streaming. - abstract def updateContext: PipelineUpdateContext
Context about this pipeline update.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def displayName: String
Returns a user-visible name for the flow.
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def exception: Option[Throwable]
Returns an optional exception that occurred during execution, if any.
- final def executeAsync(): Unit
Executes this FlowExecution asynchronously to perform its intended update.
Executes this FlowExecution asynchronously to perform its intended update. A future that can be used to track execution status is saved, and can be retrieved with
getFuture. - implicit val executionContext: ExecutionContext
The thread execution context for the current
FlowExecution. - final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def getFuture: Future[ExecutionResult]
Retrieves the future that can be used to track execution status.
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def isCompleted: Boolean
Returns true if and only if this
FlowExecutionhas been completed with either success or an exception. - final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def spark: classic.SparkSession
SparkSession to execute this physical flow with.
SparkSession to execute this physical flow with.
The default value for streaming flows is the pipeline's spark session because the source dataframe is resolved using the pipeline's spark session, and a new session will be started implicitly by the streaming query.
The default value for batch flows is a cloned spark session from the pipeline's spark session.
Please make sure that the execution thread runs in a different spark session than the pipeline's spark session.
- Attributes
- protected
- def stop(): Unit
Stops execution of this
FlowExecution.Stops execution of this
FlowExecution. If you override this, please be sure to callsuper.stop()at the beginning of your method, so we can properly handle errors when a user tries to stop a flow. - final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)