Class BatchTableWrite
Object
org.apache.spark.sql.pipelines.graph.BatchTableWrite
- All Implemented Interfaces:
FlowExecution
A `FlowExecution` that writes a batch `DataFrame` to a `Table`.
-
Constructor Summary
ConstructorsConstructorDescriptionBatchTableWrite
(org.apache.spark.sql.catalyst.TableIdentifier identifier, ResolvedFlow flow, DataflowGraph graph, Table destination, PipelineUpdateContext updateContext, scala.collection.immutable.Map<String, String> sqlConf) -
Method Summary
Modifier and TypeMethodDescriptionThe destination that this `FlowExecution` is writing to.scala.concurrent.Future<scala.runtime.BoxedUnit>
Executes this FlowExecution synchronously to perform its intended update.scala.concurrent.ExecutionContext
The thread execution context for the current `FlowExecution`.flow()
Origin to use when recording events for this flow.graph()
org.apache.spark.sql.catalyst.TableIdentifier
Identifier of this physical flowboolean
Returns true iff this `FlowExecution` executes using Spark Structured Streaming.sqlConf()
Context about this pipeline update.Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Methods inherited from interface org.apache.spark.sql.pipelines.graph.FlowExecution
displayName, exception, executeAsync, getFuture, isCompleted, spark, stop
-
Constructor Details
-
BatchTableWrite
public BatchTableWrite(org.apache.spark.sql.catalyst.TableIdentifier identifier, ResolvedFlow flow, DataflowGraph graph, Table destination, PipelineUpdateContext updateContext, scala.collection.immutable.Map<String, String> sqlConf)
-
-
Method Details
-
destination
Description copied from interface:FlowExecution
The destination that this `FlowExecution` is writing to.- Specified by:
destination
in interfaceFlowExecution
-
executeInternal
public scala.concurrent.Future<scala.runtime.BoxedUnit> executeInternal()Description copied from interface:FlowExecution
Executes this FlowExecution synchronously to perform its intended update. This method should be overridden by subclasses to provide the actual execution logic.- Specified by:
executeInternal
in interfaceFlowExecution
- Returns:
- a Future that completes when the execution is finished or stopped.
-
executionContext
public scala.concurrent.ExecutionContext executionContext()Description copied from interface:FlowExecution
The thread execution context for the current `FlowExecution`.- Specified by:
executionContext
in interfaceFlowExecution
-
flow
-
getOrigin
Description copied from interface:FlowExecution
Origin to use when recording events for this flow.- Specified by:
getOrigin
in interfaceFlowExecution
- Returns:
- (undocumented)
-
graph
-
identifier
public org.apache.spark.sql.catalyst.TableIdentifier identifier()Description copied from interface:FlowExecution
Identifier of this physical flow- Specified by:
identifier
in interfaceFlowExecution
-
isStreaming
public boolean isStreaming()Description copied from interface:FlowExecution
Returns true iff this `FlowExecution` executes using Spark Structured Streaming.- Specified by:
isStreaming
in interfaceFlowExecution
-
sqlConf
-
updateContext
Description copied from interface:FlowExecution
Context about this pipeline update.- Specified by:
updateContext
in interfaceFlowExecution
-