Interface PipelineUpdateContext
- All Known Implementing Classes:
PipelineUpdateContextImpl
public interface PipelineUpdateContext
-
Method Summary
Modifier and TypeMethodDescriptionBuffer containing internal events that are emitted during a run of a pipeline.Emits internal flow progress events into the event buffer.Filter for which tables should be full refreshed when performing this update.Defines operations relates to end to end execution of a `DataflowGraph`.Filter for which flows should be refreshed when performing this update.Filter for which tables should be refreshed when performing this update.org.apache.spark.sql.classic.SparkSession
spark()
The SparkSession for this update.The unresolved graph for this update.
-
Method Details
-
eventBuffer
PipelineRunEventBuffer eventBuffer()Buffer containing internal events that are emitted during a run of a pipeline. -
flowProgressEventLogger
FlowProgressEventLogger flowProgressEventLogger()Emits internal flow progress events into the event buffer. -
fullRefreshTables
TableFilter fullRefreshTables()Filter for which tables should be full refreshed when performing this update. -
pipelineExecution
PipelineExecution pipelineExecution()Defines operations relates to end to end execution of a `DataflowGraph`. -
refreshFlows
FlowFilter refreshFlows()Filter for which flows should be refreshed when performing this update. Should be a superset of fullRefreshFlows.- Returns:
- (undocumented)
-
refreshTables
TableFilter refreshTables()Filter for which tables should be refreshed when performing this update. -
resetCheckpointFlows
FlowFilter resetCheckpointFlows() -
spark
org.apache.spark.sql.classic.SparkSession spark()The SparkSession for this update. -
unresolvedGraph
DataflowGraph unresolvedGraph()The unresolved graph for this update.
-