Interface StreamingFlowExecution
- All Superinterfaces:
FlowExecution
,org.apache.spark.internal.Logging
- All Known Implementing Classes:
StreamingTableWrite
A 'FlowExecution' that processes data statefully using Structured Streaming.
-
Nested Class Summary
Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter
-
Method Summary
Modifier and TypeMethodDescriptionStructured Streaming checkpoint.scala.concurrent.Future<scala.runtime.BoxedUnit>
Executes thisStreamingFlowExecution
by starting its stream with the correct scheduling pool and confs.flow()
The `ResolvedFlow` that this `StreamingFlowExecution` is executing.boolean
Returns true iff this `FlowExecution` executes using Spark Structured Streaming.sqlConf()
Spark confs that must be set when starting this flow.Starts a stream and returns its streaming query.trigger()
Structured Streaming trigger.Methods inherited from interface org.apache.spark.sql.pipelines.graph.FlowExecution
destination, displayName, exception, executeAsync, executionContext, getFuture, getOrigin, identifier, isCompleted, spark, stop, updateContext
Methods inherited from interface org.apache.spark.internal.Logging
initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext
-
Method Details
-
checkpointPath
String checkpointPath()Structured Streaming checkpoint. -
executeInternal
scala.concurrent.Future<scala.runtime.BoxedUnit> executeInternal()Executes thisStreamingFlowExecution
by starting its stream with the correct scheduling pool and confs.- Specified by:
executeInternal
in interfaceFlowExecution
- Returns:
- (undocumented)
-
flow
ResolvedFlow flow()The `ResolvedFlow` that this `StreamingFlowExecution` is executing. -
isStreaming
boolean isStreaming()Description copied from interface:FlowExecution
Returns true iff this `FlowExecution` executes using Spark Structured Streaming.- Specified by:
isStreaming
in interfaceFlowExecution
-
sqlConf
Spark confs that must be set when starting this flow. -
startStream
StreamingQuery startStream()Starts a stream and returns its streaming query. -
trigger
Trigger trigger()Structured Streaming trigger.
-