Package org.apache.spark.sql.pipelines.graph

Interface ResolvedFlow

All Superinterfaces:
Flow, GraphElement, Input, org.apache.spark.internal.Logging, ResolutionCompletedFlow
All Known Implementing Classes:
AppendOnceFlow, CompleteFlow, StreamingFlow
public interface ResolvedFlow extends ResolutionCompletedFlow, Input
A Flow whose flow function has successfully resolved.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.LogStringContext, org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.classic.Dataset<Row>
    df()
    The logical plan for this flow's query.
    scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier>
    inputs()
     
    org.apache.spark.sql.classic.Dataset<Row>
    load(InputReadOptions readOptions)
    Returns a DataFrame that is a result of loading data from this Input.
    StructType
    schema()
    Returns the schema of the output of this Flow.

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.Flow

    once

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.GraphElement

    displayName, spark

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logBasedOnLevel, logDebug, logDebug, logDebug, logDebug, logError, logError, logError, logError, logInfo, logInfo, logInfo, logInfo, logName, LogStringContext, logTrace, logTrace, logTrace, logTrace, logWarning, logWarning, logWarning, logWarning, MDC, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq, withLogContext

    Methods inherited from interface org.apache.spark.sql.pipelines.graph.ResolutionCompletedFlow

    destinationIdentifier, flow, func, funcResult, identifier, origin, queryContext, sqlConf

  • Method Details

    • df

      org.apache.spark.sql.classic.Dataset<Row> df()
      The logical plan for this flow's query.

    • inputs

      scala.collection.immutable.Set<org.apache.spark.sql.catalyst.TableIdentifier> inputs()

    • load

      org.apache.spark.sql.classic.Dataset<Row> load(InputReadOptions readOptions)
      Description copied from interface: Input
      Returns a DataFrame that is a result of loading data from this Input.
      Specified by:
      load in interface Input
      Parameters:
      readOptions - Type of input. Used to determine streaming/batch
      Returns:
      Streaming or batch DataFrame of this Input's data.

    • schema

      StructType schema()
      Returns the schema of the output of this Flow.