Packages

c

org.apache.spark.sql.pipelines.graph

FlowFunctionResult

case class FlowFunctionResult(requestedInputs: Set[TableIdentifier], batchInputs: Set[ResolvedInput], streamingInputs: Set[ResolvedInput], usedExternalInputs: Set[TableIdentifier], dataFrame: Try[classic.DataFrame], sqlConf: Map[String, String], analysisWarnings: Seq[AnalysisWarning] = Nil) extends Product with Serializable

Holds the DataFrame returned by a FlowFunction along with the inputs used to construct it.

batchInputs

the complete inputs read by the flow

streamingInputs

the incremental inputs read by the flow

usedExternalInputs

the identifiers of the external inputs read by the flow

dataFrame

the DataFrame expression executed by the flow if the flow can be resolved

Source
Flow.scala
Linear Supertypes
Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. FlowFunctionResult
  2. Serializable
  3. Product
  4. Equals
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new FlowFunctionResult(requestedInputs: Set[TableIdentifier], batchInputs: Set[ResolvedInput], streamingInputs: Set[ResolvedInput], usedExternalInputs: Set[TableIdentifier], dataFrame: Try[classic.DataFrame], sqlConf: Map[String, String], analysisWarnings: Seq[AnalysisWarning] = Nil)

    batchInputs

    the complete inputs read by the flow

    streamingInputs

    the incremental inputs read by the flow

    usedExternalInputs

    the identifiers of the external inputs read by the flow

    dataFrame

    the DataFrame expression executed by the flow if the flow can be resolved

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. val analysisWarnings: Seq[AnalysisWarning]
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. val batchInputs: Set[ResolvedInput]
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  8. val dataFrame: Try[classic.DataFrame]
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def failure: Seq[Throwable]

    Returns errors that occurred when attempting to analyze this Flow.

  11. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  12. def inputs: Set[TableIdentifier]

    Returns the names of all of the Inputs used when resolving this Flow.

    Returns the names of all of the Inputs used when resolving this Flow. If the flow failed to resolve, we return the all the datasets that were requested when evaluating the flow.

  13. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  17. def productElementNames: Iterator[String]
    Definition Classes
    Product
  18. val requestedInputs: Set[TableIdentifier]
  19. final def resolved: Boolean

    Whether this Flow is successfully analyzed.

  20. val sqlConf: Map[String, String]
  21. val streamingInputs: Set[ResolvedInput]
  22. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  23. val usedExternalInputs: Set[TableIdentifier]
  24. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  25. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped