Trait

org.apache.spark.sql.streaming

StreamingQuery

Related Doc: package streaming

Permalink

trait StreamingQuery extends AnyRef

:: Experimental :: A handle to a query that is executing continuously in the background as new data arrives. All these methods are thread-safe.

Annotations
@Experimental()
Source
StreamingQuery.scala
Since

2.0.0

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. StreamingQuery
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def awaitTermination(timeoutMs: Long): Boolean

    Permalink

    Waits for the termination of this query, either by query.stop() or by an exception.

    Waits for the termination of this query, either by query.stop() or by an exception. If the query has terminated with an exception, then the exception will be thrown. Otherwise, it returns whether the query has terminated or not within the timeoutMs milliseconds.

    If the query has terminated, then all subsequent calls to this method will either return true immediately (if the query was terminated by stop()), or throw the exception immediately (if the query has terminated with exception).

    Since

    2.0.0

    Exceptions thrown

    StreamingQueryException, if this query has terminated with an exception

  2. abstract def awaitTermination(): Unit

    Permalink

    Waits for the termination of this query, either by query.stop() or by an exception.

    Waits for the termination of this query, either by query.stop() or by an exception. If the query has terminated with an exception, then the exception will be thrown.

    If the query has terminated, then all subsequent calls to this method will either return immediately (if the query was terminated by stop()), or throw the exception immediately (if the query has terminated with exception).

    Since

    2.0.0

    Exceptions thrown

    StreamingQueryException, if this query has terminated with an exception.

  3. abstract def exception: Option[StreamingQueryException]

    Permalink

    Returns the StreamingQueryException if the query was terminated by an exception.

    Returns the StreamingQueryException if the query was terminated by an exception.

    Since

    2.0.0

  4. abstract def explain(extended: Boolean): Unit

    Permalink

    Prints the physical plan to the console for debugging purposes.

    Prints the physical plan to the console for debugging purposes.

    extended

    whether to do extended explain or not

    Since

    2.0.0

  5. abstract def explain(): Unit

    Permalink

    Prints the physical plan to the console for debugging purposes.

    Prints the physical plan to the console for debugging purposes.

    Since

    2.0.0

  6. abstract def id: Long

    Permalink

    Returns the unique id of this query.

    Returns the unique id of this query. This id is automatically generated and is unique across all queries that have been started in the current process.

    Since

    2.0.0

  7. abstract def isActive: Boolean

    Permalink

    Whether the query is currently active or not

    Whether the query is currently active or not

    Since

    2.0.0

  8. abstract def name: String

    Permalink

    Returns the name of the query.

    Returns the name of the query. This name is unique across all active queries. This can be set in the DataStreamWriter as dataframe.writeStream.queryName("query").start().

    Since

    2.0.0

  9. abstract def processAllAvailable(): Unit

    Permalink

    Blocks until all available data in the source has been processed and committed to the sink.

    Blocks until all available data in the source has been processed and committed to the sink. This method is intended for testing. Note that in the case of continually arriving data, this method may block forever. Additionally, this method is only guaranteed to block until data that has been synchronously appended data to a org.apache.spark.sql.execution.streaming.Source prior to invocation. (i.e. getOffset must immediately reflect the addition).

    Since

    2.0.0

  10. abstract def sinkStatus: SinkStatus

    Permalink

    Returns current status of the sink.

  11. abstract def sourceStatuses: Array[SourceStatus]

    Permalink

    Returns current status of all the sources.

    Returns current status of all the sources.

    Since

    2.0.0

  12. abstract def sparkSession: SparkSession

    Permalink

    Returns the SparkSession associated with this.

    Returns the SparkSession associated with this.

    Since

    2.0.0

  13. abstract def stop(): Unit

    Permalink

    Stops the execution of this query if it is running.

    Stops the execution of this query if it is running. This method blocks until the threads performing execution has stopped.

    Since

    2.0.0

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  13. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  14. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  15. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  16. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  17. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  18. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  19. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped