pyspark.sql.streaming.StreamingQueryManager

class pyspark.sql.streaming.StreamingQueryManager(jsqm: py4j.java_gateway.JavaObject)[source]

A class to manage all the StreamingQuery StreamingQueries active.

New in version 2.0.0.

Changed in version 3.5.0: Supports Spark Connect.

Notes

This API is evolving.

Methods

addListener(listener)

Register a StreamingQueryListener to receive up-calls for life cycle events of StreamingQuery.

awaitAnyTermination([timeout])

Wait until any of the queries on the associated SparkSession has terminated since the creation of the context, or since resetTerminated() was called.

get(id)

Returns an active query from this SparkSession.

removeListener(listener)

Deregister a StreamingQueryListener.

resetTerminated()

Forget about past terminated queries so that awaitAnyTermination() can be used again to wait for new terminations.

Attributes

active

Returns a list of active queries associated with this SparkSession

previous

pyspark.sql.streaming.StreamingQuery

next

pyspark.sql.streaming.StreamingQueryListener