pyspark.sql.streaming.StreamingQueryListener

class pyspark.sql.streaming.StreamingQueryListener[source]

Interface for listening to events related to StreamingQuery.

New in version 3.4.0.

Notes

The methods are not thread-safe as they may be called from different threads. The events received are identical with Scala API. Refer to its documentation.

This API is evolving.

Examples

>>> class MyListener(StreamingQueryListener):
...    def onQueryStarted(self, event: QueryStartedEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryProgress(self, event: QueryProgressEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryIdle(self, event: QueryIdleEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryTerminated(self, event: QueryTerminatedEvent) -> None:
...        # Do something with event.
...        pass
...
>>> spark.streams.addListener(MyListener())

Methods

onQueryIdle(event)

Called when the query is idle and waiting for new data to process.

onQueryProgress(event)

Called when there is some status update (ingestion rate updated, etc.)

onQueryStarted(event)

Called when a query is started.

onQueryTerminated(event)

Called when a query is stopped, with or without error.

Attributes

spark