pyspark.sql.streaming.StreamingQueryListener¶

class pyspark.sql.streaming.StreamingQueryListener[source]¶

Interface for listening to events related to StreamingQuery.

New in version 3.4.0.

Notes

The methods are not thread-safe as they may be called from different threads. The events received are identical with Scala API. Refer to its documentation.

This API is evolving.

Examples

>>> class MyListener(StreamingQueryListener):
...    def onQueryStarted(self, event: QueryStartedEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryProgress(self, event: QueryProgressEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryIdle(self, event: QueryIdleEvent) -> None:
...        # Do something with event.
...        pass
...
...    def onQueryTerminated(self, event: QueryTerminatedEvent) -> None:
...        # Do something with event.
...        pass
...
>>> spark.streams.addListener(MyListener())

Methods

`onQueryIdle`(event)	Called when the query is idle and waiting for new data to process.
`onQueryProgress`(event)	Called when there is some status update (ingestion rate updated, etc.)
`onQueryStarted`(event)	Called when a query is started.
`onQueryTerminated`(event)	Called when a query is stopped, with or without error.

Attributes

spark

pyspark.sql.streaming.StreamingQueryManager

Input/Output