pyspark.sql.SparkSession.streams

property SparkSession.streams

Returns a StreamingQueryManager that allows managing all the StreamingQuery instances active on this context.

New in version 2.0.0.

Returns

StreamingQueryManager

Notes

This API is evolving.

Examples

>>> spark.streams
<pyspark.sql.streaming.query.StreamingQueryManager object ...>

Get the list of active streaming queries

>>> sq = spark.readStream.format(
...     "rate").load().writeStream.format('memory').queryName('this_query').start()
>>> sqm = spark.streams
>>> [q.name for q in sqm.active]
['this_query']
>>> sq.stop()

pyspark.sql.SparkSession.stop pyspark.sql.SparkSession.table