pyspark.sql.streaming.StreamingQuery.id

property StreamingQuery.id

Returns the unique id of this query that persists across restarts from checkpoint data. That is, this id is generated when a query is started for the first time, and will be the same every time it is restarted from checkpoint data. There can only be one query with the same id active in a Spark cluster. Also see, runId.

New in version 2.0.0.

Changed in version 3.5.0: Supports Spark Connect.

Returns

str

The unique id of query that persists across restarts from checkpoint data.

Examples

>>> sdf = spark.readStream.format("rate").load()
>>> sq = sdf.writeStream.format('memory').queryName('this_query').start()

Get the unique id of this query that persists across restarts from checkpoint data

>>> sq.id
'...'

>>> sq.stop()