pyspark.sql.streaming.DataStreamReader.table

DataStreamReader.table(tableName: str) → DataFrame

Define a Streaming DataFrame on a Table. The DataSource corresponding to the table should support streaming mode.

New in version 3.1.0.

Parameters

tableNamestr

string, for the name of the table.

Returns

DataFrame

Notes

This API is evolving.

Examples

Load a data stream from a table.

>>> import tempfile
>>> import time
>>> _ = spark.sql("DROP TABLE IF EXISTS my_table")
>>> with tempfile.TemporaryDirectory() as d:
...     # Create a table with Rate source.
...     q1 = spark.readStream.format("rate").load().writeStream.toTable(
...         "my_table", checkpointLocation=d)
...
...     # Read the table back and print out in the console.
...     q2 = spark.readStream.table("my_table").writeStream.format("console").start()
...     time.sleep(3)
...     q1.stop()
...     q2.stop()
...     _ = spark.sql("DROP TABLE my_table")

pyspark.sql.streaming.DataStreamReader.schema pyspark.sql.streaming.DataStreamReader.text