pyspark.sql.DataFrame.writeStream¶
-
property
DataFrame.
writeStream
¶ Interface for saving the content of the streaming
DataFrame
out into external storage.New in version 2.0.0.
Changed in version 3.5.0: Supports Spark Connect.
- Returns
DataStreamWriter
Notes
This API is evolving.
Examples
>>> import tempfile >>> df = spark.readStream.format("rate").load() >>> type(df.writeStream) <class '...streaming.readwriter.DataStreamWriter'>
>>> with tempfile.TemporaryDirectory() as d: ... # Create a table with Rate source. ... df.writeStream.toTable( ... "my_table", checkpointLocation=d) <...streaming.query.StreamingQuery object at 0x...>