pyspark.sql.DataFrameReader.format#
- DataFrameReader.format(source)[source]#
Specifies the input data source format.
New in version 1.4.0.
Changed in version 3.4.0: Supports Spark Connect.
- Parameters
- sourcestr
string, name of the data source, e.g. ‘json’, ‘parquet’.
Examples
>>> spark.read.format('json') <...readwriter.DataFrameReader object ...>
Write a DataFrame into a JSON file and read it back.
>>> import tempfile >>> with tempfile.TemporaryDirectory(prefix="format") as d: ... # Write a DataFrame into a JSON file ... spark.createDataFrame( ... [{"age": 100, "name": "Hyukjin Kwon"}] ... ).write.mode("overwrite").format("json").save(d) ... ... # Read the JSON file as a DataFrame. ... spark.read.format('json').load(d).show() +---+------------+ |age| name| +---+------------+ |100|Hyukjin Kwon| +---+------------+