Input/Output¶
Data Generator¶
|
Create a DataFrame with some range of numbers. |
Spark Metastore Table¶
|
Read a Spark table and return a DataFrame. |
|
Write the DataFrame into a Spark table. |
Delta Lake¶
|
Read a Delta Lake table on some file system and return a DataFrame. |
|
Write the DataFrame out as a Delta Lake table. |
Parquet¶
|
Load a parquet object from the file path, returning a DataFrame. |
|
Write the DataFrame out as a Parquet file or directory. |
ORC¶
|
Load an ORC object from the file path, returning a DataFrame. |
|
Write a DataFrame to the ORC format. |
Generic Spark I/O¶
|
Load a DataFrame from a Spark data source. |
|
Write the DataFrame out to a Spark data source. |
Flat File / CSV¶
|
Read CSV (comma-separated) file into DataFrame or Series. |
|
Write object to a comma-separated values (csv) file. |
Clipboard¶
|
Read text from clipboard and pass to read_csv. |
|
Copy object to the system clipboard. |
Excel¶
|
Read an Excel file into a pandas-on-Spark DataFrame or Series. |
|
Write object to an Excel sheet. |
JSON¶
|
Convert a JSON string to DataFrame. |
|
Convert the object to a JSON string. |
HTML¶
|
Read HTML tables into a |
|
Render a DataFrame as an HTML table. |
SQL¶
|
Read SQL database table into a DataFrame. |
|
Read SQL query into a DataFrame. |
|
Read SQL query or database table into a DataFrame. |