org.apache.spark.sql.connector.write

Interface LogicalWriteInfo

@Evolving
public interface LogicalWriteInfo

This interface contains logical write information that data sources can use when generating a WriteBuilder.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
default java.util.Optional<StructType>	metadataSchema() the schema of the input metadata from Spark to data source.
CaseInsensitiveStringMap	options() the options that the user specified when writing the dataset
String	queryId() queryId is a unique string of the query.
default java.util.Optional<StructType>	rowIdSchema() the schema of the ID columns from Spark to data source.
StructType	schema() the schema of the input data from Spark to data source.

Method Detail

options

CaseInsensitiveStringMap options()

the options that the user specified when writing the dataset

queryId

String queryId()

queryId is a unique string of the query. It's possible that there are many queries running at the same time, or a query is restarted and resumed. BatchWrite can use this id to identify the query.

schema

StructType schema()

the schema of the input data from Spark to data source.

rowIdSchema

default java.util.Optional<StructType> rowIdSchema()

the schema of the ID columns from Spark to data source.

metadataSchema

default java.util.Optional<StructType> metadataSchema()

the schema of the input metadata from Spark to data source.