Interface SupportsMetadataColumns
- All Superinterfaces:
Table
The columns returned by metadataColumns()
may be passed as StructField
in
requested projections. Sources that implement this interface and column projection using
SupportsPushDownRequiredColumns
must accept metadata fields passed to
SupportsPushDownRequiredColumns.pruneColumns(StructType)
.
If a table column and a metadata column have the same name, the conflict is resolved by either
renaming or suppressing the metadata column. See canRenameConflictingMetadataColumns()
.
- Since:
- 3.1.0
-
Method Summary
Modifier and TypeMethodDescriptiondefault boolean
Determines how this data source handles name conflicts between metadata and data columns.Metadata columns that are supported by thisTable
.Methods inherited from interface org.apache.spark.sql.connector.catalog.Table
capabilities, columns, name, partitioning, properties, schema
-
Method Details
-
metadataColumns
MetadataColumn[] metadataColumns()Metadata columns that are supported by thisTable
.The columns returned by this method may be passed as
StructField
in requested projections usingSupportsPushDownRequiredColumns.pruneColumns(StructType)
.If a table column and a metadata column have the same name, the conflict is resolved by either renaming or suppressing the metadata column. See
canRenameConflictingMetadataColumns()
.- Returns:
- an array of
MetadataColumn
-
canRenameConflictingMetadataColumns
default boolean canRenameConflictingMetadataColumns()Determines how this data source handles name conflicts between metadata and data columns.If true, spark will automatically rename the metadata column to resolve the conflict. End users can reliably select metadata columns (renamed or not) with
Dataset.metadataColumn
, and internal code can useMetadataAttributeWithLogicalName
to extract the logical name from a metadata attribute.If false, the data column will hide the metadata column. It is recommended that Table implementations which do not support renaming should reject data column names that conflict with metadata column names.
-