Package org.apache.spark.sql.connector.read

Class VariantAccessInfo

Object
org.apache.spark.sql.connector.read.VariantAccessInfo
All Implemented Interfaces:
Serializable
@Evolving public final class VariantAccessInfo extends Object implements Serializable
Variant access information that describes how variant fields are accessed in a query.

This class captures the information needed by data sources to optimize reading variant columns. Instead of reading the entire variant value, the data source can read only the fields that are actually accessed, represented as a structured schema.

For example, if a query accesses `variant_get(v, '$.a', 'int')` and `variant_get(v, '$.b', 'string')`, the extracted schema would be `struct<0:int, 1:string>` where field ordinals correspond to the access order.

Since:
4.1.0
See Also:

  • Constructor Details

    • VariantAccessInfo

      public VariantAccessInfo(String columnName, StructType extractedSchema)
      Creates variant access information for a variant column.
      Parameters:
      columnName - The name of the variant column
      extractedSchema - The schema representing extracted fields from the variant. Each field represents one variant field access, with field names typically being ordinals (e.g., "0", "1", "2") and metadata containing variant-specific information like JSON path.

  • Method Details

    • columnName

      public String columnName()
      Returns the name of the variant column.

    • extractedSchema

      public StructType extractedSchema()
      Returns the schema representing fields extracted from the variant column.

      The schema structure is:

      • Field names: Typically ordinals ("0", "1", "2", ...) representing access order
      • Field types: The target data type for each field extraction
      • Field metadata: Contains variant-specific information such as JSON path, timezone, and error handling mode

      Data sources should use this schema to determine what fields to extract from the variant and what types they should be converted to.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object