Package org.apache.spark.sql.types

Class UserDefinedType<UserType>

Object
org.apache.spark.sql.types.DataType
org.apache.spark.sql.types.UserDefinedType<UserType>
All Implemented Interfaces:
Serializable
Direct Known Subclasses:
VectorUDT
public abstract class UserDefinedType<UserType> extends DataType implements Serializable
The data type for User Defined Types (UDTs).

This interface allows a user to make their own classes more interoperable with SparkSQL; e.g., by creating a UserDefinedType for a class X, it becomes possible to create a DataFrame which has class X in the schema.

For SparkSQL to recognize UDTs, the UDT must be annotated with SQLUserDefinedType.

The conversion via serialize occurs when instantiating a DataFrame from another RDD. The conversion via deserialize occurs when reading from a DataFrame.

See Also:

  • Constructor Details

    • UserDefinedType

      public UserDefinedType()

  • Method Details

    • sqlType

      public abstract DataType sqlType()
      Underlying storage type for this UDT

    • pyUDT

      public String pyUDT()
      Paired Python UDT class, if exists.

    • serializedPyClass

      public String serializedPyClass()
      Serialized Python UDT class, if exists.

    • serialize

      public abstract Object serialize(UserType obj)
      Convert the user type to a SQL datum
      Parameters:
      obj - (undocumented)
      Returns:
      (undocumented)

    • deserialize

      public abstract UserType deserialize(Object datum)
      Convert a SQL datum to the user type

    • userClass

      public abstract Class<UserType> userClass()
      Class object for the UserType
      Returns:
      (undocumented)

    • defaultSize

      public int defaultSize()
      Description copied from class: DataType
      The default size of a value of this data type, used internally for size estimation.
      Specified by:
      defaultSize in class DataType
      Returns:
      (undocumented)

    • sql

      public String sql()
      Overrides:
      sql in class DataType

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public boolean equals(Object other)
      Overrides:
      equals in class Object

    • catalogString

      public String catalogString()
      Description copied from class: DataType
      String representation for the type saved in external catalogs.
      Overrides:
      catalogString in class DataType