Convert a SQL datum to the user type
Convert the user type to a SQL datum
Convert the user type to a SQL datum
TODO: Can we make this take obj: UserType? The issue is in ScalaReflection.convertToCatalyst, where we need to convert Any to UserType.
Underlying storage type for this UDT
Class object for the UserType
The default size of a value of the UserDefinedType is 4096 bytes.
The default size of a value of the UserDefinedType is 4096 bytes.
Paired Python UDT class, if exists.
Matches any expression that evaluates to this DataType
Matches any expression that evaluates to this DataType
::DeveloperApi:: The data type for User Defined Types (UDTs).
This interface allows a user to make their own classes more interoperable with SparkSQL; e.g., by creating a UserDefinedType for a class X, it becomes possible to create a
DataFrame
which has class X in the schema.For SparkSQL to recognize UDTs, the UDT must be annotated with SQLUserDefinedType.
The conversion via
serialize
occurs when instantiating aDataFrame
from another RDD. The conversion viadeserialize
occurs when reading from aDataFrame
.