org.apache.spark.sql.types
Class UserDefinedType<UserType>

Object
  org.apache.spark.sql.types.DataType
      org.apache.spark.sql.types.UserDefinedType<UserType>

All Implemented Interfaces:

java.io.Serializable

public abstract class UserDefinedType<UserType>
extends DataType
implements scala.Serializable

::DeveloperApi:: The data type for User Defined Types (UDTs).

This interface allows a user to make their own classes more interoperable with SparkSQL; e.g., by creating a UserDefinedType for a class X, it becomes possible to create a DataFrame which has class X in the schema.

For SparkSQL to recognize UDTs, the UDT must be annotated with SQLUserDefinedType.

The conversion via serialize occurs when instantiating a DataFrame from another RDD. The conversion via deserialize occurs when reading from a DataFrame.

See Also:

Serialized Form

Constructor Summary

UserDefinedType()

Method Summary

int	defaultSize() The default size of a value of the UserDefinedType is 4096 bytes.
abstract UserType	deserialize(Object datum) Convert a SQL datum to the user type
String	pyUDT() Paired Python UDT class, if exists.
abstract Object	serialize(Object obj) Convert the user type to a SQL datum
abstract DataType	sqlType() Underlying storage type for this UDT
abstract Class<UserType>	userClass() Class object for the UserType

Methods inherited from class org.apache.spark.sql.types.DataType

fromCaseClassString, fromJson, json, prettyJson, simpleString, typeName

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UserDefinedType

public UserDefinedType()

Method Detail

sqlType

public abstract DataType sqlType()

Underlying storage type for this UDT

pyUDT

public String pyUDT()

Paired Python UDT class, if exists.

serialize

public abstract Object serialize(Object obj)

Convert the user type to a SQL datum

TODO: Can we make this take obj: UserType? The issue is in CatalystTypeConverters.convertToCatalyst, where we need to convert Any to UserType.

Parameters:

obj - (undocumented)

Returns:

(undocumented)

deserialize

public abstract UserType deserialize(Object datum)

Convert a SQL datum to the user type

userClass

public abstract Class<UserType> userClass()

Class object for the UserType

Returns:

(undocumented)

defaultSize

public int defaultSize()

The default size of a value of the UserDefinedType is 4096 bytes.

Specified by:

defaultSize in class DataType

Returns:

(undocumented)