org.apache.spark.sql.types

Class UserDefinedType<UserType>

Object

org.apache.spark.sql.types.DataType

org.apache.spark.sql.types.UserDefinedType<UserType>

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

VectorUDT

public abstract class UserDefinedType<UserType>
extends DataType
implements scala.Serializable

The data type for User Defined Types (UDTs).

This interface allows a user to make their own classes more interoperable with SparkSQL; e.g., by creating a UserDefinedType for a class X, it becomes possible to create a DataFrame which has class X in the schema.

For SparkSQL to recognize UDTs, the UDT must be annotated with SQLUserDefinedType.

The conversion via serialize occurs when instantiating a DataFrame from another RDD. The conversion via deserialize occurs when reading from a DataFrame.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
UserDefinedType()

Method Summary

Modifier and Type	Method and Description
String	catalogString() String representation for the type saved in external catalogs.
int	defaultSize() The default size of a value of this data type, used internally for size estimation.
abstract UserType	deserialize(Object datum) Convert a SQL datum to the user type
boolean	equals(Object other)
int	hashCode()
String	pyUDT() Paired Python UDT class, if exists.
abstract Object	serialize(UserType obj) Convert the user type to a SQL datum
String	serializedPyClass() Serialized Python UDT class, if exists.
String	sql()
abstract DataType	sqlType() Underlying storage type for this UDT
abstract Class<UserType>	userClass() Class object for the UserType

Methods inherited from class org.apache.spark.sql.types.DataType

equalsIgnoreCaseAndNullability, equalsIgnoreNullability, equalsStructurally, equalsStructurallyByName, fromDDL, fromJson, json, parseTypeWithFallback, prettyJson, simpleString, typeName

Methods inherited from class Object

getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UserDefinedType

public UserDefinedType()

Method Detail

sqlType

public abstract DataType sqlType()

Underlying storage type for this UDT

pyUDT

public String pyUDT()

Paired Python UDT class, if exists.

serializedPyClass

public String serializedPyClass()

Serialized Python UDT class, if exists.

serialize

public abstract Object serialize(UserType obj)

Convert the user type to a SQL datum

Parameters:

obj - (undocumented)

Returns:

(undocumented)

deserialize

public abstract UserType deserialize(Object datum)

Convert a SQL datum to the user type

userClass

public abstract Class<UserType> userClass()

Class object for the UserType

Returns:

(undocumented)

defaultSize

public int defaultSize()

Description copied from class: DataType

The default size of a value of this data type, used internally for size estimation.

Specified by:

defaultSize in class DataType

Returns:

(undocumented)

sql

public String sql()

Overrides:

sql in class DataType

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object other)

Overrides:

equals in class Object

catalogString

public String catalogString()

Description copied from class: DataType

String representation for the type saved in external catalogs.

Overrides:

catalogString in class DataType