DecimalType

class pyspark.sql.types.DecimalType(precision=10, scale=0)

Decimal (decimal.Decimal) data type.

The DecimalType must have fixed precision (the maximum total number of digits) and scale (the number of digits on the right of dot). For example, (5, 2) can support the value from [-999.99 to 999.99].

The precision can be up to 38, the scale must be less or equal to precision.

When creating a DecimalType, the default precision and scale is (10, 0). When inferring schema from decimal.Decimal objects, it will be DecimalType(38, 18).

Parameters

precisionint, optional

the maximum (i.e. total) number of digits (default: 10)

scaleint, optional

the number of digits on right side of dot. (default: 0)

Methods

fromDDL(ddl)	Creates DataType for a given DDL-formatted string.
fromInternal(obj)	Converts an internal SQL object into a native Python object.
json()
jsonValue()
needConversion()	Does this type needs conversion between Python object and internal SQL object.
simpleString()
toInternal(obj)	Converts a Python object into an internal SQL object.
typeName()

Methods Documentation

classmethod fromDDL(ddl)

Creates DataType for a given DDL-formatted string.

New in version 4.0.0.

Parameters

ddlstr

DDL-formatted string representation of types, e.g. pyspark.sql.types.DataType.simpleString, except that top level struct type can omit the struct<> for the compatibility reason with spark.createDataFrame and Python UDFs.

Returns

DataType

Examples

Create a StructType by the corresponding DDL formatted string.

>>> from pyspark.sql.types import DataType
>>> DataType.fromDDL("b string, a int")
StructType([StructField('b', StringType(), True), StructField('a', IntegerType(), True)])

Create a single DataType by the corresponding DDL formatted string.

>>> DataType.fromDDL("decimal(10,10)")
DecimalType(10,10)

Create a StructType by the legacy string format.

>>> DataType.fromDDL("b: string, a: int")
StructType([StructField('b', StringType(), True), StructField('a', IntegerType(), True)])

fromInternal(obj)

Converts an internal SQL object into a native Python object.

json()

jsonValue()

needConversion()

Does this type needs conversion between Python object and internal SQL object.

This is used to avoid the unnecessary conversion for ArrayType/MapType/StructType.

simpleString()

toInternal(obj)

Converts a Python object into an internal SQL object.

classmethod typeName()