pyspark.sql.functions.kll_sketch_get_quantile_bigint#

pyspark.sql.functions.kll_sketch_get_quantile_bigint(sketch, rank)[source]#

Extracts a quantile value from a KLL bigint sketch given an input rank value. The rank can be a single value or an array.

New in version 4.1.0.

Parameters

sketchColumn or column name: The KLL bigint sketch binary representation
rankColumn or column name: The rank value(s) to extract (between 0.0 and 1.0)

Returns

Column: The quantile value(s).

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([1,2,3,4,5], "INT")
>>> sketch_df = df.agg(sf.kll_sketch_agg_bigint("value").alias("sketch"))
>>> sketch_df.select(sf.kll_sketch_get_quantile_bigint("sketch", sf.lit(0.5))).show()
+-------------------------------------------+
|kll_sketch_get_quantile_bigint(sketch, 0.5)|
+-------------------------------------------+
|                                          3|
+-------------------------------------------+