pyspark.sql.functions.map_keys#

pyspark.sql.functions.map_keys(col)[source]#

Map function: Returns an unordered array containing the keys of the map.

New in version 2.3.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: Name of column or expression

Returns

Column: Keys of the map as an array.

Examples

Example 1: Extracting keys from a simple map

>>> from pyspark.sql import functions as sf
>>> df = spark.sql("SELECT map(1, 'a', 2, 'b') as data")
>>> df.select(sf.sort_array(sf.map_keys("data"))).show()
+--------------------------------+
|sort_array(map_keys(data), true)|
+--------------------------------+
|                          [1, 2]|
+--------------------------------+

Example 2: Extracting keys from a map with complex keys

>>> from pyspark.sql import functions as sf
>>> df = spark.sql("SELECT map(array(1, 2), 'a', array(3, 4), 'b') as data")
>>> df.select(sf.sort_array(sf.map_keys("data"))).show()
+--------------------------------+
|sort_array(map_keys(data), true)|
+--------------------------------+
|                [[1, 2], [3, 4]]|
+--------------------------------+

Example 3: Extracting keys from a map with duplicate keys

>>> from pyspark.sql import functions as sf
>>> originalmapKeyDedupPolicy = spark.conf.get("spark.sql.mapKeyDedupPolicy")
>>> spark.conf.set("spark.sql.mapKeyDedupPolicy", "LAST_WIN")
>>> df = spark.sql("SELECT map(1, 'a', 1, 'b') as data")
>>> df.select(sf.map_keys("data")).show()
+--------------+
|map_keys(data)|
+--------------+
|           [1]|
+--------------+
>>> spark.conf.set("spark.sql.mapKeyDedupPolicy", originalmapKeyDedupPolicy)

Example 4: Extracting keys from an empty map

>>> from pyspark.sql import functions as sf
>>> df = spark.sql("SELECT map() as data")
>>> df.select(sf.map_keys("data")).show()
+--------------+
|map_keys(data)|
+--------------+
|            []|
+--------------+