pyspark.sql.functions.inline#

pyspark.sql.functions.inline(col)[source]#

Explodes an array of structs into a table.

This function takes an input column containing an array of structs and returns a new column where each struct in the array is exploded into a separate row.

New in version 3.4.0.

Parameters

colColumn or str: Input column of values to explode.

Returns

Column: Generator expression with the inline exploded result.

See also

pyspark.functions.explode()
pyspark.functions.inline_outer()

Examples

Example 1: Using inline with a single struct array column

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([Row(structlist=[Row(a=1, b=2), Row(a=3, b=4)])])
>>> df.select(sf.inline(df.structlist)).show()
+---+---+
|  a|  b|
+---+---+
|  1|  2|
|  3|  4|
+---+---+

Example 2: Using inline with a column name

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([Row(structlist=[Row(a=1, b=2), Row(a=3, b=4)])])
>>> df.select(sf.inline("structlist")).show()
+---+---+
|  a|  b|
+---+---+
|  1|  2|
|  3|  4|
+---+---+

Example 3: Using inline with an alias

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([Row(structlist=[Row(a=1, b=2), Row(a=3, b=4)])])
>>> df.select(sf.inline("structlist").alias("c1", "c2")).show()
+---+---+
| c1| c2|
+---+---+
|  1|  2|
|  3|  4|
+---+---+

Example 4: Using inline with multiple struct array columns

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([
...     Row(structlist1=[Row(a=1, b=2), Row(a=3, b=4)],
...         structlist2=[Row(c=5, d=6), Row(c=7, d=8)])
... ])
>>> df.select(sf.inline("structlist1"), "structlist2") \
...     .select("a", "b", sf.inline("structlist2")).show()
+---+---+---+---+
|  a|  b|  c|  d|
+---+---+---+---+
|  1|  2|  5|  6|
|  1|  2|  7|  8|
|  3|  4|  5|  6|
|  3|  4|  7|  8|
+---+---+---+---+

Example 5: Using inline with a nested struct array column

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([
...     Row(structlist=Row(a=1, b=2, nested=[Row(c=3, d=4), Row(c=5, d=6)]))
... ])
>>> df.select(sf.inline("structlist.nested")).show()
+---+---+
|  c|  d|
+---+---+
|  3|  4|
|  5|  6|
+---+---+

Example 6: Using inline with an empty struct array column

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame(
...     [Row(structlist=[])], "structlist: array<struct<a:int,b:int>>")
>>> df.select(sf.inline(df.structlist)).show()
+---+---+
|  a|  b|
+---+---+
+---+---+

Example 7: Using inline with a struct array column containing null values

>>> import pyspark.sql.functions as sf
>>> from pyspark.sql import Row
>>> df = spark.createDataFrame([Row(structlist=[Row(a=1, b=2), None, Row(a=3, b=4)])])
>>> df.select(sf.inline(df.structlist)).show()
+----+----+
|   a|   b|
+----+----+
|   1|   2|
|NULL|NULL|
|   3|   4|
+----+----+