pyspark.sql.functions.try_variant_array_append#

pyspark.sql.functions.try_variant_array_append(v, path, value)[source]#

Appends a value to the array in a variant at the given JSONPath location. Returns the variant unchanged if a path key or index is absent. Returns NULL if a path segment hits a value of an incompatible type, the target is not an array, or if any argument is NULL.

Added in version 4.3.0.

Parameters:

vColumn or str: a variant column or column name
pathColumn or str: the JSONPath target array. A str is a literal path; a Column supplies the path at runtime. A valid path should start with $ and is followed by zero or more segments like [123], .name, [‘name’], or [“name”].
valueColumn or str: the value to append. Any expression castable to variant.

Returns:

Column: a variant column with value appended to the array at path

Examples

>>> from pyspark.sql.functions import lit, parse_json, to_json, try_variant_array_append
>>> df = spark.createDataFrame([{
...     'json': '''[[1, 2], 5]''',
...     'path': '$[0]'
... }])
>>> v = parse_json(df.json)
>>> df.select(to_json(try_variant_array_append(v, "$", lit(3))).alias("r")).collect()
[Row(r='[[1,2],5,3]')]
>>> df.select(to_json(try_variant_array_append(v, "$[5]", lit(3))).alias("r")).collect()
[Row(r='[[1,2],5]')]
>>> df.select(to_json(try_variant_array_append(v, df.path, lit(9))).alias("r")).collect()
[Row(r='[[1,2,9],5]')]
>>> df.select(to_json(try_variant_array_append(v, "$[1]", lit(9))).alias("r")).collect()
[Row(r=None)]
>>> df.select(try_variant_array_append(v, "$", lit(None)).alias("r")).collect()
[Row(r=None)]