pyspark.sql.functions.exists#
- pyspark.sql.functions.exists(col, f)[source]#
Returns whether a predicate holds for one or more elements in the array.
New in version 3.1.0.
Changed in version 3.4.0: Supports Spark Connect.
- Parameters
- col
Column
or str name of column or expression
- ffunction
(x: Column) -> Column: ...
returning the Boolean expression. Can use methods ofColumn
, functions defined inpyspark.sql.functions
and ScalaUserDefinedFunctions
. PythonUserDefinedFunctions
are not supported (SPARK-27052).
- col
- Returns
Column
True if “any” element of an array evaluates to True when passed as an argument to given function and False otherwise.
Examples
>>> df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values")) >>> df.select(exists("values", lambda x: x < 0).alias("any_negative")).show() +------------+ |any_negative| +------------+ | false| | true| +------------+