pyspark.sql.functions.isnan#

pyspark.sql.functions.isnan(col)[source]#

An expression that returns true if the column is NaN.

Added in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

colColumn or column name: target column to compute on. A column that evaluates to a double or float.

Returns:

Column: True if value is NaN and False otherwise. Returns a column that evaluates to a boolean.

See also

pyspark.sql.functions.isnull()
pyspark.sql.functions.isnotnull()

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([(1.0, float('nan')), (float('nan'), 2.0)], ("a", "b"))
>>> df.select("*", sf.isnan("a"), sf.isnan(df.b)).show()
+---+---+--------+--------+
|  a|  b|isnan(a)|isnan(b)|
+---+---+--------+--------+
|1.0|NaN|   false|    true|
|NaN|2.0|    true|   false|
+---+---+--------+--------+