public final class DataFrameNaFunctions
extends Object
DataFrame
s.Constructor and Description |
---|
DataFrameNaFunctions(DataFrame df) |
Modifier and Type | Method and Description |
---|---|
DataFrame |
drop()
Returns a new
DataFrame that drops rows containing any null values. |
DataFrame |
drop(int minNonNulls)
Returns a new
DataFrame that drops rows containing less than minNonNulls non-null values. |
DataFrame |
drop(int minNonNulls,
scala.collection.Seq<String> cols)
(Scala-specific) Returns a new
DataFrame that drops rows containing less than
minNonNulls non-null values in the specified columns. |
DataFrame |
drop(int minNonNulls,
String[] cols)
Returns a new
DataFrame that drops rows containing less than minNonNulls non-null
values in the specified columns. |
DataFrame |
drop(scala.collection.Seq<String> cols)
(Scala-specific) Returns a new
DataFrame that drops rows containing any null values
in the specified columns. |
DataFrame |
drop(String how)
Returns a new
DataFrame that drops rows containing null values. |
DataFrame |
drop(String[] cols)
Returns a new
DataFrame that drops rows containing any null values
in the specified columns. |
DataFrame |
drop(String how,
scala.collection.Seq<String> cols)
(Scala-specific) Returns a new
DataFrame that drops rows containing null values
in the specified columns. |
DataFrame |
drop(String how,
String[] cols)
Returns a new
DataFrame that drops rows containing null values
in the specified columns. |
DataFrame |
fill(double value)
Returns a new
DataFrame that replaces null values in numeric columns with value . |
DataFrame |
fill(double value,
scala.collection.Seq<String> cols)
(Scala-specific) Returns a new
DataFrame that replaces null values in specified
numeric columns. |
DataFrame |
fill(double value,
String[] cols)
Returns a new
DataFrame that replaces null values in specified numeric columns. |
DataFrame |
fill(java.util.Map<String,Object> valueMap)
Returns a new
DataFrame that replaces null values. |
DataFrame |
fill(scala.collection.immutable.Map<String,Object> valueMap)
(Scala-specific) Returns a new
DataFrame that replaces null values. |
DataFrame |
fill(String value)
Returns a new
DataFrame that replaces null values in string columns with value . |
DataFrame |
fill(String value,
scala.collection.Seq<String> cols)
(Scala-specific) Returns a new
DataFrame that replaces null values in
specified string columns. |
DataFrame |
fill(String value,
String[] cols)
Returns a new
DataFrame that replaces null values in specified string columns. |
public DataFrameNaFunctions(DataFrame df)
public DataFrame drop(String how)
DataFrame
that drops rows containing null values.
If how
is "any", then drop rows containing any null values.
If how
is "all", then drop rows only if every column is null for that row.
public DataFrame drop(String[] cols)
DataFrame
that drops rows containing any null values
in the specified columns.public DataFrame drop(scala.collection.Seq<String> cols)
DataFrame
that drops rows containing any null values
in the specified columns.public DataFrame drop(String how, String[] cols)
DataFrame
that drops rows containing null values
in the specified columns.
If how
is "any", then drop rows containing any null values in the specified columns.
If how
is "all", then drop rows only if every specified column is null for that row.
public DataFrame drop(String how, scala.collection.Seq<String> cols)
DataFrame
that drops rows containing null values
in the specified columns.
If how
is "any", then drop rows containing any null values in the specified columns.
If how
is "all", then drop rows only if every specified column is null for that row.
public DataFrame drop(int minNonNulls)
DataFrame
that drops rows containing less than minNonNulls
non-null values.public DataFrame drop(int minNonNulls, String[] cols)
DataFrame
that drops rows containing less than minNonNulls
non-null
values in the specified columns.public DataFrame drop(int minNonNulls, scala.collection.Seq<String> cols)
DataFrame
that drops rows containing less than
minNonNulls
non-null values in the specified columns.public DataFrame fill(double value)
DataFrame
that replaces null values in numeric columns with value
.public DataFrame fill(String value)
DataFrame
that replaces null values in string columns with value
.public DataFrame fill(double value, String[] cols)
DataFrame
that replaces null values in specified numeric columns.
If a specified column is not a numeric column, it is ignored.public DataFrame fill(double value, scala.collection.Seq<String> cols)
DataFrame
that replaces null values in specified
numeric columns. If a specified column is not a numeric column, it is ignored.public DataFrame fill(String value, String[] cols)
DataFrame
that replaces null values in specified string columns.
If a specified column is not a string column, it is ignored.public DataFrame fill(String value, scala.collection.Seq<String> cols)
DataFrame
that replaces null values in
specified string columns. If a specified column is not a string column, it is ignored.public DataFrame fill(java.util.Map<String,Object> valueMap)
DataFrame
that replaces null values.
The key of the map is the column name, and the value of the map is the replacement value.
The value must be of the following type: Integer
, Long
, Float
, Double
, String
.
For example, the following replaces null values in column "A" with string "unknown", and null values in column "B" with numeric value 1.0.
import com.google.common.collect.ImmutableMap;
df.na.fill(ImmutableMap.of("A", "unknown", "B", 1.0));
public DataFrame fill(scala.collection.immutable.Map<String,Object> valueMap)
DataFrame
that replaces null values.
The key of the map is the column name, and the value of the map is the replacement value.
The value must be of the following type: Int
, Long
, Float
, Double
, String
.
For example, the following replaces null values in column "A" with string "unknown", and null values in column "B" with numeric value 1.0.
df.na.fill(Map(
"A" -> "unknown",
"B" -> 1.0
))