public class RFormula extends Estimator<RFormulaModel> implements RFormulaBase, DefaultParamsWritable
The basic operators are:
- ~
separate target and terms
- +
concat terms, "+ 0" means removing intercept
- -
remove a term, "- 1" means removing intercept
- :
interaction (multiplication for numeric values, or binarized categorical values)
- .
all columns except target
- *
factor crossing, includes the terms and interactions between them
- ^
factor crossing to a specified degree
Suppose a
and b
are double columns, we use the following simple examples
to illustrate the effect of RFormula
:
- y ~ a + b
means model y ~ w0 + w1 * a + w2 * b
where w0
is the intercept and w1, w2
are coefficients.
- y ~ a + b + a:b - 1
means model y ~ w1 * a + w2 * b + w3 * a * b
where w1, w2, w3
are coefficients.
- y ~ a * b
means model y ~ w0 + w1 * a + w2 * b + w3 * a * b
where w0
is the
intercept and w1, w2, w3
are coefficients
- y ~ (a + b)^2
means model y ~ w0 + w1 * a + w2 * b + w3 * a * b
where w0
is the
intercept and w1, w2, w3
are coefficients
RFormula produces a vector column of features and a double or string column of label.
Like when formulas are used in R for linear regression, string input columns will be one-hot
encoded, and numeric columns will be cast to doubles.
If the label column is of type string, it will be first transformed to double with
StringIndexer
. If the label column does not exist in the DataFrame, the output label column
will be created from the specified response variable in the formula.
Modifier and Type | Method and Description |
---|---|
RFormula |
copy(ParamMap extra)
Creates a copy of this instance with the same UID and some extra params.
|
Param<String> |
featuresCol()
Param for features column name.
|
RFormulaModel |
fit(Dataset<?> dataset)
Fits a model to the input data.
|
BooleanParam |
forceIndexLabel()
Force to index label whether it is numeric or string type.
|
Param<String> |
formula()
R formula parameter.
|
Param<String> |
handleInvalid()
Param for how to handle invalid data (unseen or NULL values) in features and label column
of string type.
|
Param<String> |
labelCol()
Param for label column name.
|
static RFormula |
load(String path) |
static MLReader<T> |
read() |
RFormula |
setFeaturesCol(String value) |
RFormula |
setForceIndexLabel(boolean value) |
RFormula |
setFormula(String value)
Sets the formula to use for this transformer.
|
RFormula |
setHandleInvalid(String value) |
RFormula |
setLabelCol(String value) |
RFormula |
setStringIndexerOrderType(String value) |
Param<String> |
stringIndexerOrderType()
Param for how to order categories of a string FEATURE column used by
StringIndexer . |
String |
toString() |
StructType |
transformSchema(StructType schema)
Check transform validity and derive the output schema from the input schema.
|
String |
uid()
An immutable unique ID for the object and its derivatives.
|
params
getForceIndexLabel, getFormula, getStringIndexerOrderType, hasLabelCol
getFeaturesCol
getLabelCol
getHandleInvalid
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn
write
save
$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize
public static RFormula load(String path)
public static MLReader<T> read()
public Param<String> formula()
RFormulaBase
formula
in interface RFormulaBase
public BooleanParam forceIndexLabel()
RFormulaBase
forceIndexLabel
in interface RFormulaBase
public Param<String> handleInvalid()
RFormulaBase
handleInvalid
in interface RFormulaBase
handleInvalid
in interface HasHandleInvalid
public final Param<String> stringIndexerOrderType()
RFormulaBase
StringIndexer
.
The last category after ordering is dropped when encoding strings.
Supported options: 'frequencyDesc', 'frequencyAsc', 'alphabetDesc', 'alphabetAsc'.
The default value is 'frequencyDesc'. When the ordering is set to 'alphabetDesc', RFormula
drops the same category as R when encoding strings.
The options are explained using an example 'b', 'a', 'b', 'a', 'c', 'b'
:
+-----------------+---------------------------------------+----------------------------------+
| Option | Category mapped to 0 by StringIndexer | Category dropped by RFormula |
+-----------------+---------------------------------------+----------------------------------+
| 'frequencyDesc' | most frequent category ('b') | least frequent category ('c') |
| 'frequencyAsc' | least frequent category ('c') | most frequent category ('b') |
| 'alphabetDesc' | last alphabetical category ('c') | first alphabetical category ('a')|
| 'alphabetAsc' | first alphabetical category ('a') | last alphabetical category ('c') |
+-----------------+---------------------------------------+----------------------------------+
Note that this ordering option is NOT used for the label column. When the label column is
indexed, it uses the default descending frequency ordering in StringIndexer
.
stringIndexerOrderType
in interface RFormulaBase
public final Param<String> labelCol()
HasLabelCol
labelCol
in interface HasLabelCol
public final Param<String> featuresCol()
HasFeaturesCol
featuresCol
in interface HasFeaturesCol
public String uid()
Identifiable
uid
in interface Identifiable
public RFormula setFormula(String value)
value
- an R formula in string form (e.g. "y ~ x + z")public RFormula setHandleInvalid(String value)
public RFormula setFeaturesCol(String value)
public RFormula setLabelCol(String value)
public RFormula setForceIndexLabel(boolean value)
public RFormula setStringIndexerOrderType(String value)
public RFormulaModel fit(Dataset<?> dataset)
Estimator
fit
in class Estimator<RFormulaModel>
dataset
- (undocumented)public StructType transformSchema(StructType schema)
PipelineStage
We check validity for interactions between parameters during transformSchema
and
raise an exception if any parameter value is invalid. Parameter value checks which
do not depend on other parameters are handled by Param.validate()
.
Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
transformSchema
in class PipelineStage
schema
- (undocumented)public RFormula copy(ParamMap extra)
Params
defaultCopy()
.copy
in interface Params
copy
in class Estimator<RFormulaModel>
extra
- (undocumented)public String toString()
toString
in interface Identifiable
toString
in class Object