org.apache.spark.ml.feature

## Class RFormula

• All Implemented Interfaces:
java.io.Serializable, Logging, RFormulaBase, Params, HasFeaturesCol, HasHandleInvalid, HasLabelCol, DefaultParamsWritable, Identifiable, MLWritable

```public class RFormula
extends Estimator<RFormulaModel>
implements RFormulaBase, DefaultParamsWritable```
Implements the transforms required for fitting a dataset against an R model formula. Currently we support a limited subset of the R operators, including '~', '.', ':', '+', '-', '*' and '^'. Also see the R formula docs here: http://stat.ethz.ch/R-manual/R-patched/library/stats/html/formula.html

The basic operators are: - `~` separate target and terms - `+` concat terms, "+ 0" means removing intercept - `-` remove a term, "- 1" means removing intercept - `:` interaction (multiplication for numeric values, or binarized categorical values) - `.` all columns except target - `*` factor crossing, includes the terms and interactions between them - `^` factor crossing to a specified degree

Suppose `a` and `b` are double columns, we use the following simple examples to illustrate the effect of `RFormula`: - `y ~ a + b` means model `y ~ w0 + w1 * a + w2 * b` where `w0` is the intercept and `w1, w2` are coefficients. - `y ~ a + b + a:b - 1` means model `y ~ w1 * a + w2 * b + w3 * a * b` where `w1, w2, w3` are coefficients. - `y ~ a * b` means model `y ~ w0 + w1 * a + w2 * b + w3 * a * b` where `w0` is the intercept and `w1, w2, w3` are coefficients - `y ~ (a + b)^2` means model `y ~ w0 + w1 * a + w2 * b + w3 * a * b` where `w0` is the intercept and `w1, w2, w3` are coefficients

RFormula produces a vector column of features and a double or string column of label. Like when formulas are used in R for linear regression, string input columns will be one-hot encoded, and numeric columns will be cast to doubles. If the label column is of type string, it will be first transformed to double with `StringIndexer`. If the label column does not exist in the DataFrame, the output label column will be created from the specified response variable in the formula.

Serialized Form
• ### Constructor Summary

Constructors
Constructor and Description
`RFormula()`
`RFormula(String uid)`
• ### Method Summary

All Methods
Modifier and Type Method and Description
`RFormula` `copy(ParamMap extra)`
Creates a copy of this instance with the same UID and some extra params.
`Param<String>` `featuresCol()`
Param for features column name.
`RFormulaModel` `fit(Dataset<?> dataset)`
Fits a model to the input data.
`BooleanParam` `forceIndexLabel()`
Force to index label whether it is numeric or string type.
`Param<String>` `formula()`
R formula parameter.
`Param<String>` `handleInvalid()`
Param for how to handle invalid data (unseen or NULL values) in features and label column of string type.
`Param<String>` `labelCol()`
Param for label column name.
`static RFormula` `load(String path)`
`static MLReader<T>` `read()`
`RFormula` `setFeaturesCol(String value)`
`RFormula` `setForceIndexLabel(boolean value)`
`RFormula` `setFormula(String value)`
Sets the formula to use for this transformer.
`RFormula` `setHandleInvalid(String value)`
`RFormula` `setLabelCol(String value)`
`RFormula` `setStringIndexerOrderType(String value)`
`Param<String>` `stringIndexerOrderType()`
Param for how to order categories of a string FEATURE column used by `StringIndexer`.
`String` `toString()`
`StructType` `transformSchema(StructType schema)`
:: DeveloperApi ::
`String` `uid()`
An immutable unique ID for the object and its derivatives.
• ### Methods inherited from class org.apache.spark.ml.Estimator

`fit, fit, fit, fit`
• ### Methods inherited from class org.apache.spark.ml.PipelineStage

`params`
• ### Methods inherited from class Object

`equals, getClass, hashCode, notify, notifyAll, wait, wait, wait`
• ### Methods inherited from interface org.apache.spark.ml.feature.RFormulaBase

`getForceIndexLabel, getFormula, getStringIndexerOrderType, hasLabelCol`
• ### Methods inherited from interface org.apache.spark.ml.param.shared.HasFeaturesCol

`getFeaturesCol`
• ### Methods inherited from interface org.apache.spark.ml.param.shared.HasLabelCol

`getLabelCol`
• ### Methods inherited from interface org.apache.spark.ml.param.shared.HasHandleInvalid

`getHandleInvalid`
• ### Methods inherited from interface org.apache.spark.ml.param.Params

`clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn`
• ### Methods inherited from interface org.apache.spark.ml.util.DefaultParamsWritable

`write`
• ### Methods inherited from interface org.apache.spark.ml.util.MLWritable

`save`
• ### Methods inherited from interface org.apache.spark.internal.Logging

`initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning`
• ### Constructor Detail

• #### RFormula

`public RFormula(String uid)`
• #### RFormula

`public RFormula()`
• ### Method Detail

`public static RFormula load(String path)`

`public static MLReader<T> read()`
• #### formula

`public Param<String> formula()`
Description copied from interface: `RFormulaBase`
R formula parameter. The formula is provided in string form.
Specified by:
`formula` in interface `RFormulaBase`
Returns:
(undocumented)
• #### forceIndexLabel

`public BooleanParam forceIndexLabel()`
Description copied from interface: `RFormulaBase`
Force to index label whether it is numeric or string type. Usually we index label only when it is string type. If the formula was used by classification algorithms, we can force to index label even it is numeric type by setting this param with true. Default: false.
Specified by:
`forceIndexLabel` in interface `RFormulaBase`
Returns:
(undocumented)
• #### handleInvalid

`public Param<String> handleInvalid()`
Description copied from interface: `RFormulaBase`
Param for how to handle invalid data (unseen or NULL values) in features and label column of string type. Options are 'skip' (filter out rows with invalid data), 'error' (throw an error), or 'keep' (put invalid data in a special additional bucket, at index numLabels). Default: "error"
Specified by:
`handleInvalid` in interface `RFormulaBase`
Specified by:
`handleInvalid` in interface `HasHandleInvalid`
Returns:
(undocumented)
• #### stringIndexerOrderType

`public final Param<String> stringIndexerOrderType()`
Description copied from interface: `RFormulaBase`
Param for how to order categories of a string FEATURE column used by `StringIndexer`. The last category after ordering is dropped when encoding strings. Supported options: 'frequencyDesc', 'frequencyAsc', 'alphabetDesc', 'alphabetAsc'. The default value is 'frequencyDesc'. When the ordering is set to 'alphabetDesc', `RFormula` drops the same category as R when encoding strings.

The options are explained using an example `'b', 'a', 'b', 'a', 'c', 'b'`:

``````
+-----------------+---------------------------------------+----------------------------------+
|      Option     | Category mapped to 0 by StringIndexer |  Category dropped by RFormula    |
+-----------------+---------------------------------------+----------------------------------+
| 'frequencyDesc' | most frequent category ('b')          | least frequent category ('c')    |
| 'frequencyAsc'  | least frequent category ('c')         | most frequent category ('b')     |
| 'alphabetDesc'  | last alphabetical category ('c')      | first alphabetical category ('a')|
| 'alphabetAsc'   | first alphabetical category ('a')     | last alphabetical category ('c') |
+-----------------+---------------------------------------+----------------------------------+
``````
Note that this ordering option is NOT used for the label column. When the label column is indexed, it uses the default descending frequency ordering in `StringIndexer`.

Specified by:
`stringIndexerOrderType` in interface `RFormulaBase`
Returns:
(undocumented)
• #### labelCol

`public final Param<String> labelCol()`
Description copied from interface: `HasLabelCol`
Param for label column name.
Specified by:
`labelCol` in interface `HasLabelCol`
Returns:
(undocumented)
• #### featuresCol

`public final Param<String> featuresCol()`
Description copied from interface: `HasFeaturesCol`
Param for features column name.
Specified by:
`featuresCol` in interface `HasFeaturesCol`
Returns:
(undocumented)
• #### uid

`public String uid()`
Description copied from interface: `Identifiable`
An immutable unique ID for the object and its derivatives.
Specified by:
`uid` in interface `Identifiable`
Returns:
(undocumented)
• #### setFormula

`public RFormula setFormula(String value)`
Sets the formula to use for this transformer. Must be called before use.
Parameters:
`value` - an R formula in string form (e.g. "y ~ x + z")
Returns:
(undocumented)
• #### setHandleInvalid

`public RFormula setHandleInvalid(String value)`
• #### setFeaturesCol

`public RFormula setFeaturesCol(String value)`
• #### setLabelCol

`public RFormula setLabelCol(String value)`
• #### setForceIndexLabel

`public RFormula setForceIndexLabel(boolean value)`
• #### setStringIndexerOrderType

`public RFormula setStringIndexerOrderType(String value)`
• #### fit

`public RFormulaModel fit(Dataset<?> dataset)`
Description copied from class: `Estimator`
Fits a model to the input data.
Specified by:
`fit` in class `Estimator<RFormulaModel>`
Parameters:
`dataset` - (undocumented)
Returns:
(undocumented)
• #### transformSchema

`public StructType transformSchema(StructType schema)`
Description copied from class: `PipelineStage`
:: DeveloperApi ::

Check transform validity and derive the output schema from the input schema.

We check validity for interactions between parameters during `transformSchema` and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by `Param.validate()`.

Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.

Specified by:
`transformSchema` in class `PipelineStage`
Parameters:
`schema` - (undocumented)
Returns:
(undocumented)
• #### copy

`public RFormula copy(ParamMap extra)`
Description copied from interface: `Params`
Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See `defaultCopy()`.
Specified by:
`copy` in interface `Params`
Specified by:
`copy` in class `Estimator<RFormulaModel>`
Parameters:
`extra` - (undocumented)
Returns:
(undocumented)
• #### toString

`public String toString()`
Specified by:
`toString` in interface `Identifiable`
Overrides:
`toString` in class `Object`