Package org.apache.spark.ml.feature

Class ChiSqSelector

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<T>
org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>
org.apache.spark.ml.feature.ChiSqSelector
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, org.apache.spark.ml.feature.SelectorParams, Params, HasFeaturesCol, HasLabelCol, HasOutputCol, DefaultParamsWritable, Identifiable, MLWritable
public final class ChiSqSelector extends org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>
Deprecated.
use UnivariateFeatureSelector instead. Since 3.1.1.
Chi-Squared feature selection, which selects categorical features to use for predicting a categorical label. The selector supports different selection methods: numTopFeatures, percentile, fpr, fdr, fwe. - numTopFeatures chooses a fixed number of top features according to a chi-squared test. - percentile is similar but chooses a fraction of all features instead of a fixed number. - fpr chooses all features whose p-value are below a threshold, thus controlling the false positive rate of selection. - fdr uses the [Benjamini-Hochberg procedure] (https://en.wikipedia.org/wiki/False_discovery_rate#Benjamini.E2.80.93Hochberg_procedure) to choose all features whose false discovery rate is below a threshold. - fwe chooses all features whose p-values are below a threshold. The threshold is scaled by 1/numFeatures, thus controlling the family-wise error rate of selection. By default, the selection method is numTopFeatures, with the default number of top features set to 50.
See Also:

  • Constructor Details

    • ChiSqSelector

      public ChiSqSelector(String uid)
      Deprecated.

    • ChiSqSelector

      public ChiSqSelector()
      Deprecated.

  • Method Details

    • load

      public static ChiSqSelector load(String path)
      Deprecated.

    • read

      public static MLReader<T> read()
      Deprecated.

    • uid

      public String uid()
      Deprecated.
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Returns:
      (undocumented)

    • setNumTopFeatures

      public ChiSqSelector setNumTopFeatures(int value)
      Deprecated.
      Overrides:
      setNumTopFeatures in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setPercentile

      public ChiSqSelector setPercentile(double value)
      Deprecated.
      Overrides:
      setPercentile in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setFpr

      public ChiSqSelector setFpr(double value)
      Deprecated.
      Overrides:
      setFpr in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setFdr

      public ChiSqSelector setFdr(double value)
      Deprecated.
      Overrides:
      setFdr in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setFwe

      public ChiSqSelector setFwe(double value)
      Deprecated.
      Overrides:
      setFwe in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setSelectorType

      public ChiSqSelector setSelectorType(String value)
      Deprecated.
      Overrides:
      setSelectorType in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setFeaturesCol

      public ChiSqSelector setFeaturesCol(String value)
      Deprecated.
      Overrides:
      setFeaturesCol in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setOutputCol

      public ChiSqSelector setOutputCol(String value)
      Deprecated.
      Overrides:
      setOutputCol in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • setLabelCol

      public ChiSqSelector setLabelCol(String value)
      Deprecated.
      Overrides:
      setLabelCol in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>

    • fit

      public ChiSqSelectorModel fit(Dataset<?> dataset)
      Deprecated.
      Description copied from class: Estimator
      Fits a model to the input data.
      Overrides:
      fit in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>
      Parameters:
      dataset - (undocumented)
      Returns:
      (undocumented)

    • transformSchema

      public StructType transformSchema(StructType schema)
      Deprecated.
      Description copied from class: PipelineStage
      Check transform validity and derive the output schema from the input schema.

      We check validity for interactions between parameters during transformSchema and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by Param.validate().

      Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.

      Overrides:
      transformSchema in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>
      Parameters:
      schema - (undocumented)
      Returns:
      (undocumented)

    • copy

      public ChiSqSelector copy(ParamMap extra)
      Deprecated.
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Overrides:
      copy in class org.apache.spark.ml.feature.Selector<ChiSqSelectorModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)