Package org.apache.spark.ml.feature

Class VarianceThresholdSelector

Object
org.apache.spark.ml.PipelineStage
org.apache.spark.ml.Estimator<VarianceThresholdSelectorModel>
org.apache.spark.ml.feature.VarianceThresholdSelector
All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, org.apache.spark.ml.feature.VarianceThresholdSelectorParams, Params, HasFeaturesCol, HasOutputCol, DefaultParamsWritable, Identifiable, MLWritable
public final class VarianceThresholdSelector extends Estimator<VarianceThresholdSelectorModel> implements org.apache.spark.ml.feature.VarianceThresholdSelectorParams, DefaultParamsWritable
Feature selector that removes all low-variance features. Features with a (sample) variance not greater than the threshold will be removed. The default is to keep all features with non-zero variance, i.e. remove the features that have the same value in all samples.
See Also:

  • Constructor Details

    • VarianceThresholdSelector

      public VarianceThresholdSelector(String uid)

    • VarianceThresholdSelector

      public VarianceThresholdSelector()

  • Method Details

    • load

      public static VarianceThresholdSelector load(String path)

    • read

      public static MLReader<T> read()

    • varianceThreshold

      public final DoubleParam varianceThreshold()
      Specified by:
      varianceThreshold in interface org.apache.spark.ml.feature.VarianceThresholdSelectorParams

    • outputCol

      public final Param<String> outputCol()
      Description copied from interface: HasOutputCol
      Param for output column name.
      Specified by:
      outputCol in interface HasOutputCol
      Returns:
      (undocumented)

    • featuresCol

      public final Param<String> featuresCol()
      Description copied from interface: HasFeaturesCol
      Param for features column name.
      Specified by:
      featuresCol in interface HasFeaturesCol
      Returns:
      (undocumented)

    • uid

      public String uid()
      Description copied from interface: Identifiable
      An immutable unique ID for the object and its derivatives.
      Specified by:
      uid in interface Identifiable
      Returns:
      (undocumented)

    • setVarianceThreshold

      public VarianceThresholdSelector setVarianceThreshold(double value)

    • setFeaturesCol

      public VarianceThresholdSelector setFeaturesCol(String value)

    • setOutputCol

      public VarianceThresholdSelector setOutputCol(String value)

    • fit

      public VarianceThresholdSelectorModel fit(Dataset<?> dataset)
      Description copied from class: Estimator
      Fits a model to the input data.
      Specified by:
      fit in class Estimator<VarianceThresholdSelectorModel>
      Parameters:
      dataset - (undocumented)
      Returns:
      (undocumented)

    • transformSchema

      public StructType transformSchema(StructType schema)
      Description copied from class: PipelineStage
      Check transform validity and derive the output schema from the input schema.

      We check validity for interactions between parameters during transformSchema and raise an exception if any parameter value is invalid. Parameter value checks which do not depend on other parameters are handled by Param.validate().

      Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.

      Specified by:
      transformSchema in class PipelineStage
      Parameters:
      schema - (undocumented)
      Returns:
      (undocumented)

    • copy

      public VarianceThresholdSelector copy(ParamMap extra)
      Description copied from interface: Params
      Creates a copy of this instance with the same UID and some extra params. Subclasses should implement this method and set the return type properly. See defaultCopy().
      Specified by:
      copy in interface Params
      Specified by:
      copy in class Estimator<VarianceThresholdSelectorModel>
      Parameters:
      extra - (undocumented)
      Returns:
      (undocumented)