Class Pipeline.SharedReadWrite$

Object
org.apache.spark.ml.Pipeline.SharedReadWrite$
Enclosing class:
Pipeline

public static class Pipeline.SharedReadWrite$ extends Object
Methods for MLReader and MLWriter shared between Pipeline and PipelineModel
  • Field Details

    • MODULE$

      public static final Pipeline.SharedReadWrite$ MODULE$
      Static reference to the singleton instance of this Scala object.
  • Constructor Details

    • SharedReadWrite$

      public SharedReadWrite$()
  • Method Details

    • validateStages

      public void validateStages(PipelineStage[] stages)
      Check that all stages are Writable
    • saveImpl

      public void saveImpl(Params instance, PipelineStage[] stages, SparkContext sc, String path)
      Deprecated.
      use saveImpl with SparkSession. Since 4.0.0.
      Save metadata and stages for a Pipeline or PipelineModel - save metadata to path/metadata - save stages to stages/IDX_UID
      Parameters:
      instance - (undocumented)
      stages - (undocumented)
      sc - (undocumented)
      path - (undocumented)
    • saveImpl

      public void saveImpl(Params instance, PipelineStage[] stages, SparkSession spark, String path)
    • load

      public scala.Tuple2<String,PipelineStage[]> load(String expectedClassName, SparkContext sc, String path)
      Deprecated.
      use load with SparkSession. Since 4.0.0.
      Load metadata and stages for a Pipeline or PipelineModel
      Parameters:
      expectedClassName - (undocumented)
      sc - (undocumented)
      path - (undocumented)
      Returns:
      (UID, list of stages)
    • load

      public scala.Tuple2<String,PipelineStage[]> load(String expectedClassName, SparkSession spark, String path)
    • getStagePath

      public String getStagePath(String stageUid, int stageIdx, int numStages, String stagesDir)
      Get path for saving the given stage.