Package org.apache.spark.ml

Class Pipeline.SharedReadWrite$

Object

org.apache.spark.ml.Pipeline.SharedReadWrite$

Enclosing class:

Pipeline

public static class Pipeline.SharedReadWrite$
extends Object

Methods for MLReader and MLWriter shared between Pipeline and PipelineModel

Field Summary

Fields

Modifier and Type	Field	Description
static final Pipeline.SharedReadWrite$	MODULE$	Static reference to the singleton instance of this Scala object.

Constructor Summary

Constructors

Constructor	Description
SharedReadWrite$()

Method Summary

Modifier and Type	Method	Description
String	getStagePath(String stageUid, int stageIdx, int numStages, String stagesDir)	Get path for saving the given stage.
scala.Tuple2<String,PipelineStage[]>	load(String expectedClassName, SparkContext sc, String path)	Deprecated. use load with SparkSession.
scala.Tuple2<String,PipelineStage[]>	load(String expectedClassName, SparkSession spark, String path)
void	saveImpl(Params instance, PipelineStage[] stages, SparkContext sc, String path)	Deprecated. use saveImpl with SparkSession.
void	saveImpl(Params instance, PipelineStage[] stages, SparkSession spark, String path)
void	validateStages(PipelineStage[] stages)	Check that all stages are Writable

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

MODULE$

public static final Pipeline.SharedReadWrite$ MODULE$

Static reference to the singleton instance of this Scala object.

Constructor Details

SharedReadWrite$

public SharedReadWrite$()

Method Details

validateStages

public void validateStages(PipelineStage[] stages)

Check that all stages are Writable

saveImpl

public void saveImpl(Params instance,
 PipelineStage[] stages,
 SparkContext sc,
 String path)

Deprecated.

use saveImpl with SparkSession. Since 4.0.0.

Save metadata and stages for a Pipeline or PipelineModel - save metadata to path/metadata - save stages to stages/IDX_UID

Parameters:

instance - (undocumented)

stages - (undocumented)

sc - (undocumented)

path - (undocumented)

saveImpl

public void saveImpl(Params instance,
 PipelineStage[] stages,
 SparkSession spark,
 String path)

load

public scala.Tuple2<String,PipelineStage[]> load(String expectedClassName,
 SparkContext sc,
 String path)

Deprecated.

use load with SparkSession. Since 4.0.0.

Load metadata and stages for a Pipeline or PipelineModel

Parameters:

expectedClassName - (undocumented)

sc - (undocumented)

path - (undocumented)

Returns:

(UID, list of stages)

load

public scala.Tuple2<String,PipelineStage[]> load(String expectedClassName,
 SparkSession spark,
 String path)

getStagePath

public String getStagePath(String stageUid,
 int stageIdx,
 int numStages,
 String stagesDir)

Get path for saving the given stage.