BaseReadWrite¶

class pyspark.ml.util.BaseReadWrite[source]¶

Base class for MLWriter and MLReader. Stores information about the SparkContext and SparkSession.

New in version 2.3.0.

Methods

session(sparkSession)

Sets the Spark Session to use for saving/loading.

Attributes

`sc`	Returns the underlying SparkContext.
`sparkSession`	Returns the user-specified Spark Session or the default.

Methods Documentation

session(sparkSession: pyspark.sql.session.SparkSession) → RW[source]¶: Sets the Spark Session to use for saving/loading.

Attributes Documentation

DeepspeedTorchDistributor

DefaultParamsReadable