Class PipelinesTableProperties
Object
org.apache.spark.sql.pipelines.graph.PipelinesTableProperties
Interface for validating and accessing Pipeline-specific table properties.
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic String
Prefix used for all table properties.static PipelineTableProperty<Object>
Whether the table should be reset when a Reset is triggered.validateAndCanonicalize
(scala.collection.immutable.Map<String, String> rawProps, scala.Function1<String, scala.runtime.BoxedUnit> warnFunction) Validates that all known pipeline properties are valid and can be parsed.
-
Constructor Details
-
PipelinesTableProperties
public PipelinesTableProperties()
-
-
Method Details
-
pipelinesPrefix
Prefix used for all table properties. -
resetAllowed
Whether the table should be reset when a Reset is triggered. -
validateAndCanonicalize
public static scala.collection.immutable.Map<String,String> validateAndCanonicalize(scala.collection.immutable.Map<String, String> rawProps, scala.Function1<String, scala.runtime.BoxedUnit> warnFunction) Validates that all known pipeline properties are valid and can be parsed. Also warn the user if they try to set an unknown pipelines property, or any property that looks suspiciously similar to a known pipeline property.- Parameters:
rawProps
- Raw table properties to validate and canonicalizewarnFunction
- Function to warn users of potential issues. Fatal errors are still thrown.- Returns:
- a map of table properties, with canonical-case keys for valid pipelines properties, and excluding invalid pipelines properties.
-