Package org.apache.spark.sql.pipelines.graph

Class PipelinesTableProperties

Object
org.apache.spark.sql.pipelines.graph.PipelinesTableProperties
public class PipelinesTableProperties extends Object
Interface for validating and accessing Pipeline-specific table properties.

  • Constructor Details

    • PipelinesTableProperties

      public PipelinesTableProperties()

  • Method Details

    • pipelinesPrefix

      public static String pipelinesPrefix()
      Prefix used for all table properties.

    • resetAllowed

      public static PipelineTableProperty<Object> resetAllowed()
      Whether the table should be reset when a Reset is triggered.

    • validateAndCanonicalize

      public static scala.collection.immutable.Map<String,String> validateAndCanonicalize(scala.collection.immutable.Map<String,String> rawProps, scala.Function1<String,scala.runtime.BoxedUnit> warnFunction)
      Validates that all known pipeline properties are valid and can be parsed. Also warn the user if they try to set an unknown pipelines property, or any property that looks suspiciously similar to a known pipeline property.

      Parameters:
      rawProps - Raw table properties to validate and canonicalize
      warnFunction - Function to warn users of potential issues. Fatal errors are still thrown.
      Returns:
      a map of table properties, with canonical-case keys for valid pipelines properties, and excluding invalid pipelines properties.