Package org.apache.spark.sql

Class MergeIntoWriter<T>

Object
org.apache.spark.sql.MergeIntoWriter<T>
public abstract class MergeIntoWriter<T> extends Object
MergeIntoWriter provides methods to define and execute merge actions based on specified conditions.

Please note that schema evolution is disabled by default.

Since:
4.0.0

  • Constructor Details

    • MergeIntoWriter

      public MergeIntoWriter()

  • Method Details

    • merge

      public abstract void merge()
      Executes the merge operation.

    • whenMatched

      public WhenMatched<T> whenMatched()
      Initialize a WhenMatched action without any condition.

      This WhenMatched action will be executed when a source row matches a target table row based on the merge condition.

      This WhenMatched can be followed by one of the following merge actions: - updateAll: Update all the matched target table rows with source dataset rows. - update(Map): Update all the matched target table rows while changing only a subset of columns based on the provided assignment. - delete: Delete all target rows that have a match in the source table.

      Returns:
      a new WhenMatched object.

    • whenMatched

      public WhenMatched<T> whenMatched(Column condition)
      Initialize a WhenMatched action with a condition.

      This WhenMatched action will be executed when a source row matches a target table row based on the merge condition and the specified condition is satisfied.

      This WhenMatched can be followed by one of the following merge actions: - updateAll: Update all the matched target table rows with source dataset rows. - update(Map): Update all the matched target table rows while changing only a subset of columns based on the provided assignment. - delete: Delete all target rows that have a match in the source table.

      Parameters:
      condition - a Column representing the condition to be evaluated for the action.
      Returns:
      a new WhenMatched object configured with the specified condition.

    • whenNotMatched

      public WhenNotMatched<T> whenNotMatched()
      Initialize a WhenNotMatched action without any condition.

      This WhenNotMatched action will be executed when a source row does not match any target row based on the merge condition.

      This WhenNotMatched can be followed by one of the following merge actions: - insertAll: Insert all rows from the source that are not already in the target table. - insert(Map): Insert all rows from the source that are not already in the target table, with the specified columns based on the provided assignment.

      Returns:
      a new WhenNotMatched object.

    • whenNotMatched

      public WhenNotMatched<T> whenNotMatched(Column condition)
      Initialize a WhenNotMatched action with a condition.

      This WhenNotMatched action will be executed when a source row does not match any target row based on the merge condition and the specified condition is satisfied.

      This WhenNotMatched can be followed by one of the following merge actions: - insertAll: Insert all rows from the source that are not already in the target table. - insert(Map): Insert all rows from the source that are not already in the target table, with the specified columns based on the provided assignment.

      Parameters:
      condition - a Column representing the condition to be evaluated for the action.
      Returns:
      a new WhenNotMatched object configured with the specified condition.

    • whenNotMatchedBySource

      public WhenNotMatchedBySource<T> whenNotMatchedBySource()
      Initialize a WhenNotMatchedBySource action without any condition.

      This WhenNotMatchedBySource action will be executed when a target row does not match any rows in the source table based on the merge condition.

      This WhenNotMatchedBySource can be followed by one of the following merge actions: - updateAll: Update all the not matched target table rows with source dataset rows. - update(Map): Update all the not matched target table rows while changing only the specified columns based on the provided assignment. - delete: Delete all target rows that have no matches in the source table.

      Returns:
      a new WhenNotMatchedBySource object.

    • whenNotMatchedBySource

      public WhenNotMatchedBySource<T> whenNotMatchedBySource(Column condition)
      Initialize a WhenNotMatchedBySource action with a condition.

      This WhenNotMatchedBySource action will be executed when a target row does not match any rows in the source table based on the merge condition and the specified condition is satisfied.

      This WhenNotMatchedBySource can be followed by one of the following merge actions: - updateAll: Update all the not matched target table rows with source dataset rows. - update(Map): Update all the not matched target table rows while changing only the specified columns based on the provided assignment. - delete: Delete all target rows that have no matches in the source table.

      Parameters:
      condition - a Column representing the condition to be evaluated for the action.
      Returns:
      a new WhenNotMatchedBySource object configured with the specified condition.

    • withSchemaEvolution

      public MergeIntoWriter<T> withSchemaEvolution()
      Enable automatic schema evolution for this merge operation.

      Returns:
      A MergeIntoWriter instance with schema evolution enabled.