Packages

c

org.apache.spark.sql

MergeIntoWriter

class MergeIntoWriter[T] extends AnyRef

Experimental

MergeIntoWriter provides methods to define and execute merge actions based on specified conditions.

T

the type of data in the Dataset.

Annotations
@Experimental()
Source
MergeIntoWriter.scala
Since

4.0.0

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MergeIntoWriter
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Value Members

  1. def merge(): Unit

    Executes the merge operation.

  2. def whenMatched(condition: Column): WhenMatched[T]

    Initialize a WhenMatched action with a condition.

    Initialize a WhenMatched action with a condition.

    This WhenMatched action will be executed when a source row matches a target table row based on the merge condition and the specified condition is satisfied.

    This WhenMatched can be followed by one of the following merge actions:

    • updateAll: Update all the matched target table rows with source dataset rows.
    • update(Map): Update all the matched target table rows while changing only a subset of columns based on the provided assignment.
    • delete: Delete all target rows that have a match in the source table.
    condition

    a Column representing the condition to be evaluated for the action.

    returns

    a new WhenMatched object configured with the specified condition.

  3. def whenMatched(): WhenMatched[T]

    Initialize a WhenMatched action without any condition.

    Initialize a WhenMatched action without any condition.

    This WhenMatched action will be executed when a source row matches a target table row based on the merge condition.

    This WhenMatched can be followed by one of the following merge actions:

    • updateAll: Update all the matched target table rows with source dataset rows.
    • update(Map): Update all the matched target table rows while changing only a subset of columns based on the provided assignment.
    • delete: Delete all target rows that have a match in the source table.
    returns

    a new WhenMatched object.

  4. def whenNotMatched(condition: Column): WhenNotMatched[T]

    Initialize a WhenNotMatched action with a condition.

    Initialize a WhenNotMatched action with a condition.

    This WhenNotMatched action will be executed when a source row does not match any target row based on the merge condition and the specified condition is satisfied.

    This WhenNotMatched can be followed by one of the following merge actions:

    • insertAll: Insert all rows from the source that are not already in the target table.
    • insert(Map): Insert all rows from the source that are not already in the target table, with the specified columns based on the provided assignment.
    condition

    a Column representing the condition to be evaluated for the action.

    returns

    a new WhenNotMatched object configured with the specified condition.

  5. def whenNotMatched(): WhenNotMatched[T]

    Initialize a WhenNotMatched action without any condition.

    Initialize a WhenNotMatched action without any condition.

    This WhenNotMatched action will be executed when a source row does not match any target row based on the merge condition.

    This WhenNotMatched can be followed by one of the following merge actions:

    • insertAll: Insert all rows from the source that are not already in the target table.
    • insert(Map): Insert all rows from the source that are not already in the target table, with the specified columns based on the provided assignment.
    returns

    a new WhenNotMatched object.

  6. def whenNotMatchedBySource(condition: Column): WhenNotMatchedBySource[T]

    Initialize a WhenNotMatchedBySource action with a condition.

    Initialize a WhenNotMatchedBySource action with a condition.

    This WhenNotMatchedBySource action will be executed when a target row does not match any rows in the source table based on the merge condition and the specified condition is satisfied.

    This WhenNotMatchedBySource can be followed by one of the following merge actions:

    • updateAll: Update all the not matched target table rows with source dataset rows.
    • update(Map): Update all the not matched target table rows while changing only the specified columns based on the provided assignment.
    • delete: Delete all target rows that have no matches in the source table.
    condition

    a Column representing the condition to be evaluated for the action.

    returns

    a new WhenNotMatchedBySource object configured with the specified condition.

  7. def whenNotMatchedBySource(): WhenNotMatchedBySource[T]

    Initialize a WhenNotMatchedBySource action without any condition.

    Initialize a WhenNotMatchedBySource action without any condition.

    This WhenNotMatchedBySource action will be executed when a target row does not match any rows in the source table based on the merge condition.

    This WhenNotMatchedBySource can be followed by one of the following merge actions:

    • updateAll: Update all the not matched target table rows with source dataset rows.
    • update(Map): Update all the not matched target table rows while changing only the specified columns based on the provided assignment.
    • delete: Delete all target rows that have no matches in the source table.
    returns

    a new WhenNotMatchedBySource object.

  8. def withSchemaEvolution(): MergeIntoWriter[T]

    Enable automatic schema evolution for this merge operation.

    Enable automatic schema evolution for this merge operation.

    returns

    A MergeIntoWriter instance with schema evolution enabled.