Package org.apache.spark.sql.connector.write

Interface DeltaWriter<T>

All Superinterfaces:
AutoCloseable, Closeable, DataWriter<T>
@Experimental public interface DeltaWriter<T> extends DataWriter<T>
A data writer returned by DeltaWriterFactory.createWriter(int, long) and is responsible for writing a delta of rows.
Since:
3.4.0

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    delete(T metadata, T id)
    Deletes a row.
    void
    insert(T row)
    Inserts a new row.
    default void
    reinsert(T metadata, T row)
    Reinserts a row with metadata.
    void
    update(T metadata, T id, T row)
    Updates a row.
    default void
    write(T row)
    Writes one record.

    Methods inherited from interface java.io.Closeable

    close

    Methods inherited from interface org.apache.spark.sql.connector.write.DataWriter

    abort, commit, currentMetricsValues, write, writeAll

  • Method Details

    • delete

      void delete(T metadata, T id) throws IOException
      Deletes a row.
      Parameters:
      metadata - values for metadata columns that were projected but are not part of the row ID
      id - a row ID to delete
      Throws:
      IOException - if failure happens during disk/network IO like writing files

    • update

      void update(T metadata, T id, T row) throws IOException
      Updates a row.
      Parameters:
      metadata - values for metadata columns that were projected but are not part of the row ID
      id - a row ID to update
      row - a row with updated values
      Throws:
      IOException - if failure happens during disk/network IO like writing files

    • reinsert

      default void reinsert(T metadata, T row) throws IOException
      Reinserts a row with metadata.

      This method handles the insert portion of updated rows split into deletes and inserts.

      Parameters:
      metadata - values for metadata columns
      row - a row to reinsert
      Throws:
      IOException - if failure happens during disk/network IO like writing files
      Since:
      4.0.0

    • insert

      void insert(T row) throws IOException
      Inserts a new row.
      Parameters:
      row - a row to insert
      Throws:
      IOException - if failure happens during disk/network IO like writing files

    • write

      default void write(T row) throws IOException
      Description copied from interface: DataWriter
      Writes one record.

      If this method fails (by throwing an exception), DataWriter.abort() will be called and this data writer is considered to have been failed.

      Specified by:
      write in interface DataWriter<T>
      Throws:
      IOException - if failure happens during disk/network IO like writing files.