trait DeltaWriter[T] extends DataWriter[T]
A data writer returned by long) and is
responsible for writing a delta of rows.
- Annotations
- @Experimental()
- Source
- DeltaWriter.java
- Since
3.4.0
- Alphabetic
- By Inheritance
- DeltaWriter
- DataWriter
- Closeable
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def abort(): Unit
Aborts this writer if it is failed.
Aborts this writer if it is failed. Implementations should clean up the data for already written records.
This method will only be called if there is one record failed to write, or
#commit()failed.If this method fails(by throwing an exception), the underlying data source may have garbage that need to be cleaned by
BatchWrite#abort(WriterCommitMessage[])or manually, but these garbage should not be visible to data source readers.- Definition Classes
- DataWriter
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files.
- abstract def close(): Unit
- Definition Classes
- Closeable → AutoCloseable
- Annotations
- @throws(classOf[java.io.IOException])
- abstract def commit(): WriterCommitMessage
Commits this writer after all records are written successfully, returns a commit message which will be sent back to driver side and passed to
BatchWrite#commit(WriterCommitMessage[]).Commits this writer after all records are written successfully, returns a commit message which will be sent back to driver side and passed to
BatchWrite#commit(WriterCommitMessage[]).The written data should only be visible to data source readers after
BatchWrite#commit(WriterCommitMessage[])succeeds, which means this method should still "hide" the written data and ask theBatchWriteat driver side to do the final commit viaWriterCommitMessage.If this method fails (by throwing an exception),
#abort()will be called and this data writer is considered to have been failed.- Definition Classes
- DataWriter
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files.
- abstract def delete(metadata: T, id: T): Unit
Deletes a row.
Deletes a row.
- metadata
values for metadata columns that were projected but are not part of the row ID
- id
a row ID to delete
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files
- abstract def insert(row: T): Unit
Inserts a new row.
Inserts a new row.
- row
a row to insert
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files
- abstract def update(metadata: T, id: T, row: T): Unit
Updates a row.
Updates a row.
- metadata
values for metadata columns that were projected but are not part of the row ID
- id
a row ID to update
- row
a row with updated values
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files
Concrete Value Members
- def currentMetricsValues(): Array[CustomTaskMetric]
Returns an array of custom task metrics.
Returns an array of custom task metrics. By default it returns empty array. Note that it is not recommended to put heavy logic in this method as it may affect writing performance.
- Definition Classes
- DataWriter
- def write(row: T): Unit
Writes one record.
Writes one record.
If this method fails (by throwing an exception),
#abort()will be called and this data writer is considered to have been failed.- Definition Classes
- DeltaWriter → DataWriter
- Annotations
- @Override()
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files.
- def writeAll(records: Iterator[T]): Unit
Writes all records provided by the given iterator.
Writes all records provided by the given iterator. By default, it calls the
#writemethod for each record in the iterator.If this method fails (by throwing an exception),
#abort()will be called and this data writer is considered to have been failed.- Definition Classes
- DataWriter
- Since
4.0.0
- Exceptions thrown
IOExceptionif failure happens during disk/network IO like writing files.