BlockMatrix (Spark 3.5.3 JavaDoc)

Object
- org.apache.spark.mllib.linalg.distributed.BlockMatrix

All Implemented Interfaces:

java.io.Serializable, org.apache.spark.internal.Logging, DistributedMatrix
```
public class BlockMatrix
extends Object
implements DistributedMatrix, org.apache.spark.internal.Logging
```
Represents a distributed matrix in blocks of local matrices.
param: blocks The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix. If multiple blocks with the same index exist, the results for operations like add and multiply will be unpredictable. param: rowsPerBlock Number of rows that make up each block. The blocks forming the final rows are not required to have the given number of rows param: colsPerBlock Number of columns that make up each block. The blocks forming the final columns are not required to have the given number of columns param: nRows Number of rows of this matrix. If the supplied value is less than or equal to zero, the number of rows will be calculated when numRows is invoked. param: nCols Number of columns of this matrix. If the supplied value is less than or equal to zero, the number of columns will be calculated when numCols is invoked.

See Also:

Serialized Form

Nested Class Summary
- Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging
  org.apache.spark.internal.Logging.SparkShellLoggingFilter

Constructor Summary

Constructors
Constructor and Description
`BlockMatrix(RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>> blocks, int rowsPerBlock, int colsPerBlock)` Alternate constructor for BlockMatrix without the input of the number of rows and columns.
`BlockMatrix(RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>> blocks, int rowsPerBlock, int colsPerBlock, long nRows, long nCols)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`BlockMatrix`	`add(BlockMatrix other)` Adds the given block matrix `other` to `this` block matrix: `this + other`.
`RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>>`	`blocks()`
`BlockMatrix`	`cache()` Caches the underlying RDD.
`int`	`colsPerBlock()`
`BlockMatrix`	`multiply(BlockMatrix other)` Left multiplies this `BlockMatrix` to `other`, another `BlockMatrix`.
`BlockMatrix`	`multiply(BlockMatrix other, int numMidDimSplits)` Left multiplies this `BlockMatrix` to `other`, another `BlockMatrix`.
`int`	`numColBlocks()`
`long`	`numCols()` Gets or computes the number of columns.
`int`	`numRowBlocks()`
`long`	`numRows()` Gets or computes the number of rows.
`BlockMatrix`	`persist(StorageLevel storageLevel)` Persists the underlying RDD with the specified storage level.
`int`	`rowsPerBlock()`
`BlockMatrix`	`subtract(BlockMatrix other)` Subtracts the given block matrix `other` from `this` block matrix: `this - other`.
`CoordinateMatrix`	`toCoordinateMatrix()` Converts to CoordinateMatrix.
`IndexedRowMatrix`	`toIndexedRowMatrix()` Converts to IndexedRowMatrix.
`Matrix`	`toLocalMatrix()` Collect the distributed matrix on the driver as a `DenseMatrix`.
`BlockMatrix`	`transpose()` Transpose this `BlockMatrix`.
`void`	`validate()` Validates the block matrix info against the matrix data (`blocks`) and throws an exception if any error is found.

Methods inherited from class Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging
$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize

- Constructor Detail
  - BlockMatrix
```
public BlockMatrix(RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>> blocks,
                   int rowsPerBlock,
                   int colsPerBlock,
                   long nRows,
                   long nCols)
```
  - BlockMatrix
```
public BlockMatrix(RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>> blocks,
                   int rowsPerBlock,
                   int colsPerBlock)
```
    Alternate constructor for BlockMatrix without the input of the number of rows and columns.
    
    Parameters:
    
    blocks - The RDD of sub-matrix blocks ((blockRowIndex, blockColIndex), sub-matrix) that form this distributed matrix. If multiple blocks with the same index exist, the results for operations like add and multiply will be unpredictable.
    
    rowsPerBlock - Number of rows that make up each block. The blocks forming the final rows are not required to have the given number of rows
    
    colsPerBlock - Number of columns that make up each block. The blocks forming the final columns are not required to have the given number of columns
- Method Detail
  - add
```
public BlockMatrix add(BlockMatrix other)
```
    Adds the given block matrix other to this block matrix: this + other. The matrices must have the same size and matching rowsPerBlock and colsPerBlock values. If one of the blocks that are being added are instances of SparseMatrix, the resulting sub matrix will also be a SparseMatrix, even if it is being added to a DenseMatrix. If two dense matrices are added, the output will also be a DenseMatrix.
    
    Parameters:
    
    other - (undocumented)
    
    Returns:
    
    (undocumented)
  - blocks
```
public RDD<scala.Tuple2<scala.Tuple2<Object,Object>,Matrix>> blocks()
```
  - cache
```
public BlockMatrix cache()
```
    Caches the underlying RDD.
  - colsPerBlock
```
public int colsPerBlock()
```
  - multiply
```
public BlockMatrix multiply(BlockMatrix other)
```
    Left multiplies this BlockMatrix to other, another BlockMatrix. The colsPerBlock of this matrix must equal the rowsPerBlock of other. If other contains SparseMatrix, they will have to be converted to a DenseMatrix. The output BlockMatrix will only consist of blocks of DenseMatrix. This may cause some performance issues until support for multiplying two sparse matrices is added.
    
    Parameters:
    
    other - (undocumented)
    
    Returns:
    
    (undocumented)
    
    Note:
    
    The behavior of multiply has changed in 1.6.0. multiply used to throw an error when there were blocks with duplicate indices. Now, the blocks with duplicate indices will be added with each other.
  - multiply
```
public BlockMatrix multiply(BlockMatrix other,
                            int numMidDimSplits)
```
    Left multiplies this BlockMatrix to other, another BlockMatrix. The colsPerBlock of this matrix must equal the rowsPerBlock of other. If other contains SparseMatrix, they will have to be converted to a DenseMatrix. The output BlockMatrix will only consist of blocks of DenseMatrix. This may cause some performance issues until support for multiplying two sparse matrices is added. Blocks with duplicate indices will be added with each other.
    
    Parameters:
    
    other - Matrix B in A * B = C
    
    numMidDimSplits - Number of splits to cut on the middle dimension when doing multiplication. For example, when multiplying a Matrix A of size m x n with Matrix B of size n x k, this parameter configures the parallelism to use when grouping the matrices. The parallelism will increase from m x k to m x k x numMidDimSplits, which in some cases also reduces total shuffled data.
    
    Returns:
    
    (undocumented)
  - numColBlocks
```
public int numColBlocks()
```
  - numCols
```
public long numCols()
```
    Description copied from interface: DistributedMatrix
    
    Gets or computes the number of columns.
    
    Specified by:
    
    numCols in interface DistributedMatrix
  - numRowBlocks
```
public int numRowBlocks()
```
  - numRows
```
public long numRows()
```
    Description copied from interface: DistributedMatrix
    
    Gets or computes the number of rows.
    
    Specified by:
    
    numRows in interface DistributedMatrix
  - persist
```
public BlockMatrix persist(StorageLevel storageLevel)
```
    Persists the underlying RDD with the specified storage level.
  - rowsPerBlock
```
public int rowsPerBlock()
```
  - subtract
```
public BlockMatrix subtract(BlockMatrix other)
```
    Subtracts the given block matrix other from this block matrix: this - other. The matrices must have the same size and matching rowsPerBlock and colsPerBlock values. If one of the blocks that are being subtracted are instances of SparseMatrix, the resulting sub matrix will also be a SparseMatrix, even if it is being subtracted from a DenseMatrix. If two dense matrices are subtracted, the output will also be a DenseMatrix.
    
    Parameters:
    
    other - (undocumented)
    
    Returns:
    
    (undocumented)
  - toCoordinateMatrix
```
public CoordinateMatrix toCoordinateMatrix()
```
    Converts to CoordinateMatrix.
  - toIndexedRowMatrix
```
public IndexedRowMatrix toIndexedRowMatrix()
```
    Converts to IndexedRowMatrix. The number of columns must be within the integer range.
  - toLocalMatrix
```
public Matrix toLocalMatrix()
```
    Collect the distributed matrix on the driver as a DenseMatrix.
    
    Returns:
    
    (undocumented)
  - transpose
```
public BlockMatrix transpose()
```
    Transpose this BlockMatrix. Returns a new BlockMatrix instance sharing the same underlying data. Is a lazy operation.
    
    Returns:
    
    (undocumented)
  - validate
```
public void validate()
```
    Validates the block matrix info against the matrix data (blocks) and throws an exception if any error is found.

Class BlockMatrix

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

Constructor Summary

Method Summary

Methods inherited from class Object

Methods inherited from interface org.apache.spark.internal.Logging

Constructor Detail

BlockMatrix

BlockMatrix

Method Detail

add

blocks

cache

colsPerBlock

multiply

multiply

numColBlocks

numCols

numRowBlocks

numRows

persist

rowsPerBlock

subtract

toCoordinateMatrix

toIndexedRowMatrix

toLocalMatrix

transpose

validate