class IndexedRowMatrix extends DistributedMatrix
Represents a row-oriented org.apache.spark.mllib.linalg.distributed.DistributedMatrix with indexed rows.
- Annotations
- @Since( "1.0.0" )
- Source
- IndexedRowMatrix.scala
- Alphabetic
- By Inheritance
- IndexedRowMatrix
- DistributedMatrix
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
IndexedRowMatrix(rows: RDD[IndexedRow])
Alternative constructor leaving matrix dimensions to be determined automatically.
Alternative constructor leaving matrix dimensions to be determined automatically.
- Annotations
- @Since( "1.0.0" )
-
new
IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)
- rows
indexed rows of this matrix
- nRows
number of rows. A non-positive value means unknown, and then the number of rows will be determined by the max row index plus one.
- nCols
number of columns. A non-positive value means unknown, and then the number of columns will be determined by the size of the first row.
- Annotations
- @Since( "1.0.0" )
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
columnSimilarities(): CoordinateMatrix
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
- returns
An n x n sparse upper-triangular matrix of cosine similarities between columns of this matrix.
- Annotations
- @Since( "1.6.0" )
-
def
computeGramianMatrix(): Matrix
Computes the Gramian matrix
A^T A
.
Computes the Gramian matrix
A^T A
.
- Annotations
- @Since( "1.0.0" )
- Note
This cannot be computed on matrices with more than 65535 columns.
-
def
computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]
Computes the singular value decomposition of this IndexedRowMatrix.
Computes the singular value decomposition of this IndexedRowMatrix. Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
The cost and implementation of this method is identical to that in org.apache.spark.mllib.linalg.distributed.RowMatrix With the addition of indices.
At most k largest non-zero singular values and associated vectors are returned. If there are k such values, then the dimensions of the return will be:
U is an org.apache.spark.mllib.linalg.distributed.IndexedRowMatrix of size m x k that satisfies U'U = eye(k), s is a Vector of size k, holding the singular values in descending order, and V is a local Matrix of size n x k that satisfies V'V = eye(k).
- k
number of singular values to keep. We might return less than k if there are numerically zero singular values. See rCond.
- computeU
whether to compute U
- rCond
the reciprocal condition number. All singular values smaller than rCond * sigma(0) are treated as zero, where sigma(0) is the largest singular value.
- returns
SingularValueDecomposition(U, s, V)
- Annotations
- @Since( "1.0.0" )
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
multiply(B: Matrix): IndexedRowMatrix
Multiply this matrix by a local matrix on the right.
Multiply this matrix by a local matrix on the right.
- B
a local matrix whose number of rows must match the number of columns of this matrix
- returns
an IndexedRowMatrix representing the product, which preserves partitioning
- Annotations
- @Since( "1.0.0" )
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
numCols(): Long
Gets or computes the number of columns.
Gets or computes the number of columns.
- Definition Classes
- IndexedRowMatrix → DistributedMatrix
- Annotations
- @Since( "1.0.0" )
-
def
numRows(): Long
Gets or computes the number of rows.
Gets or computes the number of rows.
- Definition Classes
- IndexedRowMatrix → DistributedMatrix
- Annotations
- @Since( "1.0.0" )
-
val
rows: RDD[IndexedRow]
- Annotations
- @Since( "1.0.0" )
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix
Converts to BlockMatrix.
Converts to BlockMatrix. Blocks may be sparse or dense depending on the sparsity of the rows.
- rowsPerBlock
The number of rows of each block. The blocks at the bottom edge may have a smaller value. Must be an integer value greater than 0.
- colsPerBlock
The number of columns of each block. The blocks at the right edge may have a smaller value. Must be an integer value greater than 0.
- returns
- Annotations
- @Since( "1.3.0" )
-
def
toBlockMatrix(): BlockMatrix
Converts to BlockMatrix.
Converts to BlockMatrix. Creates blocks with size 1024 x 1024.
- Annotations
- @Since( "1.3.0" )
-
def
toCoordinateMatrix(): CoordinateMatrix
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
- Annotations
- @Since( "1.3.0" )
-
def
toRowMatrix(): RowMatrix
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
- Annotations
- @Since( "1.0.0" )
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()