class IndexedRowMatrix extends DistributedMatrix
Represents a row-oriented org.apache.spark.mllib.linalg.distributed.DistributedMatrix with indexed rows.
- Annotations
- @Since("1.0.0")
- Source
- IndexedRowMatrix.scala
- Alphabetic
- By Inheritance
- IndexedRowMatrix
- DistributedMatrix
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Instance Constructors
- new IndexedRowMatrix(rows: RDD[IndexedRow])
Alternative constructor leaving matrix dimensions to be determined automatically.
Alternative constructor leaving matrix dimensions to be determined automatically.
- Annotations
- @Since("1.0.0")
- new IndexedRowMatrix(rows: RDD[IndexedRow], nRows: Long, nCols: Int)
- rows
indexed rows of this matrix
- nRows
number of rows. A non-positive value means unknown, and then the number of rows will be determined by the max row index plus one.
- nCols
number of columns. A non-positive value means unknown, and then the number of columns will be determined by the size of the first row.
- Annotations
- @Since("1.0.0")
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- def columnSimilarities(): CoordinateMatrix
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
Compute all cosine similarities between columns of this matrix using the brute-force approach of computing normalized dot products.
- returns
An n x n sparse upper-triangular matrix of cosine similarities between columns of this matrix.
- Annotations
- @Since("1.6.0")
- def computeGramianMatrix(): Matrix
Computes the Gramian matrix
A^T A.Computes the Gramian matrix
A^T A.- Annotations
- @Since("1.0.0")
- Note
This cannot be computed on matrices with more than 65535 columns.
- def computeSVD(k: Int, computeU: Boolean = false, rCond: Double = 1e-9): SingularValueDecomposition[IndexedRowMatrix, Matrix]
Computes the singular value decomposition of this IndexedRowMatrix.
Computes the singular value decomposition of this IndexedRowMatrix. Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
The cost and implementation of this method is identical to that in org.apache.spark.mllib.linalg.distributed.RowMatrix With the addition of indices.
At most k largest non-zero singular values and associated vectors are returned. If there are k such values, then the dimensions of the return will be:
U is an org.apache.spark.mllib.linalg.distributed.IndexedRowMatrix of size m x k that satisfies U'U = eye(k), s is a Vector of size k, holding the singular values in descending order, and V is a local Matrix of size n x k that satisfies V'V = eye(k).
- k
number of singular values to keep. We might return less than k if there are numerically zero singular values. See rCond.
- computeU
whether to compute U
- rCond
the reciprocal condition number. All singular values smaller than rCond * sigma(0) are treated as zero, where sigma(0) is the largest singular value.
- returns
SingularValueDecomposition(U, s, V)
- Annotations
- @Since("1.0.0")
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def multiply(B: Matrix): IndexedRowMatrix
Multiply this matrix by a local matrix on the right.
Multiply this matrix by a local matrix on the right.
- B
a local matrix whose number of rows must match the number of columns of this matrix
- returns
an IndexedRowMatrix representing the product, which preserves partitioning
- Annotations
- @Since("1.0.0")
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- def numCols(): Long
Gets or computes the number of columns.
Gets or computes the number of columns.
- Definition Classes
- IndexedRowMatrix → DistributedMatrix
- Annotations
- @Since("1.0.0")
- def numRows(): Long
Gets or computes the number of rows.
Gets or computes the number of rows.
- Definition Classes
- IndexedRowMatrix → DistributedMatrix
- Annotations
- @Since("1.0.0")
- val rows: RDD[IndexedRow]
- Annotations
- @Since("1.0.0")
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toBlockMatrix(rowsPerBlock: Int, colsPerBlock: Int): BlockMatrix
Converts to BlockMatrix.
Converts to BlockMatrix. Blocks may be sparse or dense depending on the sparsity of the rows.
- rowsPerBlock
The number of rows of each block. The blocks at the bottom edge may have a smaller value. Must be an integer value greater than 0.
- colsPerBlock
The number of columns of each block. The blocks at the right edge may have a smaller value. Must be an integer value greater than 0.
- returns
- Annotations
- @Since("1.3.0")
- def toBlockMatrix(): BlockMatrix
Converts to BlockMatrix.
Converts to BlockMatrix. Creates blocks with size 1024 x 1024.
- Annotations
- @Since("1.3.0")
- def toCoordinateMatrix(): CoordinateMatrix
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
Converts this matrix to a org.apache.spark.mllib.linalg.distributed.CoordinateMatrix.
- Annotations
- @Since("1.3.0")
- def toRowMatrix(): RowMatrix
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
Drops row indices and converts this matrix to a org.apache.spark.mllib.linalg.distributed.RowMatrix.
- Annotations
- @Since("1.0.0")
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated
(Since version 9)