Packages

  • package root
    Definition Classes
    root
  • package org
    Definition Classes
    root
  • package apache
    Definition Classes
    org
  • package spark

    Core Spark functionality.

    Core Spark functionality. org.apache.spark.SparkContext serves as the main entry point to Spark, while org.apache.spark.rdd.RDD is the data type representing a distributed collection, and provides most parallel operations.

    In addition, org.apache.spark.rdd.PairRDDFunctions contains operations available only on RDDs of key-value pairs, such as groupByKey and join; org.apache.spark.rdd.DoubleRDDFunctions contains operations available only on RDDs of Doubles; and org.apache.spark.rdd.SequenceFileRDDFunctions contains operations available on RDDs that can be saved as SequenceFiles. These operations are automatically available on any RDD of the right type (e.g. RDD[(Int, Int)] through implicit conversions.

    Java programmers should reference the org.apache.spark.api.java package for Spark programming APIs in Java.

    Classes and methods marked with Experimental are user-facing features which have not been officially adopted by the Spark project. These are subject to change or removal in minor releases.

    Classes and methods marked with Developer API are intended for advanced users want to extend Spark through lower level interfaces. These are subject to changes or removal in minor releases.

    Definition Classes
    apache
  • package mllib

    RDD-based machine learning APIs (in maintenance mode).

    RDD-based machine learning APIs (in maintenance mode).

    The spark.mllib package is in maintenance mode as of the Spark 2.0.0 release to encourage migration to the DataFrame-based APIs under the org.apache.spark.ml package. While in maintenance mode,

    • no new features in the RDD-based spark.mllib package will be accepted, unless they block implementing new features in the DataFrame-based spark.ml package;
    • bug fixes in the RDD-based APIs will still be accepted.

    The developers will continue adding more features to the DataFrame-based APIs in the 2.x series to reach feature parity with the RDD-based APIs. And once we reach feature parity, this package will be deprecated.

    Definition Classes
    spark
    See also

    SPARK-4591 to track the progress of feature parity

  • package linalg
    Definition Classes
    mllib
  • package distributed
    Definition Classes
    linalg
  • DenseMatrix
  • DenseVector
  • Matrices
  • Matrix
  • QRDecomposition
  • SingularValueDecomposition
  • SparseMatrix
  • SparseVector
  • Vector
  • VectorUDT
  • Vectors

class DenseMatrix extends Matrix

Column-major dense matrix. The entry values are stored in a single array of doubles with columns listed in sequence. For example, the following matrix

1.0 2.0
3.0 4.0
5.0 6.0

is stored as [1.0, 3.0, 5.0, 2.0, 4.0, 6.0].

Annotations
@Since("1.0.0") @SQLUserDefinedType()
Source
Matrices.scala
Linear Supertypes
Matrix, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DenseMatrix
  2. Matrix
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new DenseMatrix(numRows: Int, numCols: Int, values: Array[Double])

    Column-major dense matrix.

    Column-major dense matrix. The entry values are stored in a single array of doubles with columns listed in sequence. For example, the following matrix

    1.0 2.0
3.0 4.0
5.0 6.0

    is stored as [1.0, 3.0, 5.0, 2.0, 4.0, 6.0].

    numRows

    number of rows

    numCols

    number of columns

    values

    matrix entries in column major

    Annotations
    @Since("1.0.0")
  2. new DenseMatrix(numRows: Int, numCols: Int, values: Array[Double], isTransposed: Boolean)

    numRows

    number of rows

    numCols

    number of columns

    values

    matrix entries in column major if not transposed or in row major otherwise

    isTransposed

    whether the matrix is transposed. If true, values stores the matrix in row major.

    Annotations
    @Since("1.3.0")

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def apply(i: Int, j: Int): Double

    Gets the (i, j)-th element.

    Gets the (i, j)-th element.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.3.0")
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def asML: ml.linalg.DenseMatrix

    Convert this matrix to the new mllib-local representation.

    Convert this matrix to the new mllib-local representation. This does NOT copy the data; it copies references.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("2.0.0")
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  8. def colIter: Iterator[Vector]

    Returns an iterator of column vectors.

    Returns an iterator of column vectors. This operation could be expensive, depending on the underlying storage.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("2.0.0")
  9. def copy: DenseMatrix

    Get a deep copy of the matrix.

    Get a deep copy of the matrix.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.4.0")
  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(o: Any): Boolean
    Definition Classes
    DenseMatrix → AnyRef → Any
  12. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  13. def hashCode(): Int
    Definition Classes
    DenseMatrix → AnyRef → Any
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. val isTransposed: Boolean

    Flag that keeps track whether the matrix is transposed or not.

    Flag that keeps track whether the matrix is transposed or not. False by default.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.3.0")
  16. def multiply(y: Vector): DenseVector

    Convenience method for Matrix-Vector multiplication.

    Convenience method for Matrix-Vector multiplication.

    Definition Classes
    Matrix
    Annotations
    @Since("1.4.0")
  17. def multiply(y: DenseVector): DenseVector

    Convenience method for Matrix-DenseVector multiplication.

    Convenience method for Matrix-DenseVector multiplication. For binary compatibility.

    Definition Classes
    Matrix
    Annotations
    @Since("1.2.0")
  18. def multiply(y: DenseMatrix): DenseMatrix

    Convenience method for Matrix-DenseMatrix multiplication.

    Convenience method for Matrix-DenseMatrix multiplication.

    Definition Classes
    Matrix
    Annotations
    @Since("1.2.0")
  19. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  20. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  21. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  22. def numActives: Int

    Find the number of values stored explicitly.

    Find the number of values stored explicitly. These values can be zero as well.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.5.0")
  23. val numCols: Int

    Number of columns.

    Number of columns.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.0.0")
  24. def numNonzeros: Int

    Find the number of non-zero active values.

    Find the number of non-zero active values.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.5.0")
  25. val numRows: Int

    Number of rows.

    Number of rows.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.0.0")
  26. def rowIter: Iterator[Vector]

    Returns an iterator of row vectors.

    Returns an iterator of row vectors. This operation could be expensive, depending on the underlying storage.

    Definition Classes
    Matrix
    Annotations
    @Since("2.0.0")
  27. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  28. def toArray: Array[Double]

    Converts to a dense array in column major.

    Converts to a dense array in column major.

    Definition Classes
    Matrix
    Annotations
    @Since("1.0.0")
  29. def toSparse: SparseMatrix

    Generate a SparseMatrix from the given DenseMatrix.

    Generate a SparseMatrix from the given DenseMatrix. The new matrix will have isTransposed set to false.

    Annotations
    @Since("1.3.0")
  30. def toString(maxLines: Int, maxLineWidth: Int): String

    A human readable representation of the matrix with maximum lines and width

    A human readable representation of the matrix with maximum lines and width

    Definition Classes
    Matrix
    Annotations
    @Since("1.4.0")
  31. def toString(): String

    A human readable representation of the matrix

    A human readable representation of the matrix

    Definition Classes
    Matrix → AnyRef → Any
  32. def transpose: DenseMatrix

    Transpose the Matrix.

    Transpose the Matrix. Returns a new Matrix instance sharing the same underlying data.

    Definition Classes
    DenseMatrix → Matrix
    Annotations
    @Since("1.3.0")
  33. val values: Array[Double]
    Annotations
    @Since("1.0.0")
  34. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  35. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  36. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

Inherited from Matrix

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped