Packages

c

org.apache.spark.resource

TaskResourceRequests

class TaskResourceRequests extends Serializable

A set of task resource requests. This is used in conjunction with the ResourceProfile to programmatically specify the resources needed for an RDD that will be applied at the stage level.

Annotations
@Evolving() @Since("3.1.0")
Source
TaskResourceRequests.scala
Linear Supertypes
Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TaskResourceRequests
  2. Serializable
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new TaskResourceRequests()

Value Members

  1. def addRequest(treq: TaskResourceRequest): TaskResourceRequests.this.type

    Add a certain TaskResourceRequest to the request set.

  2. def cpus(amount: Int): TaskResourceRequests.this.type

    Specify number of cpus per Task.

    Specify number of cpus per Task. This is a convenient API to add TaskResourceRequest for cpus.

    amount

    Number of cpus to allocate per Task.

  3. def requests: Map[String, TaskResourceRequest]

    Returns all the resource requests for the task.

  4. def requestsJMap: Map[String, TaskResourceRequest]

    (Java-specific) Returns all the resource requests for the task.

  5. def resource(resourceName: String, amount: Double): TaskResourceRequests.this.type

    Amount of a particular custom resource(GPU, FPGA, etc) to use.

    Amount of a particular custom resource(GPU, FPGA, etc) to use. This is a convenient API to add TaskResourceRequest for custom resources.

    resourceName

    Name of the resource.

    amount

    Amount requesting as a Double to support fractional resource requests. Valid values are less than or equal to 0.5 or whole numbers. This essentially lets you configure X number of tasks to run on a single resource, ie amount equals 0.5 translates into 2 tasks per resource address.

  6. def toString(): String
    Definition Classes
    TaskResourceRequests → AnyRef → Any