Package org.apache.spark.resource

Class TaskResourceRequests

Object
org.apache.spark.resource.TaskResourceRequests
All Implemented Interfaces:
Serializable, scala.Serializable
public class TaskResourceRequests extends Object implements scala.Serializable
A set of task resource requests. This is used in conjunction with the ResourceProfile to programmatically specify the resources needed for an RDD that will be applied at the stage level.
See Also:

  • Constructor Details

    • TaskResourceRequests

      public TaskResourceRequests()

  • Method Details

    • addRequest

      public TaskResourceRequests addRequest(TaskResourceRequest treq)
      Add a certain TaskResourceRequest to the request set.
      Parameters:
      treq - (undocumented)
      Returns:
      (undocumented)

    • cpus

      public TaskResourceRequests cpus(int amount)
      Specify number of cpus per Task. This is a convenient API to add TaskResourceRequest for cpus.

      Parameters:
      amount - Number of cpus to allocate per Task.
      Returns:
      (undocumented)

    • requests

      public scala.collection.immutable.Map<String,TaskResourceRequest> requests()
      Returns all the resource requests for the task.
      Returns:
      (undocumented)

    • requestsJMap

      public Map<String,TaskResourceRequest> requestsJMap()
      (Java-specific) Returns all the resource requests for the task.
      Returns:
      (undocumented)

    • resource

      public TaskResourceRequests resource(String resourceName, double amount)
      Amount of a particular custom resource(GPU, FPGA, etc) to use. This is a convenient API to add TaskResourceRequest for custom resources.

      Parameters:
      resourceName - Name of the resource.
      amount - Amount requesting as a Double to support fractional resource requests. Valid values are less than or equal to 0.5 or whole numbers. This essentially lets you configure X number of tasks to run on a single resource, ie amount equals 0.5 translates into 2 tasks per resource address.
      Returns:
      (undocumented)

    • toString

      public String toString()
      Overrides:
      toString in class Object