Packages

package scheduler

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. case class BatchInfo(batchTime: Time, streamIdToInputInfo: Map[Int, StreamInputInfo], submissionTime: Long, processingStartTime: Option[Long], processingEndTime: Option[Long], outputOperationInfos: Map[Int, OutputOperationInfo]) extends Product with Serializable

    :: DeveloperApi :: Class having information on completed batches.

    :: DeveloperApi :: Class having information on completed batches.

    batchTime

    Time of the batch

    streamIdToInputInfo

    A map of input stream id to its input info

    submissionTime

    Clock time of when jobs of this batch was submitted to the streaming scheduler queue

    processingStartTime

    Clock time of when the first job of this batch started processing

    processingEndTime

    Clock time of when the last job of this batch finished processing

    outputOperationInfos

    The output operations in this batch

    Annotations
    @DeveloperApi()
  2. case class OutputOperationInfo(batchTime: Time, id: Int, name: String, description: String, startTime: Option[Long], endTime: Option[Long], failureReason: Option[String]) extends Product with Serializable

    :: DeveloperApi :: Class having information on output operations.

    :: DeveloperApi :: Class having information on output operations.

    batchTime

    Time of the batch

    id

    Id of this output operation. Different output operations have different ids in a batch.

    name

    The name of this output operation.

    description

    The description of this output operation.

    startTime

    Clock time of when the output operation started processing

    endTime

    Clock time of when the output operation started processing

    failureReason

    Failure reason if this output operation fails

    Annotations
    @DeveloperApi()
  3. case class ReceiverInfo(streamId: Int, name: String, active: Boolean, location: String, executorId: String, lastErrorMessage: String = "", lastError: String = "", lastErrorTime: Long = -1L) extends Product with Serializable

    :: DeveloperApi :: Class having information about a receiver

    :: DeveloperApi :: Class having information about a receiver

    Annotations
    @DeveloperApi()
  4. class StatsReportListener extends StreamingListener

    :: DeveloperApi :: A simple StreamingListener that logs summary statistics across Spark Streaming batches

    :: DeveloperApi :: A simple StreamingListener that logs summary statistics across Spark Streaming batches

    Annotations
    @DeveloperApi()
  5. case class StreamInputInfo(inputStreamId: Int, numRecords: Long, metadata: Map[String, Any] = Map.empty) extends Product with Serializable

    :: DeveloperApi :: Track the information of input stream at specified batch time.

    :: DeveloperApi :: Track the information of input stream at specified batch time.

    inputStreamId

    the input stream id

    numRecords

    the number of records in a batch

    metadata

    metadata for this batch. It should contain at least one standard field named "Description" which maps to the content that will be shown in the UI.

    Annotations
    @DeveloperApi()
  6. trait StreamingListener extends AnyRef

    :: DeveloperApi :: A listener interface for receiving information about an ongoing streaming computation.

    :: DeveloperApi :: A listener interface for receiving information about an ongoing streaming computation.

    Annotations
    @DeveloperApi()
  7. case class StreamingListenerBatchCompleted(batchInfo: BatchInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  8. case class StreamingListenerBatchStarted(batchInfo: BatchInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  9. case class StreamingListenerBatchSubmitted(batchInfo: BatchInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  10. sealed trait StreamingListenerEvent extends AnyRef

    :: DeveloperApi :: Base trait for events related to StreamingListener

    :: DeveloperApi :: Base trait for events related to StreamingListener

    Annotations
    @DeveloperApi()
  11. case class StreamingListenerOutputOperationCompleted(outputOperationInfo: OutputOperationInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  12. case class StreamingListenerOutputOperationStarted(outputOperationInfo: OutputOperationInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  13. case class StreamingListenerReceiverError(receiverInfo: ReceiverInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  14. case class StreamingListenerReceiverStarted(receiverInfo: ReceiverInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  15. case class StreamingListenerReceiverStopped(receiverInfo: ReceiverInfo) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()
  16. case class StreamingListenerStreamingStarted(time: Long) extends StreamingListenerEvent with Product with Serializable
    Annotations
    @DeveloperApi()

Value Members

  1. object RateController extends Serializable
  2. object StreamInputInfo extends Serializable
    Annotations
    @DeveloperApi()

Ungrouped