Type Members

1.  sealed trait EventDetails extends AnyRef
2.  sealed trait EventLevel extends AnyRef
3.  case class FlowProgress(status: FlowStatus) extends EventDetails with Product with Serializable
4.  class FlowProgressEventLogger extends Logging

   This class should be used for all flow progress events logging, it controls the level at which events are logged.

   This class should be used for all flow progress events logging, it controls the level at which events are logged. It uses execution mode, flow name and previous flow statuses to infer the level at which an event is to be logged. Below is a more details description of how flow progress events for batch/streaming flows will be logged:

   For batch & streaming flows in triggered execution mode:

   • All flow progress events other than errors/warnings will be logged at INFO level (including flow progress events with metrics) and error/warning messages will be logged at their level.
5.  case class PipelineEvent(id: String, timestamp: Timestamp, origin: PipelineEventOrigin, level: EventLevel, message: String, details: EventDetails, error: Option[Throwable]) extends Product with Serializable

   An internal event that is emitted during the run of a pipeline.

   An internal event that is emitted during the run of a pipeline.

   id

   A globally unique id

   timestamp

   The time of the event

   origin

   Where the event originated from

   level

   Security level of the event

   message

   A user friendly description of the event

   details

   The details of the event

   error

   An error that occurred during the event

6.  case class PipelineEventOrigin(datasetName: Option[String], flowName: Option[String], sourceCodeLocation: Option[QueryOrigin]) extends Product with Serializable

   Describes where the event originated from

   Describes where the event originated from

   datasetName

   The name of the dataset

   flowName

   The name of the flow

   sourceCodeLocation

   The location of the source code

7.  case class RunProgress(state: RunState) extends EventDetails with Product with Serializable
8.  class StreamListener extends StreamingQueryListener with Logging

   A streaming listener that converts streaming events into pipeline events for the relevant flows.