public class ReliableKafkaReceiver<K,V,U extends kafka.serializer.Decoder<?>,T extends kafka.serializer.Decoder<?>> extends Receiver<scala.Tuple2<K,V>> implements Logging
Note: ReliableKafkaReceiver will set auto.commit.enable to false to turn off automatic offset commit mechanism in Kafka consumer. So setting this configuration manually within kafkaParams will not take effect.
Constructor and Description |
---|
ReliableKafkaReceiver(scala.collection.Map<String,String> kafkaParams,
scala.collection.Map<String,Object> topics,
StorageLevel storageLevel,
scala.reflect.ClassTag<K> evidence$1,
scala.reflect.ClassTag<V> evidence$2,
scala.reflect.ClassTag<U> evidence$3,
scala.reflect.ClassTag<T> evidence$4) |
Modifier and Type | Method and Description |
---|---|
void |
onStart()
This method is called by the system when the receiver is started.
|
void |
onStop()
This method is called by the system when the receiver is stopped.
|
attachExecutor, executor_, isStarted, isStopped, preferredLocation, reportError, restart, restart, restart, setReceiverId, stop, stop, storageLevel, store, store, store, store, store, store, store, store, store, streamId
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public ReliableKafkaReceiver(scala.collection.Map<String,String> kafkaParams, scala.collection.Map<String,Object> topics, StorageLevel storageLevel, scala.reflect.ClassTag<K> evidence$1, scala.reflect.ClassTag<V> evidence$2, scala.reflect.ClassTag<U> evidence$3, scala.reflect.ClassTag<T> evidence$4)
public void onStart()
Receiver
store(data)
.
If there are errors in threads started here, then following options can be done
(i) reportError(...)
can be called to report the error to the driver.
The receiving of data will continue uninterrupted.
(ii) stop(...)
can be called to stop receiving data. This will call onStop()
to
clear up all resources allocated (threads, buffers, etc.) during onStart()
.
(iii) restart(...)
can be called to restart the receiver. This will call onStop()
immediately, and then onStart()
after a delay.