def getReceiver(): Receiver[T]
ReceiverInputDStreams - Input Streams with Receivers
|Receiver input streams run receivers as long-running tasks that occupy a core per stream.|
ReceiverInputDStream abstract class defines the following abstract method that custom implementations use to create receivers:
The receiver is then sent to and run on workers (when ReceiverTracker is started).
A fine example of a very minimalistic yet still useful implementation of
ReceiverRateController when spark.streaming.backpressure.enabled is enabled.
Read ReceiverTrackerEndpoint.startReceiver for more details.
The source code of
ReceiverInputDStream is here at GitHub.
compute(validTime: Time): Option[RDD[T]] method (from DStream) uses start time of DStreamGraph, i.e. the start time of StreamingContext, to check whether
validTime input parameter is really valid.
If the time to generate RDDs (
validTime) is earlier than the start time of StreamingContext, an empty
BlockRDD is generated.
Otherwise, ReceiverTracker is requested for all the blocks that have been allocated to this stream for this batch (using
If all BlockIds have
WriteAheadLogBackedBlockRDD is generated. Otherwise, a