ReceivedBlockHandler represents how to handle the storage of blocks received by receivers.
ReceivedBlockHandler is a
private[streaming] trait. It comes with two methods:
storeBlock(blockId: StreamBlockId, receivedBlock: ReceivedBlock): ReceivedBlockStoreResultto store a received block as
cleanupOldBlocks(threshTime: Long)to clean up blocks older than
There are two implementations of
Read BlockManagerBasedBlockHandler in this document.
Read WriteAheadLogBasedBlockHandler in this document.
BlockManagerBasedBlockHandler is the default
ReceivedBlockHandler in Spark Streaming.
cleanupOldBlocks is not used as blocks are cleared by some other means (FIXME)
BlockManagerBasedStoreResult. It uses
BlockManager.putIterator to store