compute(split: Partition, context: TaskContext): Iterator[T]BlockRDD
BlockRDD is an RDD that is created when Spark Streaming’s ReceiverInputDStream is requested to compute and createBlockRDD.
Spark Streaming calls BlockRDD.removeBlocks() while clearing metadata.
|
Note
|
It appears that BlockRDD is used in Spark Streaming exclusively.
|
Computing Partition (in TaskContext) — compute Method
|
Note
|
compute is part of RDD Contract to compute a partition (in a TaskContext).
|
compute…FIXME
getPartitions Method
getPartitions: Array[Partition]|
Note
|
getPartitions is part of RDD Contract to…FIXME.
|
getPartitions…FIXME
getPreferredLocations Method
getPreferredLocations(split: Partition): Seq[String]|
Note
|
getPreferredLocations is part of RDD Contract to…FIXME.
|
getPreferredLocations…FIXME
Creating BlockRDD Instance
BlockRDD takes the following when created:
-
Collection of BlockIds
BlockRDD initializes the internal registries and counters.