BlockRDD

BlockRDD is an RDD that is created when Spark Streaming’s ReceiverInputDStream is requested to compute and createBlockRDD.

Spark Streaming calls BlockRDD.removeBlocks() while clearing metadata.

Note
It appears that BlockRDD is used in Spark Streaming exclusively.

Computing Partition (in TaskContext) — compute Method

compute(split: Partition, context: TaskContext): Iterator[T]
Note
compute is part of RDD Contract to compute a partition (in a TaskContext).

compute…​FIXME

getPartitions Method

getPartitions: Array[Partition]
Note
getPartitions is part of RDD Contract to…​FIXME.

getPartitions…​FIXME

getPreferredLocations Method

getPreferredLocations(split: Partition): Seq[String]
Note
getPreferredLocations is part of RDD Contract to…​FIXME.

getPreferredLocations…​FIXME

Creating BlockRDD Instance

BlockRDD takes the following when created:

BlockRDD initializes the internal registries and counters.

results matching ""

    No results matching ""