SchedulableBuilder Contract — Builders of Schedulable Pools

SchedulableBuilder is a private[spark] Scala trait that is used exclusively by TaskSchedulerImpl (the default Spark scheduler). When requested to initialize, TaskSchedulerImpl uses the spark.scheduler.mode configuration property (default: FIFO) to select one of the available schedulable builders.

Table 1. SchedulableBuilder Contract
Method Description

addTaskSetManager

addTaskSetManager(manager: Schedulable, properties: Properties): Unit

Registers a new Schedulable with the rootPool

Used exclusively when TaskSchedulerImpl is requested to submit tasks (of TaskSet) for execution (and registers a new TaskSetManager for the TaskSet)

buildPools

buildPools(): Unit

Builds a tree of pools (of Schedulables)

Used exclusively when TaskSchedulerImpl is requested to initialize (and creates a SchedulableBuilder per spark.scheduler.mode configuration property)

rootPool

rootPool: Pool

Root (top-level) pool (of Schedulables)

Used when:

Table 2. SchedulableBuilders
SchedulableBuilder Description

FairSchedulableBuilder

Used when the spark.scheduler.mode configuration property is FAIR

FIFOSchedulableBuilder

Default SchedulableBuilder that is used when the spark.scheduler.mode configuration property is FIFO (default)

results matching ""

    No results matching ""