Schedulable Builders

SchedulableBuilder is a contract of schedulable builders that operate on a pool of TaskSetManagers (from an owning TaskSchedulerImpl).

A SchedulableBuilder is created when TaskSchedulerImpl is being initialized. You can select the SchedulableBuilder to use by spark.scheduler.mode setting.

Spark comes with two implementations of the SchedulableBuilder Contract:

SchedulableBuilder is a private[spark] Scala trait. You can find the sources in org.apache.spark.scheduler.SchedulableBuilder.

SchedulableBuilder Contract

Every SchedulableBuilder provides the following services:

Root Pool (rootPool method)

rootPool: Pool

rootPool method returns a Pool (of Schedulables).

This is the data structure managed (aka wrapped) by SchedulableBuilders.

Build Pools (buildPools method)

buildPools(): Unit
It is exclusively called by TaskSchedulerImpl.initialize.

Adding Schedulable (to Pool) (addTaskSetManager method)

addTaskSetManager(manager: Schedulable, properties: Properties): Unit

addTaskSetManager registers the manager Schedulable (with additional properties) to the rootPool.

addTaskSetManager is exclusively used by TaskSchedulerImpl to submit a TaskSetManager for a stage for execution.

results matching ""

    No results matching ""