AppStatusStore

AppStatusStore is…​FIXME

spark core AppStatusStore createLiveStore.png
Figure 1. AppStatusStore in Spark Application

AppStatusStore is created when:

When created for a live Spark application, AppStatusStore creates a AppStatusListener that is later used to get the active stages.

streamBlocksList Method

streamBlocksList(): Seq[StreamBlockData]

streamBlocksList…​FIXME

Note
streamBlocksList is used when…​FIXME

activeStages Method

activeStages(): Seq[v1.StageData]

activeStages…​FIXME

Note
activeStages is used when…​FIXME

Creating AppStatusStore Instance

AppStatusStore takes the following when created:

Creating Live Store (AppStatusStore For Active Spark Application) — createLiveStore Factory Method

createLiveStore(conf: SparkConf): AppStatusStore

createLiveStore creates a fully-initialized AppStatusStore.

Internally, createLiveStore creates a ElementTrackingStore (with a new InMemoryStore and the input SparkConf).

createLiveStore creates a AppStatusListener (with the ElementTrackingStore created, the input SparkConf and the live flag enabled).

In the end, createLiveStore creates an AppStatusStore (with the ElementTrackingStore and AppStatusListener just created).

Note
createLiveStore is used exclusively when SparkContext is created.

Closing AppStatusStore — close Method

close(): Unit

close simply requests KVStore to close.

Note
close is used when…​FIXME

rddList Method

rddList(cachedOnly: Boolean = true): Seq[v1.RDDStorageInfo]

rddList requests the KVStore for a view over RDDStorageInfoWrapper entities.

In the end, rddList takes RDDStorageInfos with at least one partition cached (when cachedOnly flag is on) or all RDDStorageInfos (when cachedOnly flag is off).

Note
cachedOnly flag is on and therefore rddList gives RDDs cached only.
Note

rddList is used when:

  • StoragePage is requested to render itself

  • AbstractApplicationResource is requested to handle storage/rdd REST API

  • StagePagedTable is requested to makeDescription

results matching ""

    No results matching ""