FsHistoryProvider — File-System-Based History Provider

FsHistoryProvider is the default ApplicationHistoryProvider for Spark History Server.

FsHistoryProvider is created exclusively when HistoryServer is started as a standalone application and spark.history.provider configuration property was not defined.

Tip

Enable INFO or DEBUG logging levels for org.apache.spark.deploy.history.FsHistoryProvider logger to see what happens inside.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.deploy.history.FsHistoryProvider=DEBUG

Refer to Logging.

rebuildAppStore Internal Method

rebuildAppStore(
  store: KVStore,
  eventLog: FileStatus,
  lastUpdated: Long): Unit

rebuildAppStore…​FIXME

Note
rebuildAppStore is used when…​FIXME

getAppUI Method

getAppUI(appId: String, attemptId: Option[String]): Option[LoadedAppUI]
Note
getAppUI is part of ApplicationHistoryProvider Contract to…​FIXME.

getAppUI…​FIXME

Creating FsHistoryProvider Instance

FsHistoryProvider takes the following when created:

FsHistoryProvider initializes the internal registries and counters.

results matching ""

    No results matching ""