WebUI — Base Web UI

WebUI is the base of the web UIs in Apache Spark:

  • Active Spark applications

  • Spark History Server

  • Spark Standalone cluster manager

  • Spark on Mesos cluster manager

Note
Spark on YARN uses a different web framework for the web UI.

WebUI is used as the parent of WebUITabs.

package org.apache.spark.ui

abstract class WebUI {
  // only required methods that have no implementation
  // the others follow
  def initialize(): Unit
}
Note
WebUI is a private[spark] contract.
Table 1. (Subset of) WebUI Contract
Method Description

initialize

Used in implementations only to let them initialize their web components

Note
initialize does not add anything special to the Scala type hierarchy but a common name to use across WebUIs (that could also be possible without it). In other words, initialize does not participate in any design pattern or a type hierarchy.

WebUI is a Scala abstract class and cannot be created directly, but only as one of the web UIs.

Table 2. WebUIs
WebUI Description

HistoryServer

Used in Spark History Server

MasterWebUI

Used in Spark Standalone cluster manager

MesosClusterUI

Used in Spark on Mesos cluster manager

SparkUI

WebUI of a Spark application

WorkerWebUI

Used in Spark Standalone cluster manager

Once bound to a Jetty HTTP server, WebUI is available at an HTTP port (and is used in the web URL as boundPort).

WebUI is available at a web URL, i.e. http://[publicHostName]:[boundPort]. The publicHostName is…​FIXME and the boundPort is the port that the port the Jetty HTTP Server bound to.

Table 3. WebUI’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description

tabs

WebUITabs

Used when…​FIXME

handlers

ServletContextHandlers

Used when…​FIXME

pageToHandlers

ServletContextHandlers per WebUIPage

Used when…​FIXME

serverInfo

Optional ServerInfo (default: None)

Used when…​FIXME

publicHostName

Host name of the UI

publicHostName is either SPARK_PUBLIC_DNS environment variable or spark.driver.host configuration property.

Defaults to the following if defined (in order):

  1. SPARK_LOCAL_HOSTNAME environment variable

  2. Host name of SPARK_LOCAL_IP environment variable

  3. Utils.findLocalInetAddress

Used exclusively when WebUI is requested for the web URL

className

Used when…​FIXME

Tip

Enable INFO or ERROR logging level for the corresponding loggers of the WebUIs, e.g. org.apache.spark.ui.SparkUI, to see what happens inside.

Add the following line to conf/log4j.properties:

log4j.logger.org.apache.spark.ui=INFO

Refer to Logging.

Creating WebUI Instance

WebUI takes the following when created:

  • SecurityManager

  • SSLOptions

  • Port number

  • SparkConf

  • basePath (default: empty)

  • Name (default: empty)

WebUI initializes the internal registries and counters.

Note
WebUI is a Scala abstract class and cannot be created directly, but only as one of the implementations.

Detaching Page And Associated Handlers from UI — detachPage Method

detachPage(page: WebUIPage): Unit

detachPage…​FIXME

Note
detachPage is used when…​FIXME

Detaching Tab And Associated Pages from UI — detachTab Method

detachTab(tab: WebUITab): Unit

detachTab…​FIXME

Note
detachTab is used when…​FIXME

Detaching Handler — detachHandler Method

detachHandler(handler: ServletContextHandler): Unit

detachHandler…​FIXME

Note
detachHandler is used when…​FIXME

Detaching Handler At Path — detachHandler Method

detachHandler(path: String): Unit

detachHandler…​FIXME

Note
detachHandler is used when…​FIXME

Attaching Page to UI — attachPage Method

attachPage(page: WebUIPage): Unit

Internally, attachPage creates the path of the WebUIPage that is / (forward slash) followed by the prefix of the page.

Note

attachPage is used when:

Attaching Tab And Associated Pages to UI — attachTab Method

attachTab(tab: WebUITab): Unit

attachTab attaches every WebUIPage of the input WebUITab.

In the end, attachTab adds the input WebUITab to WebUITab tabs.

Note
attachTab is used when…​FIXME

Attaching Static Handler — addStaticHandler Method

addStaticHandler(resourceBase: String, path: String): Unit

addStaticHandler…​FIXME

Note
addStaticHandler is used when…​FIXME

Attaching Handler to UI — attachHandler Method

attachHandler(handler: ServletContextHandler): Unit

attachHandler simply adds the input Jetty ServletContextHandler to handlers registry and requests the ServerInfo to addHandler (only if the ServerInfo is defined).

Note

attachHandler is used when:

getBasePath Method

getBasePath: String

getBasePath simply returns the base path.

Note
getBasePath is used exclusively when WebUITab is requested for the base path.

Requesting Header Tabs — getTabs Method

getTabs: Seq[WebUITab]

getTabs simply returns the registered tabs.

Note
getTabs is used exclusively when WebUITab is requested for the header tabs.

Requesting Handlers — getHandlers Method

getHandlers: Seq[ServletContextHandler]

getHandlers simply returns the registered handlers.

Note
getHandlers is used when…​FIXME

Binding UI to Jetty HTTP Server on Host — bind Method

bind(): Unit

bind…​FIXME

Note
bind is used when…​FIXME

Stopping UI — stop Method

stop(): Unit

stop…​FIXME

Note
stop is used when…​FIXME

results matching ""

    No results matching ""