Web UI — Spark Application’s Web Console

Web UI (aka Application UI or webUI or Spark UI) is the web interface of a Spark application to monitor and inspect Spark job executions in a web browser.

spark webui jobs.png
Figure 1. Welcome page - Jobs page

Every time you create a SparkContext in a Spark application you also launch an instance of web UI. The web UI is available at http://[driverHostname]:4040 by default.

Note
The default port can be changed using spark.ui.port configuration property. SparkContext will increase the port if it is already taken until an open port is found.

web UI comes with the following tabs (which may not all be visible immediately, but only after the respective modules are in use, e.g. the SQL or Streaming tabs):

Tip
You can use the web UI after the application has finished by persisting events (using EventLoggingListener) and using Spark History Server.
Note
All the information that is displayed in web UI is available thanks to JobProgressListener and other SparkListeners. One could say that web UI is a web layer to Spark listeners.

results matching ""

    No results matching ""