SQL Tab — Monitoring Structured Queries in web UI

SQL tab in web UI shows SQLMetrics per physical operator in a structured query physical plan.

You can access the SQL tab under /SQL URL, e.g. http://localhost:4040/SQL/.

By default, it displays all SQL query executions. However, after a query has been selected, the SQL tab displays the details for the structured query execution.


AllExecutionsPage displays all SQL query executions in a Spark application per state sorted by their submission time reversed.

spark webui sql.png
Figure 1. SQL Tab in web UI (AllExecutionsPage)

Internally, the page requests SQLListener for query executions in running, completed, and failed states (the states correspond to the respective tables on the page).

ExecutionPage — Details for Query

ExecutionPage shows details for structured query execution by id.

The id request parameter is mandatory.

ExecutionPage displays a summary with Submitted Time, Duration, the clickable identifiers of the Running Jobs, Succeeded Jobs, and Failed Jobs.

It also display a visualization (using accumulator updates and the SparkPlanGraph for the query) with the expandable Details section (that corresponds to SQLExecutionUIData.physicalPlanDescription).

spark webui sql execution graph.png
Figure 2. Details for Query in web UI

If there is no information to display for a given query id, you should see the following page.

spark webui sql no details for query.png
Figure 3. No Details for SQL Query

Internally, it uses SQLListener exclusively to get the SQL query execution metrics. It requests SQLListener for SQL execution data to display for the id request parameter.

Creating SQLTab Instance

SQLTab is created when SharedState is or at the first SparkListenerSQLExecutionStart event when Spark History Server is used.

spark SQLTab creating instance.png
Figure 4. Creating SQLTab Instance
SharedState represents the shared state across SparkSessions.

results matching ""

    No results matching ""