MetricsServlet JSON Metrics Sink

MetricsServlet is a metrics sink that gives metrics snapshots in JSON format.

MetricsServlet is a "special" sink as it is only available to the metrics instances with a web UI:

  • Driver of a Spark application

  • Spark Standalone’s Master and Worker

You can access the metrics from MetricsServlet at /metrics/json URI by default. The entire URL depends on a metrics instance, e.g. http://localhost:4040/metrics/json/ for a running Spark application.

$ http http://localhost:4040/metrics/json/
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Length: 5005
Content-Type: text/json;charset=utf-8
Date: Mon, 11 Jun 2018 06:29:03 GMT
Server: Jetty(9.3.z-SNAPSHOT)
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block

{
    "counters": {
        "local-1528698499919.driver.HiveExternalCatalog.fileCacheHits": {
            "count": 0
        },
        "local-1528698499919.driver.HiveExternalCatalog.filesDiscovered": {
            "count": 0
        },
        "local-1528698499919.driver.HiveExternalCatalog.hiveClientCalls": {
            "count": 0
        },
        "local-1528698499919.driver.HiveExternalCatalog.parallelListingJobCount": {
            "count": 0
        },
        "local-1528698499919.driver.HiveExternalCatalog.partitionsFetched": {
            "count": 0
        },
        "local-1528698499919.driver.LiveListenerBus.numEventsPosted": {
            "count": 7
        },
        "local-1528698499919.driver.LiveListenerBus.queue.appStatus.numDroppedEvents": {
            "count": 0
        },
        "local-1528698499919.driver.LiveListenerBus.queue.executorManagement.numDroppedEvents": {
            "count": 0
        }
    },
    ...

MetricsServlet is created exclusively when MetricsSystem is started (and requested to register metrics sinks).

MetricsServlet can be configured using configuration properties with sink.servlet prefix (in metrics configuration). That is not required since MetricsConfig makes sure that MetricsServlet is always configured.

MetricsServlet uses jackson-databind, the general data-binding package for Jackson (as ObjectMapper) with Dropwizard Metrics library (i.e. registering a Coda Hale MetricsModule).

Table 1. MetricsServlet’s Configuration Properties
Name Default Description

path

/metrics/json/

Path URI prefix to bind to

sample

false

Whether to show entire set of samples for histograms

Table 2. MetricsServlet’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description

mapper

Jaxson’s com.fasterxml.jackson.databind.ObjectMapper that "provides functionality for reading and writing JSON, either to and from basic POJOs (Plain Old Java Objects), or to and from a general-purpose JSON Tree Model (JsonNode), as well as related functionality for performing conversions."

When created, mapper is requested to register a Coda Hale com.codahale.metrics.json.MetricsModule.

Used exclusively when MetricsServlet is requested to getMetricsSnapshot.

servletPath

Value of path configuration property

servletShowSample

Flag to control whether to show samples (true) or not (false).

servletShowSample is the value of sample configuration property (if defined) or false.

Used when ObjectMapper is requested to register a Coda Hale com.codahale.metrics.json.MetricsModule.

Creating MetricsServlet Instance

MetricsServlet takes the following when created:

  • Configuration Properties (as Java Properties)

  • Dropwizard Metrics' MetricRegistry

  • SecurityManager

MetricsServlet initializes the internal registries and counters.

Requesting Metrics Snapshot — getMetricsSnapshot Method

getMetricsSnapshot(request: HttpServletRequest): String

getMetricsSnapshot simply requests the Jackson ObjectMapper to serialize the MetricRegistry to a JSON string (using ObjectMapper.writeValueAsString).

Note
getMetricsSnapshot is used exclusively when MetricsServlet is requested to getHandlers.

Requesting JSON Servlet Handler — getHandlers Method

getHandlers(conf: SparkConf): Array[ServletContextHandler]

getHandlers returns just a single ServletContextHandler (in a collection) that gives metrics snapshot in JSON format at every request at servletPath URI path.

Note
getHandlers is used exclusively when MetricsSystem is requested for metrics ServletContextHandlers.

results matching ""

    No results matching ""