StreamingQueryListenerBus — Notification Bus for Streaming Events

StreamingQueryListenerBus is an event bus (i.e. ListenerBus) to dispatch streaming events to StreamingQueryListener streaming event listeners.

StreamingQueryListenerBus is created when StreamingQueryManager is created (as the internal listenerBus).

StreamingQueryListenerBus.png
Figure 1. StreamingQueryListenerBus is Created Once In SparkSession

StreamingQueryListenerBus is also a SparkListener and registers itself with LiveListenerBus to intercept a QueryStartedEvent.

Table 1. StreamingQueryListenerBus’s Internal Registries and Counters
Name Description

activeQueryRunIds

Collection of active streaming queries by their runIds.

Used mainly when StreamingQueryListenerBus dispatches an event to listeners (for queries started in the same SparkSession).

Posting StreamingQueryListener Events to LiveListenerBus — post Method

post(event: StreamingQueryListener.Event): Unit

post simply posts the input event straight to LiveListenerBus except QueryStartedEvent events.

For QueryStartedEvent events, post adds the query’s runId to activeQueryRunIds registry first before posting the event to LiveListenerBus followed by postToAll.

Note
post is used exclusively when StreamingQueryManager posts StreamingQueryListener event.

onOtherEvent Method

Caution
FIXME

doPostEvent Method

Caution
FIXME

postToAll Method

Caution
FIXME

Creating StreamingQueryListenerBus Instance

StreamingQueryListenerBus takes the following when created:

  • LiveListenerBus

StreamingQueryListenerBus registers itself with LiveListenerBus.

StreamingQueryListenerBus initializes the internal registries and counters.

results matching ""

    No results matching ""