ConsumerCoordinator

ConsumerCoordinator is a concrete AbstractCoordinator that…​FIXME

ConsumerCoordinator is created exclusively when KafkaConsumer is created.

ConsumerCoordinator.png
Figure 1. ConsumerCoordinator and KafkaConsumer

KafkaConsumer uses the Consumer configuration properties to create a ConsumerCoordinator.

Table 1. ConsumerCoordinator’s Consumer Configuration Properties
Configuration Property ConsumerCoordinator’s Property

max.poll.interval.ms

rebalanceTimeoutMs

session.timeout.ms

sessionTimeoutMs

heartbeat.interval.ms

heartbeatIntervalMs

retry.backoff.ms

retryBackoffMs

enable.auto.commit

autoCommitEnabled

auto.commit.interval.ms

autoCommitIntervalMs

exclude.internal.topics

excludeInternalTopics

internal.leave.group.on.close

leaveGroupOnClose

With autoCommitEnabled enabled (i.e. enable.auto.commit is true), ConsumerCoordinator does:

Table 2. ConsumerCoordinator’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description

Sending Asynchronous Auto-Commit of Offsets — maybeAutoCommitOffsetsAsync Method

void maybeAutoCommitOffsetsAsync(long now)

maybeAutoCommitOffsetsAsync…​FIXME

Note

maybeAutoCommitOffsetsAsync is used when:

Sending Synchronous Auto-Commit of Offsets — maybeAutoCommitOffsetsSync Internal Method

void maybeAutoCommitOffsetsSync(long timeoutMs)

maybeAutoCommitOffsetsSync…​FIXME

Note
maybeAutoCommitOffsetsSync is used when ConsumerCoordinator is requested to close or onJoinPrepare

doAutoCommitOffsetsAsync Internal Method

void doAutoCommitOffsetsAsync()

doAutoCommitOffsetsAsync…​FIXME

Note
doAutoCommitOffsetsAsync is used when…​FIXME

close Method

void close(long timeoutMs)

close…​FIXME

Note
close is used when…​FIXME

commitOffsetsAsync Method

void commitOffsetsAsync(
  final Map<TopicPartition,
  OffsetAndMetadata> offsets,
  final OffsetCommitCallback callback)

commitOffsetsAsync…​FIXME

Note
commitOffsetsAsync is used when…​FIXME

commitOffsetsSync Method

boolean commitOffsetsSync(
  Map<TopicPartition,
  OffsetAndMetadata> offsets,
  long timeoutMs)

commitOffsetsSync…​FIXME

Note
commitOffsetsSync is used when…​FIXME

refreshCommittedOffsetsIfNeeded Method

void refreshCommittedOffsetsIfNeeded()

refreshCommittedOffsetsIfNeeded…​FIXME

Note
refreshCommittedOffsetsIfNeeded is used when…​FIXME

onJoinComplete Method

void onJoinComplete(
  int generation,
  String memberId,
  String assignmentStrategy,
  ByteBuffer assignmentBuffer)
Note
onJoinComplete is part of AbstractCoordinator Contract to…​FIXME.

onJoinComplete…​FIXME

onJoinPrepare Method

void onJoinPrepare(int generation, String memberId)
Note
onJoinPrepare is part of AbstractCoordinator Contract to…​FIXME.

onJoinPrepare…​FIXME

performAssignment Method

Map<String, ByteBuffer> performAssignment(
  String leaderId,
  String assignmentStrategy,
  Map<String, ByteBuffer> allSubscriptions)
Note
performAssignment is part of AbstractCoordinator Contract to…​FIXME.

performAssignment…​FIXME

maybeLeaveGroup Method

synchronized void maybeLeaveGroup()

maybeLeaveGroup…​FIXME

Note
maybeLeaveGroup is used when…​FIXME

updatePatternSubscription Method

void updatePatternSubscription(Cluster cluster)

updatePatternSubscription…​FIXME

Note
updatePatternSubscription is used when…​FIXME

needRejoin Method

boolean needRejoin()
Note
needRejoin is part of AbstractCoordinator Contract to…​FIXME.

needRejoin…​FIXME

timeToNextPoll Method

long timeToNextPoll(long now)

timeToNextPoll…​FIXME

Note
timeToNextPoll is used when…​FIXME

Polling for Coordinator Events — poll Method

void poll(long now, long remainingMs)

poll…​FIXME

Note
poll is used exclusively when KafkaConsumer is requested to pollOnce.

addMetadataListener Internal Method

void addMetadataListener()

addMetadataListener…​FIXME

Note
addMetadataListener is used when…​FIXME

fetchCommittedOffsets Method

Map<TopicPartition, OffsetAndMetadata> fetchCommittedOffsets(Set<TopicPartition> partitions)

fetchCommittedOffsets…​FIXME

Note
fetchCommittedOffsets is used when…​FIXME

Creating ConsumerCoordinator Instance

ConsumerCoordinator takes the following when created:

  • LogContext

  • ConsumerNetworkClient

  • Group ID

  • rebalanceTimeoutMs

  • sessionTimeoutMs

  • heartbeatIntervalMs

  • Collection of PartitionAssignors

  • Metadata

  • SubscriptionState

  • Metrics

  • Prefix of the metric group

  • Time

  • retryBackoffMs

  • autoCommitEnabled flag

  • autoCommitIntervalMs

  • ConsumerInterceptors

  • excludeInternalTopics flag

  • leaveGroupOnClose flag

ConsumerCoordinator initializes the internal registries and counters.

In the end, ConsumerCoordinator requests Metadata to update and addMetadataListener.

results matching ""

    No results matching ""