Kafka Server and Periodic Tasks

Apache Kafka uses the Scheduler to schedule periodic tasks. Kafka internal services use the interface to schedule maintenance tasks.

The page describes the tasks (that should further improve comprehension of the interaction of the internal services and how Kafka server works internally).

Table 1. Tasks
Kafka Service Task Name Task Method

KafkaController

delete-expired-tokens

expireTokens

auto-leader-rebalance-task

Sends AutoPreferredReplicaLeaderElection controller event to ControllerEventManager

GroupMetadataManager

delete-expired-group-metadata

cleanupGroupMetadata

__consumer_offsets-[offsetsPartition]

loadGroupsAndOffsets

__consumer_offsets-[offsetsPartition]

removeGroupsAndOffsets

handleTxnCompletion-[producerId]

handleTxnCompletion

TransactionCoordinator

transaction-abort

abortTimedOutTransactions

TransactionStateManager

transactionalId-expiration

enableTransactionalIdExpiration

load-txns-for-partition-[topicPartition]

loadTransactions

remove-txns-for-partition-[topicPartition]

removeTransactions

Log

PeriodicProducerExpirationCheck

removeExpiredProducers

flush-log

flush

delete-file

deleteSeg

LogManager

kafka-log-retention

cleanupLogs

kafka-log-flusher

flushDirtyLogs

kafka-recovery-point-checkpoint

checkpointLogRecoveryOffsets

kafka-log-start-offset-checkpoint

checkpointLogStartOffsets

kafka-delete-logs

deleteLogs

ReplicaManager

highwatermark-checkpoint

checkpointHighWatermarks

isr-expiration

maybeShrinkIsr

isr-change-propagation

maybePropagateIsrChanges

shutdown-idle-replica-alter-log-dirs-thread

shutdownIdleReplicaAlterLogDirsThread

results matching ""

    No results matching ""