Kafka Server and Periodic Tasks

Apache Kafka uses the Scheduler to schedule periodic tasks. Kafka internal services use the interface to schedule maintenance tasks.

The page describes the tasks (that should further improve comprehension of the interaction of the internal services and how Kafka server works internally).

Table 1. Tasks
Kafka Service Task Name Description

KafkaController

delete-expired-tokens

Executes expireTokens

auto-leader-rebalance-task

Sends AutoPreferredReplicaLeaderElection controller event to ControllerEventManager

GroupMetadataManager

delete-expired-group-metadata

Executes cleanupGroupMetadata

__consumer_offsets-[offsetsPartition]

Executes loadGroupsAndOffsets

__consumer_offsets-[offsetsPartition]

Executes removeGroupsAndOffsets

handleTxnCompletion-[producerId]

Executes handleTxnCompletion

TransactionCoordinator

transaction-abort

Executes abortTimedOutTransactions

TransactionStateManager

transactionalId-expiration

Executes enableTransactionalIdExpiration

load-txns-for-partition-[topicPartition]

Executes loadTransactions

remove-txns-for-partition-[topicPartition]

Executes removeTransactions

Log

PeriodicProducerExpirationCheck

Executes removeExpiredProducers

flush-log

Executes flush

delete-file

Executes deleteSeg

LogManager

kafka-log-retention

Executes cleanupLogs

kafka-log-flusher

Executes flushDirtyLogs

kafka-recovery-point-checkpoint

Executes checkpointLogRecoveryOffsets

kafka-log-start-offset-checkpoint

Executes checkpointLogStartOffsets

kafka-delete-logs

Executes deleteLogs

ReplicaManager

highwatermark-checkpoint

Executes checkpointHighWatermarks

isr-expiration

Executes maybeShrinkIsr

isr-change-propagation

Executes maybePropagateIsrChanges

shutdown-idle-replica-alter-log-dirs-thread

Executes shutdownIdleReplicaAlterLogDirsThread

results matching ""

    No results matching ""