Kafka Server and Periodic Tasks

Apache Kafka uses the Scheduler to schedule periodic tasks. Kafka internal services use the interface to schedule maintenance tasks.

The page describes the tasks (that should further improve comprehension of the interaction of the internal services and how Kafka server works internally).

Table 1. Tasks
Kafka Service	Task Name	Task
KafkaController	`delete-expired-tokens`	expireTokens
KafkaController	`auto-leader-rebalance-task`	Enqueues AutoPreferredReplicaLeaderElection controller event to ControllerEventManager (when `KafkaController` is requested to scheduleAutoLeaderRebalanceTask)
GroupMetadataManager	`delete-expired-group-metadata`	cleanupGroupMetadata
	`__consumer_offsets-[offsetsPartition]`	loadGroupsAndOffsets
	`__consumer_offsets-[offsetsPartition]`	removeGroupsAndOffsets
	`handleTxnCompletion-[producerId]`	handleTxnCompletion
TransactionCoordinator	`transaction-abort`	abortTimedOutTransactions
TransactionStateManager	`transactionalId-expiration`	enableTransactionalIdExpiration
	`load-txns-for-partition-[topicPartition]`	loadTransactions
	`remove-txns-for-partition-[topicPartition]`	removeTransactions
Log	`PeriodicProducerExpirationCheck`	removeExpiredProducers
	`flush-log`	flush
	`delete-file`	deleteSeg
LogManager	`kafka-log-retention`	cleanupLogs
	`kafka-log-flusher`	flushDirtyLogs
	`kafka-recovery-point-checkpoint`	checkpointLogRecoveryOffsets
	`kafka-log-start-offset-checkpoint`	checkpointLogStartOffsets
	`kafka-delete-logs`	deleteLogs
ReplicaManager	`highwatermark-checkpoint`	checkpointHighWatermarks
	`isr-expiration`	maybeShrinkIsr
	`isr-change-propagation`	maybePropagateIsrChanges
	`shutdown-idle-replica-alter-log-dirs-thread`	shutdownIdleReplicaAlterLogDirsThread

Kafka Server and Periodic Tasks

Kafka Server and Periodic Tasks

results matching ""

No results matching ""