CleanerThread

CleanerThread is a non-daemon thread of execution for cleaning up dirty logs (one at a time until no more is left).

CleanerThread uses the thread name of the format kafka-log-cleaner-thread- followed by the threadId.

CleanerThread is created and immediately started when LogCleaner is requested to start up.

Note
 The number of CleanerThreads (that LogCleaner uses) is controlled by log.cleaner.threads dynamic configuration (default: 1).

CleanerThread takes a threadId to be created.

CleanerThread uses kafka.log.LogCleaner logger.

Tip

Enable ALL logging level for kafka.log.LogCleaner logger to see what happens inside.
Note
 CleanerThread is a Java private inner class of LogCleaner. It can only be used by LogCleaner and have a direct access to the internals of the LogCleaner.

CleanerThread and Cleaner

When created, CleanerThread creates a Cleaner (with the ID as the given threadId).

CleanerThread uses the Cleaner exclusively for cleaning up a log.

checkDone Internal Method

checkDone(topicPartition: TopicPartition): Unit

checkDone…​FIXME

Note
 checkDone is used exclusively when CleanerThread is created (to create the Cleaner).

Cleaning Up Dirty Logs — doWork Method

doWork(): Unit
Note
 doWork is part of the ShutdownableThread Contract to do the work.

doWork simply cleanFilthiestLog.

If there was no dirty log cleaned up, doWork pauses for the backOffMs (of the CleanerConfig of the parent LogCleaner) (default: 15 seconds).

cleanFilthiestLog Internal Method

cleanFilthiestLog(): Boolean

cleanFilthiestLog requests the LogCleanerManager to grabFilthiestCompactedLog.

If there is any filthiest log to work on, cleanFilthiestLog cleanLog and turns the cleaned flag on.

With no logs to work on, cleanFilthiestLog simply turns the cleaned flag off.

Note
 The value of cleaned flag is what cleanFilthiestLog returns.

cleanFilthiestLog requests the LogCleanerManager for the deletable logs and then requests every Log to deleteOldSegments.

In the end, cleanFilthiestLog requests the LogCleanerManager to doneDeleting (with the TopicPartitions of the deletable logs).

In case of an exception, cleanFilthiestLog…​FIXME

Note
 cleanFilthiestLog is used exclusively when CleanerThread is requested to doWork.

Cleaning Log — cleanLog Internal Method

cleanLog(cleanable: LogToClean): Unit

cleanLog requests the Cleaner to clean the given Log (as LogToClean) followed by recordStats.

In the end, cleanLog requests the LogCleanerManager to doneCleaning.

Note
 cleanLog is used exclusively when CleanerThread is requested to cleanFilthiestLog.

recordStats Internal Method

recordStats(
  id: Int,
  name: String,
  from: Long,
  to: Long,
  stats: CleanerStats): Unit

recordStats…​FIXME

Note
 recordStats is used exclusively when CleanerThread is requested to cleanLog.

results matching ""

    No results matching ""