Cleaner

Cleaner is created exclusively for a CleanerThread.

Cleaner requires that the number of slots in the given OffsetMap multiplied by the given dupBufferLoadFactor is greater than 1 or the following exception is thrown:

offset map is too small to fit in even a single message, so log cleaning will never make progress. You can increase log.cleaner.dedupe.buffer.size or decrease log.cleaner.threads

Cleaner uses Cleaner [id]: prefix for logging (with the given ID).

Tip
 Cleaner uses kafka.log.LogCleaner logger.

Creating Cleaner Instance

Cleaner takes the following to be created:

Cleaner initializes the internal properties.

Cleaning Log — clean Method

clean(cleanable: LogToClean): (Long, CleanerStats)

clean…​FIXME

Note
 clean is used exclusively when CleanerThread is requested to clean a log.

Cleaning Log — doClean Method

doClean(
  cleanable: LogToClean,
  deleteHorizonMs: Long): (Long, CleanerStats)

doClean…​FIXME

Note
 doClean is used when Cleaner is requested to clean a log.

cleanSegments Internal Method

cleanSegments(
  log: Log,
  segments: Seq[LogSegment],
  map: OffsetMap,
  deleteHorizonMs: Long,
  stats: CleanerStats,
  transactionMetadata: CleanedTransactionMetadata): Unit

cleanSegments…​FIXME

Note
 cleanSegments is used exclusively when Cleaner is requested to clean a log.

cleanInto Internal Method

cleanInto(
  topicPartition: TopicPartition,
  sourceRecords: FileRecords,
  dest: LogSegment,
  map: OffsetMap,
  retainDeletesAndTxnMarkers: Boolean,
  maxLogMessageSize: Int,
  transactionMetadata: CleanedTransactionMetadata,
  lastRecordsOfActiveProducers: Map[Long, LastRecord],
  stats: CleanerStats): Unit

cleanInto…​FIXME

Note
 cleanInto is used exclusively when Cleaner is requested to cleanSegments.

buildOffsetMap Internal Method

buildOffsetMap(
  log: Log,
  start: Long,
  end: Long,
  map: OffsetMap,
  stats: CleanerStats): Unit

buildOffsetMap…​FIXME

Note
 buildOffsetMap is used exclusively when Cleaner is requested to doClean.

buildOffsetMapForSegment Internal Method

buildOffsetMapForSegment(
  topicPartition: TopicPartition,
  segment: LogSegment,
  map: OffsetMap,
  startOffset: Long,
  maxLogMessageSize: Int,
  transactionMetadata: CleanedTransactionMetadata,
  stats: CleanerStats): Boolean

buildOffsetMapForSegment…​FIXME

Note
 buildOffsetMapForSegment is used exclusively when Cleaner is requested to buildOffsetMap.

Internal Properties

Name Description

decompressionBufferSupplier

FIXME

Used when…​FIXME

readBuffer

FIXME

Used when…​FIXME

writeBuffer

FIXME

Used when…​FIXME

results matching ""

    No results matching ""