Log Cleanup Policies (Strategies) — Log Compaction And Retention

Kafka uses log.cleanup.policy configuration property to define cleanup strategies (policy) of logs:

Kafka uses log.retention.check.interval.ms configuration property as the interval between regular log checks.

log.cleanup.policy and cleanup.policy Configuration Properties

The cluster-wide log.cleanup.policy and the per-topic cleanup.policy configuration properties are comma-separated lists of cleanup strategies:

Unless defined, cleanup.policy is exactly log.cleanup.policy.

Log Compaction

Log Compaction is a cleanup strategy in which…​FIXME

Kafka brokers use LogCleaner for compact retention strategy.

Log compaction can be reconfigured dynamically at runtime.

Log Retention

Log Retention (Garbage Collection) is a cleanup strategy to discard (delete) old log segments when their retention time or size limit has been reached.

By default there is only a time limit and no size limit.

Retention time is controlled by the cluster-wide log.retention.ms, log.retention.minutes or log.retention.hours configuration properties (from the highest to the lowest priority) or their per-topic retention.ms configuration property.

Retention size is controlled by the cluster-wide log.retention.bytes or per-topic retention.bytes configuration property.

Enable ALL logging level for kafka.log.Log logger to see messages related to log retention.

Kafka brokers schedule kafka-log-retention periodic task for delete retention strategy.

results matching ""

    No results matching ""