DumpLogSegments

DumpLogSegments is a standalone application to print-data-log, verify-index-only and index-sanity-check of data and index log files.

DumpLogSegments can be executed on command line using kafka-dump-log.sh shell script (that simply uses kafka-run-class shell script to launch the command).

DumpLogSegments uses custom record parsers to dump records of log files of __consumer_offsets and __transaction_state internal topics. For other topics, DumpLogSegments uses key-decoder-class and value-decoder-class decoders (default: kafka.serializer.StringDecoder).

Table 1. Options
Option	Description
`deep-iteration`	Uses deep iteration (not shallow). Implied with print-data-log
`files`	(required) Comma-separated list of data and index log files to dump contents of
`help`	Prints usage information
`index-sanity-check`	Checks the index sanity without printing content. This is the same check that is executed on broker startup to determine if an index needs rebuilding or not.
`key-decoder-class`	Fully-qualified class name of the key deserializer of messages in the log files. Should implement `kafka.serializer.Decoder` trait, and the custom jar should be available in `kafka/libs` directory. Default: `kafka.serializer.StringDecoder`
`max-message-size`	Size of the largest message Default: `5 * 1024 * 1024`
`offsets-decoder`	Uses OffsetsMessageParser to parse log data as offset data from the `__consumer_offsets` topic
`print-data-log`	Prints message content when dumping data logs. Implied with any decoder option specified.
`transaction-log-decoder`	Uses TransactionLogMessageParser to parse log data as transaction metadata from the `__transaction_state` topic
`value-decoder-class`	Fully-qualified class name of the value deserializer of messages in the log files. Should implement `kafka.serializer.Decoder` trait, and the custom jar should be available in `kafka/libs` directory. Default: `kafka.serializer.StringDecoder`
`verify-index-only`	Verifies the index log without printing content
`version`	Displays Kafka version

Executing DumpLogSegments — `main` Object Method

main(args: Array[String]): Unit

main is the entry point of DumpLogSegments when launched on command line (e.g. using kafka-dump-log.sh shell script).

Internally, main parses the options. For every file (in the required --files option) main prints out the following message to the console:

Dumping [file]

main then branches off per file suffix:

For .log files, dumpLog
For .index files, dumpIndex
For .timeindex files, dumpTimeIndex
For .snapshot files, dumpProducerIdSnapshot
For .txnindex files, dumpTxnIndex

main…FIXME (misMatchesForIndexFilesMap)

main…FIXME (timeIndexDumpErrors)

main…FIXME (nonConsecutivePairsForLogFilesMap)

main simply ignores other file suffices and prints out the following message to standard error output:

Ignoring unknown file [file]

Dumping Content Of Log File (Of Log Segment) — `dumpLog` Internal Method

dumpLog(
  file: File,
  printContents: Boolean,
  nonConsecutivePairsForLogFilesMap: mutable.Map[String, List[(Long, Long)]],
  isDeepIteration: Boolean,
  maxMessageSize: Int,
  parser: MessageParser[_, _]): Unit

dumpLog reads the starting offset from the name of the given file and prints out the following message to the console:

Starting offset: [startOffset]

dumpLog opens the log file (and creates a FileRecords).

For every FileChannelRecordBatch dumpLog…FIXME

Note	`dumpLog` is used when `DumpLogSegments` tool is executed with .log files.

Dumping Content Of Transaction Index (Of Log Segment) — `dumpTxnIndex` Internal Method

dumpTxnIndex(
  file: File): Unit

dumpTxnIndex…FIXME

Note	`dumpTxnIndex` is used when `DumpLogSegments` tool is executed with .txnindex files.

Dumping Content Of Time Index (Of Log Segment) — `dumpTimeIndex` Internal Method

dumpTimeIndex(
  file: File,
  indexSanityOnly: Boolean,
  verifyOnly: Boolean,
  timeIndexDumpErrors: TimeIndexDumpErrors,
  maxMessageSize: Int): Unit

dumpTimeIndex…FIXME

Note	`dumpTimeIndex` is used when `DumpLogSegments` tool is executed with .timeindex files.

Dumping Content Of Offset Index (Of Log Segment) — `dumpIndex` Internal Method

dumpIndex(
  file: File,
  indexSanityOnly: Boolean,
  verifyOnly: Boolean,
  misMatchesForIndexFilesMap: mutable.Map[String, List[(Long, Long)]],
  maxMessageSize: Int): Unit

dumpIndex…FIXME

Note	`dumpIndex` is used when `DumpLogSegments` tool is executed with .index files.

Dumping Content Of Producer Snapshot (Of Log Segment) — `dumpProducerIdSnapshot` Internal Method

dumpProducerIdSnapshot(
  file: File): Unit

dumpProducerIdSnapshot…FIXME

Note	`dumpProducerIdSnapshot` is used when `DumpLogSegments` tool is executed with .snapshot files.

DumpLogSegments