TimestampExtractor Contract

TimestampExtractor is a contract of timestamp extractors that extract a timestamp from a record for event time (aka stream time) semantics.

package org.apache.kafka.streams.processor;

interface TimestampExtractor {
  long extract(ConsumerRecord<Object, Object> record, long previousTimestamp);
}

The extracted timestamp is in milliseconds and can never be negative (or will be dropped).

You can define a custom timestamp extractor for reading a topic as a KStream or a KTable in an Consumed object (using with or withTimestampExtractor).

val timestampExtractor = new TimestampExtractor {
  def extract(record: ConsumerRecord[Object, Object], previousTimestamp: Long) = ???
}
val consumed = Consumed.`with`(timestampExtractor)

val builder = new StreamsBuilder
builder.stream(topic = "t1", consumed)
builder.table(topic = "t1", consumed)

When not defined using a Consumed, Kafka Streams uses the default extractor as configured using default.timestamp.extractor configuration property.

val props = new java.util.Properties
import org.apache.kafka.streams.StreamsConfig
props.setProperty(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG, ...)

TimestampExtractor can be used to schedule a periodic operation for processors (using ProcessorContext.schedule with PunctuationType.STREAM_TIME).

Table 1. TimestampExtractor Contract
Method Description

extract

Used exclusively when RecordQueue is requested to add Kafka ConsumerRecords (as StampedRecords).
Table 2. TimestampExtractors
TimestampExtractor Description

WallclockTimestampExtractor

ExtractRecordMetadataTimestamp

Note
 TimestampExtractor is an Evolving contract which means that compatibility may be broken at a minor release.

TimestampExtractor is used to create a SourceNodeFactory, RecordQueue, SourceNodeFactory.

results matching ""

    No results matching ""