The Internals of Kafka Streams
Introduction
Kafka Streams — Stream Processing Library on Apache Kafka
Why Kafka Streams
Multi-Instance Kafka Streams Applications
Exactly-Once Support (EOS)
KafkaStreams, StreamThreads, StreamTasks and StandbyTasks
Demos
Creating Topology with State Store with Logging Enabled
Stateful Stream Processing
Stateful Stream Processing
StateStore
Stores
Materialized
StoreSupplier
StoreBuilder
TimestampedKeyValueStore
ReadOnlyKeyValueStore
Stream Processing with Kafka Streams
KafkaStreams
Topology
TopologyDescription
- Node
- Processor
- Sink
- Source
StreamsConfig
Configuration Properties
Logging
KafkaClientSupplier
High-Level Streams DSL
High-Level Streams DSL
StreamsBuilder
KStream
KTable
GlobalKTable
KGroupedStream
- SessionWindowedKStream
- TimeWindowedKStream
KGroupedTable
Consumed
Produced
Grouped
Joined
Printed
KeyValueMapper
ValueJoiner
ValueTransformer
- ValueTransformerSupplier
ValueTransformerWithKey
Transformer
TransformerSupplier
Windows
Window
WindowedSerdes
- Windowed
- TimeWindowedSerializer
Scala API for Kafka Streams
Low-Level Processor API
Low-Level Processor API
Processor Contract — Stream Processing Node
- AbstractProcessor — Base for Stream Processors
ProcessorContext
- TaskId
- To
Punctuator Contract — Scheduled Periodic Actions
Cancellable
ProcessorSupplier Contract
StreamPartitioner
TopicNameExtractor — Dynamic Routing of Output Records
- RecordContext — Record Metadata
TimestampExtractor Contract
PartitionGrouper Contract
- DefaultPartitionGrouper
StateRestoreCallback
- BatchingStateRestoreCallback
StateRestoreListener
- AbstractNotifyingBatchingRestoreCallback
- AbstractNotifyingRestoreCallback
Monitoring Kafka Streams Applications
StateListener — KafkaStreams State Listener
CacheFlushListener
StreamsMetrics
- StreamsMetricsImpl
- StreamsMetricsThreadImpl
TaskMetrics
StoreChangeLogger
DelegatingStateRestoreListener
Testing
TopologyTestDriver
Internals of Kafka Streams
Low-Level Stream Processing Graph
InternalTopologyBuilder
AbstractNode
- Processor
- Sink
- Source
NodeFactory
InternalTopologyBuilder.TopologyDescription
GlobalStore
StateStoreFactory
High-Level Stream Processing Graph
InternalStreamsBuilder
StreamsGraphNode
- BaseJoinProcessorNode
- BaseRepartitionNode
ProcessorParameters
GlobalStoreNode
GroupedTableOperationRepartitionNode
- GroupedTableOperationRepartitionNodeBuilder
KTableKTableJoinNode
OptimizableRepartitionNode
ProcessorGraphNode
StatefulProcessorNode
StateStoreNode
StreamSinkNode
StreamSourceNode
StreamStreamJoinNode
StreamTableJoinNode
TableProcessorNode
TableSourceNode
Processors and ProcessorSuppliers
KStreamAggProcessorSupplier
KStreamBranch
KStreamFilterProcessor
- KStreamFilter
KStreamJoinWindowProcessor
KStreamPeek
KStreamPassThrough
KStreamSessionWindowAggregateProcessor
- KStreamSessionWindowAggregate
KStreamTransformProcessor
- KStreamTransform
KStreamTransformValuesProcessor
- KStreamTransformValues
KStreamWindowAggregateProcessor
- KStreamWindowAggregate
KTableSourceProcessor
- KTableSource
KTableSuppressProcessor
KTableValueGetter
KTableValueGetterSupplier
Internals of State Stores
WrappedStateStore
AbstractStoreBuilder
InMemorySessionBytesStoreSupplier
RocksDbKeyValueBytesStoreSupplier
RocksDbSessionBytesStoreSupplier
RocksDbWindowBytesStoreSupplier
KeyValueToTimestampedKeyValueByteStoreAdapter
WindowToTimestampedWindowByteStoreAdapter
SegmentedBytesStore
- AbstractRocksDBSegmentedBytesStore
- RocksDBSegmentedBytesStore
TimeOrderedKeyValueBuffer
- InMemoryTimeOrderedKeyValueBuffer
CachedStateStore
Logical Plan of Stream Processing Execution
ProcessorNode
- SourceNode
- SinkNode
NodeMetrics
InternalTopicConfig
- WindowedChangelogTopicConfig
- UnwindowedChangelogTopicConfig
WindowedStreamPartitioner
WindowedSerializer
DefaultKafkaClientSupplier
SessionWindow
TimeWindow
UnlimitedWindow
AbstractStream
MaterializedInternal
KeyValueStoreMaterializer
InternalNameProvider Contract
GroupedStreamAggregateBuilder
KStreamImplJoin
StaticTopicNameExtractor
ConsumedInternal
ProducedInternal
QuickUnion
TopicsInfo
Physical Plan of Stream Processing Execution
ProcessorTopology
Task
ProcessorContextImpl
ProducerSupplier
AssignedTasks
- AssignedStandbyTasks
- AssignedStreamsTasks
ProcessorNodePunctuator
Kafka Streams Execution Engine
TaskManager
StreamThread
RebalanceListener
StateListener
StreamsMetadataState
StreamsPartitionAssignor
InternalProcessorContext Contract
ForwardingDisabledProcessorContext
GlobalStreamThread
- StateConsumer
GlobalStateMaintainer
- GlobalStateUpdateTask
RecordCollector
- RecordCollectorImpl
ThreadCache
- NamedCache
Stamped
PartitionGroup
- RecordInfo
RecordQueue
- StampedRecord
PunctuationQueue
- PunctuationSchedule
QueryableStoreProvider
StateStoreProvider
RecordDeserializer
StateDirectory
ProcessorRecordContext
CopartitionedTopicsValidator
State (Store) Management
StateManager
- AbstractStateManager
ProcessorStateManager
GlobalStateManager
- GlobalStateManagerImpl
Checkpointable
OffsetCheckpoint
ChangelogReader
- StoreChangelogReader
StateRestorer
RestoringTasks
RecordBatchingStateRestoreCallback
- CompositeRestoreListener
StateRestoreCallbackAdapter

Powered by GitBook

CopartitionedTopicsValidator

CopartitionedTopicsValidator

CopartitionedTopicsValidator is…FIXME

results matching ""

No results matching ""