// Building a topology using Streams DSL
// Scala API for Kafka Streams
import org.apache.kafka.streams.scala._
import ImplicitConversions._
import Serdes._
val builder = new StreamsBuilder
builder
.table[String, String]("input")
.groupBy[String, String]((k,v) => (k,v)) // creates a KGroupedTableImpl
.count // creates a GroupedTableOperationRepartitionNode
val topology = builder.build
// Note the "*-repartition" topic
scala> println(topology.describe)
Topologies:
Sub-topology: 0
Source: KSTREAM-SOURCE-0000000001 (topics: [input])
--> KTABLE-SOURCE-0000000002
Processor: KTABLE-SOURCE-0000000002 (stores: [input-STATE-STORE-0000000000])
--> KTABLE-SELECT-0000000003
<-- KSTREAM-SOURCE-0000000001
Processor: KTABLE-SELECT-0000000003 (stores: [])
--> KSTREAM-SINK-0000000005
<-- KTABLE-SOURCE-0000000002
Sink: KSTREAM-SINK-0000000005 (topic: KTABLE-AGGREGATE-STATE-STORE-0000000004-repartition)
<-- KTABLE-SELECT-0000000003
Sub-topology: 1
Source: KSTREAM-SOURCE-0000000006 (topics: [KTABLE-AGGREGATE-STATE-STORE-0000000004-repartition])
--> KTABLE-AGGREGATE-0000000007
Processor: KTABLE-AGGREGATE-0000000007 (stores: [KTABLE-AGGREGATE-STATE-STORE-0000000004])
--> KTABLE-MAPVALUES-0000000008
<-- KSTREAM-SOURCE-0000000006
Processor: KTABLE-MAPVALUES-0000000008 (stores: [])
--> none
<-- KTABLE-AGGREGATE-0000000007
GroupedTableOperationRepartitionNode
GroupedTableOperationRepartitionNode
is a StreamsGraphNode (indirectly as a BaseRepartitionNode) that is used by KGroupedTableImpl
for all supported streaming operators, i.e. reduce, count and aggregate.
GroupedTableOperationRepartitionNode
is created exclusively when GroupedTableOperationRepartitionNodeBuilder
is requested to build one.
GroupedTableOperationRepartitionNode
takes the following when created:
GroupedTableOperationRepartitionNode
allows creating GroupedTableOperationRepartitionNodeBuilders using groupedTableOperationNodeBuilder
factory method.
GroupedTableOperationRepartitionNodeBuilder<K1, V1> groupedTableOperationNodeBuilder()
groupedTableOperationNodeBuilder
is used exclusively when KGroupedTableImpl
is requested to createRepartitionNode (for all supported streaming operators, i.e. reduce, count and aggregate).
writeToTopology
Method
void writeToTopology(final InternalTopologyBuilder topologyBuilder)
Note
|
writeToTopology is part of the StreamsGraphNode Contract to…FIXME.
|
writeToTopology
…FIXME