// Building a topology using Streams DSL
// Scala API for Kafka Streams
import org.apache.kafka.streams.scala._
import ImplicitConversions._
import Serdes._
val builder = new StreamsBuilder
builder
.table[String, String]("input")
.groupBy[String, String]((k,v) => (k,v)) // creates a KGroupedTableImpl
.count // creates a GroupedTableOperationRepartitionNode
val topology = builder.build
// Note the "*-repartition" topic
scala> println(topology.describe)
Topologies:
Sub-topology: 0
Source: KSTREAM-SOURCE-0000000001 (topics: [input])
--> KTABLE-SOURCE-0000000002
Processor: KTABLE-SOURCE-0000000002 (stores: [input-STATE-STORE-0000000000])
--> KTABLE-SELECT-0000000003
<-- KSTREAM-SOURCE-0000000001
Processor: KTABLE-SELECT-0000000003 (stores: [])
--> KSTREAM-SINK-0000000005
<-- KTABLE-SOURCE-0000000002
Sink: KSTREAM-SINK-0000000005 (topic: KTABLE-AGGREGATE-STATE-STORE-0000000004-repartition)
<-- KTABLE-SELECT-0000000003
Sub-topology: 1
Source: KSTREAM-SOURCE-0000000006 (topics: [KTABLE-AGGREGATE-STATE-STORE-0000000004-repartition])
--> KTABLE-AGGREGATE-0000000007
Processor: KTABLE-AGGREGATE-0000000007 (stores: [KTABLE-AGGREGATE-STATE-STORE-0000000004])
--> KTABLE-MAPVALUES-0000000008
<-- KSTREAM-SOURCE-0000000006
Processor: KTABLE-MAPVALUES-0000000008 (stores: [])
--> none
<-- KTABLE-AGGREGATE-0000000007
GroupedTableOperationRepartitionNode
GroupedTableOperationRepartitionNode is a StreamsGraphNode (indirectly as a BaseRepartitionNode) that is used by KGroupedTableImpl for all supported streaming operators, i.e. reduce, count and aggregate.
GroupedTableOperationRepartitionNode is created exclusively when GroupedTableOperationRepartitionNodeBuilder is requested to build one.
GroupedTableOperationRepartitionNode takes the following when created:
GroupedTableOperationRepartitionNode allows creating GroupedTableOperationRepartitionNodeBuilders using groupedTableOperationNodeBuilder factory method.
GroupedTableOperationRepartitionNodeBuilder<K1, V1> groupedTableOperationNodeBuilder()
groupedTableOperationNodeBuilder is used exclusively when KGroupedTableImpl is requested to createRepartitionNode (for all supported streaming operators, i.e. reduce, count and aggregate).
writeToTopology Method
void writeToTopology(final InternalTopologyBuilder topologyBuilder)
|
Note
|
writeToTopology is part of the StreamsGraphNode Contract to…FIXME.
|
writeToTopology…FIXME