KStreamImpl

KStreamImpl is a concrete stream and the default KStream.

KStreamImpl is created every time a transformation is executed and when…​FIXME

KStreamImpl uses KSTREAM-SOURCE- as the prefix for…​FIXME

KStreamImpl uses KSTREAM-SINK- as the prefix for…​FIXME

KStreamImpl uses KSTREAM-TRANSFORMVALUES- as the prefix for…​FIXME

Table 1. KStreamImpl’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description

defaultKeyValueMapper

KeyValueMapper that simply concatenates the key and the value of a record when executed (i.e. applied to a record stream)

doStreamTableJoin Internal Method

KStream<K, VR> doStreamTableJoin(
  KTable<K, VO> other,
  ValueJoiner<? super V, ? super VO, ? extends VR> joiner,
  Joined<K, V, VO> joined,
  boolean leftJoin)

doStreamTableJoin…​FIXME

Note
 doStreamTableJoin is used when KStreamImpl is requested to join and leftJoin.

doJoin Internal Method

KStream<K, VR> doJoin(
  KStream<K, VO> other,
  ValueJoiner<? super V, ? super VO, ? extends VR> joiner,
  JoinWindows windows,
  Joined<K, V, VO> joined,
  KStreamImplJoin join)

doJoin…​FIXME

Note
 doJoin is used when KStreamImpl is requested to join, outerJoin and leftJoin.

Transforming Values with Optional State — transformValues Method

KStream<K, V1> transformValues(
  final ValueTransformerSupplier<? super V, ? extends V1> valueTransformerSupplier,
  final String... stateStoreNames)
KStream<K, VR> transformValues(
  final ValueTransformerWithKeySupplier<? super K, ? super V, ? extends VR> valueTransformerSupplier,
  final String... stateStoreNames)
Note
 transformValues is part of KStream Contract for a stateful record-by-record value transformation.

transformValues…​FIXME

transformValues reports a NullPointerException when either ValueTransformerSupplier or ValueTransformerWithKeySupplier is null.

process Method

void process(
  ProcessorSupplier<? super K, ? super V> processorSupplier,
  String... stateStoreNames)
Note
 process is part of KStream Contract for…​FIXME

process…​FIXME

Transforming Record Into Zero or More Output Records with State (Stateful Transformation) — transform Operator

KStream<K1, V1> transform(
  TransformerSupplier<
    ? super K,
    ? super V,
    KeyValue<K1, V1>> transformerSupplier,
  String... stateStoreNames)
Note
 transform is part of KStream API to transform a record into zero or more output records with state.

transform requests InternalStreamsBuilder for a new processor name with KSTREAM-TRANSFORM prefix.

transform then creates a new KStreamTransform (for the input transformerSupplier) and requests InternalTopologyBuilder to register a new processor supplier under the name.

Note
 transform uses InternalStreamsBuilder to access the current InternalTopologyBuilder.

transform requests InternalTopologyBuilder to connect the processor supplier with state stores if specified.

In the end, transform creates a new KStreamImpl for the processor name and repartitionRequired flag turned on.

transform reports a NullPointerException when transformerSupplier is null.

Registering Processor Node with KStreamMapValues Processor Supplier and KSTREAM-MAPVALUES Prefix — mapValues Operator

<V1> KStream<K, V1> mapValues(final ValueMapper<? super V, ? extends V1> mapper)
Note
 mapValues is part of KStream Contract to…​FIXME.

mapValues creates a new KStreamImpl for a new processor (with the current InternalStreamsBuilder, source nodes and repartitionRequired flag).

Internally, mapValues requests InternalStreamsBuilder for a new processor name with KSTREAM-MAPVALUES prefix and registers a KStreamMapValues processor supplier under the name.

Note
 mapValues uses InternalStreamsBuilder to access the current InternalTopologyBuilder.

Passing Stream Through Topic — through Method

KStream<K, V> through(
  String topic) (1)
KStream<K, V> through(
  String topic,
  Produced<K, V> produced)

  1. Uses Produced.with factory method to create a Produced

Note
 through is part of KStream Contract to materialize a stream to a topic (pass it through) and create a new KStream from the topic (using the Produced for configuration).

through creates a new ProducedInternal for the given Produced and sets the key and value serdes if both null with the keySerde and valSerde, respectively.

through writes the stream to the topic (with the ProducedInternal).

In the end, through creates a ConsumedInternal (with FailOnInvalidTimestamp timestamp extractor) and requests the InternalStreamsBuilder to create a KStream to read from the topic.

Registering Processor Node with KStreamPrint Processor Supplier and KSTREAM-PRINTER Prefix — print Operator

void print(final Printed<K, V> printed)
Note
 print is part of KStream Contract to…​FIXME.

print creates a PrintedInternal for the input Printed.

print requests InternalStreamsBuilder for a new processor name with KSTREAM-PRINTER prefix and registers a KStreamPrint (with PrintForeachAction) processor supplier under the name.

Note
 print uses InternalStreamsBuilder to access the current InternalTopologyBuilder.

Adding StreamSinkNode to Node Graph — to Operator

void to(
  String topic) (1)
void to(
  String topic,
  Produced<K, V> produced)
void to(
  TopicNameExtractor<K, V> topicExtractor)
void to(
  TopicNameExtractor<K, V> topicExtractor,
  Produced<K, V> produced)

  1. Calls the other to with Produced of nulls

Note
 to is part of KStream Contract to…​FIXME.

to merely passes the call on to the internal to with a new ProducedInternal for the input Produced.

to Internal Method

void to(
  final TopicNameExtractor<K, V> topicExtractor,
  final ProducedInternal<K, V> produced)

to requests the InternalStreamsBuilder for a new processor name with KSTREAM-SINK prefix.

to creates a new StreamSinkNode and requests the InternalStreamsBuilder to add it to the parent StreamsGraphNode.

Note
 to is used in to operators.

repartitionForJoin Internal Method

KStreamImpl<K, V> repartitionForJoin(
  final Serde<K> keySerde,
  final Serde<V> valSerde)

repartitionForJoin…​FIXME

Note
 repartitionForJoin is used when…​FIXME

Creating KStreamImpl Instance

KStreamImpl takes the following when created:

KStreamImpl initializes the internal registries and counters.

Transforming Values with State — transformValues Internal Method

private <VR> KStream<K, VR> transformValues(
  final InternalValueTransformerWithKeySupplier<? super K, ? super V, ? extends VR> internalValueTransformerWithKeySupplier,
  final String... stateStoreNames)

transformValues requests InternalStreamsBuilder for a new processor name with KSTREAM-TRANSFORMVALUES prefix.

transformValues then creates a new KStreamTransformValues (for the input internalValueTransformerWithKeySupplier) and requests InternalTopologyBuilder to register a new processor supplier under the name.

Note
 transformValues uses InternalStreamsBuilder to access the current InternalTopologyBuilder.

transformValues requests InternalTopologyBuilder to connect the processor supplier with state stores if specified.

In the end, transformValues creates a new KStreamImpl for the processor name.

Note
 transformValues is used exclusively when KStreamImpl is requested to transformValues.

createReparitionedSource Static Method

String createReparitionedSource(
  final InternalStreamsBuilder builder,
  final Serde<K1> keySerde,
  final Serde<V1> valSerde,
  final String topicNamePrefix,
  final String name)

createReparitionedSource requests the input InternalStreamsBuilder for the InternalTopologyBuilder and does the following:

  • Requests the InternalTopologyBuilder to addInternalTopic with the topic name as the input topicNamePrefix (if defined) or the input name and -repartition suffix

  • Requests the InternalStreamsBuilder for a new processor name with KSTREAM-FILTER- prefix and requests the InternalTopologyBuilder to addProcessor with the new processor name and a new KStreamFilter (that filters out null keys) and the name predecessor

  • Requests the InternalStreamsBuilder for a new processor name with KSTREAM-SINK- prefix and requests the InternalTopologyBuilder to add a sink node with the new processor name, the repartition topic and the new KStreamFilter as a predecessor

  • Requests the InternalStreamsBuilder for a new processor name with KSTREAM-SOURCE- prefix (aka sourceName) and requests the InternalTopologyBuilder to add a source node with the new processor name, a FailOnInvalidTimestamp and the repartition topic

In the end, createReparitionedSource returns the source name.

// CAUTION: FIXME Example
Note

createReparitionedSource is used when:

createWindowedStateStore Internal Static Method

<K, V> StoreBuilder<WindowStore<K, V>> createWindowedStateStore(
  final JoinWindows windows,
  final Serde<K> keySerde,
  final Serde<V> valueSerde,
  final String storeName)

createWindowedStateStore…​FIXME

Note
 createWindowedStateStore is used exclusively when KStreamImplJoin is requested to join.

groupBy Method

KGroupedStream<KR, V> groupBy(
  final KeyValueMapper<? super K, ? super V, KR> selector)
KGroupedStream<KR, V> groupBy(
  final KeyValueMapper<? super K, ? super V, KR> selector,
  final Grouped<KR, V> grouped)
Note
 groupBy is part of the KStream Contract to…​FIXME.

groupBy…​FIXME

groupByKey Method

KGroupedStream<K, V> groupByKey()
KGroupedStream<K, V> groupByKey(final Grouped<K, V> grouped)
Note
 groupByKey is part of the KStream Contract to…​FIXME.

groupByKey…​FIXME

filter Method

KStream<K, V> filter(final Predicate<? super K, ? super V> predicate)
Note
 filter is part of the KStream Contract to…​FIXME.

filter…​FIXME

filterNot Method

KStream<K, V> filterNot(final Predicate<? super K, ? super V> predicate)
Note
 filterNot is part of the KStream Contract to…​FIXME.

filterNot…​FIXME

flatTransform Method

KStream<K1, V1> flatTransform(
  TransformerSupplier<
    ? super K,
    ? super V,
    Iterable<KeyValue<K1, V1>>> transformerSupplier,
  String... stateStoreNames)
Note
 flatTransform is part of the KStream Contract to…​FIXME.

flatTransform…​FIXME

flatTransformValues Method

KStream<K, VR> flatTransformValues(
  ValueTransformerSupplier<
    ? super V,
    Iterable<VR>> valueTransformerSupplier,
  String... stateStoreNames)
KStream<K, VR> flatTransformValues(
  ValueTransformerWithKeySupplier<
    ? super K,
    ? super V,
    Iterable<VR>> valueTransformerSupplier,
  String... stateStoreNames)
Note
 flatTransformValues is part of the KStream Contract to…​FIXME.

flatTransformValues…​FIXME

doTransformValues Internal Method

KStream<K, VR> doTransformValues(
  ValueTransformerWithKeySupplier<
    ? super K,
    ? super V,
    ? extends VR> valueTransformerWithKeySupplier,
  String... stateStoreNames)

doTransformValues requests the InternalStreamsBuilder for a new processor name with KSTREAM-TRANSFORMVALUES prefix.

doTransformValues creates a new StatefulProcessorNode with the new processor name, the given stateStoreNames and the repartitionRequired flag.

doTransformValues requests the StatefulProcessorNode to setValueChangingOperation.

doTransformValues requests the InternalStreamsBuilder to add the StatefulProcessorNode (with the StreamsGraphNode as the parent).

In the end, doTransformValues creates a new KStreamImpl (with the new processor name, the sourceNodes, the repartitionRequired flag, the StatefulProcessorNode itself and the InternalStreamsBuilder).

Note
 doTransformValues is used exclusively when KStreamImpl is requested to transformValues.

internalSelectKey Internal Method

ProcessorGraphNode<K, V> internalSelectKey(
  final KeyValueMapper<? super K, ? super V, ? extends KR> mapper)

internalSelectKey…​FIXME

Note
 internalSelectKey is used when KStreamImpl is requested to selectKey and groupBy.

doFlatTransform Internal Method

KStream<K1, V1> doFlatTransform(
  TransformerSupplier<
    ? super K,
    ? super V,
    Iterable<KeyValue<K1, V1>>> transformerSupplier,
  String... stateStoreNames)

doFlatTransform…​FIXME

Note
 doFlatTransform is used when KStreamImpl is requested to transform and flatTransform.

doFlatTransformValues Internal Method

KStream<K, VR> doFlatTransformValues(
  ValueTransformerWithKeySupplier<
    ? super K,
    ? super V,
    Iterable<VR>> valueTransformerWithKeySupplier,
  String... stateStoreNames)

doFlatTransformValues…​FIXME

Note
 doFlatTransformValues is used exclusively when KStreamImpl is requested to flatTransformValues.

globalTableJoin Internal Method

KStream<K, VR> globalTableJoin(
  GlobalKTable<KG, VG> globalTable,
  KeyValueMapper<
    ? super K,
    ? super V,
    ? extends KG> keyMapper,
  ValueJoiner<
    ? super V,
    ? super VG,
    ? extends VR> joiner,
  boolean leftJoin)

globalTableJoin…​FIXME

Note
 globalTableJoin is used when KStreamImpl is requested to join and leftJoin.

results matching ""

    No results matching ""