StreamProgress — Collection of Offsets per Streaming Source

StreamProgress is a collection of Offsets per streaming source.

StreamProgress is created when:

StreamProgress is an extension of Scala’s scala.collection.immutable.Map with streaming sources as keys and their Offsets as values.

StreamProgress takes the following to be created:

Optional collection of offsets per streaming source (Map[BaseStreamingSource, Offset]) (default: empty)

get(key: BaseStreamingSource): Option[Offset]

Note	`get` is part of the Scala’s `scala.collection.MapLike` to…FIXME.

get simply looks up an Offsets for the given BaseStreamingSource in the baseMap.

++(
  updates: GenTraversableOnce[(BaseStreamingSource, Offset)]): StreamProgress

++ simply creates a new StreamProgress with the baseMap and the given updates.

Note	`++` is used exclusively when `OffsetSeq` is requested to convert to StreamProgress.

toOffsetSeq(
  sources: Seq[BaseStreamingSource],
  metadata: OffsetSeqMetadata): OffsetSeq

toOffsetSeq creates a OffsetSeq with offsets that are looked up for every BaseStreamingSource.

Note	`toOffsetSeq` is used when: `MicroBatchExecution` stream execution engine is requested to construct the next streaming micro-batch (to commit available offsets for a batch to the write-ahead log) `StreamExecution` is requested to run stream processing (that failed with a Throwable)

StreamProgress