StreamWriter Contract

StreamWriter is the extension of the DataSourceWriter contract to support epochs, i.e. streaming writers that can abort and commit writing jobs for a specified epoch.

Table 1. StreamWriter Contract
Method Description

abort

void abort(
  long epochId,
  WriterCommitMessage[] messages)

Aborts the writing job for a specified epochId and WriterCommitMessages

Used exclusively when MicroBatchWriter is requested to abort

commit

void commit(
  long epochId,
  WriterCommitMessage[] messages)

Commits the writing job for a specified epochId and WriterCommitMessages

Used when:

  • EpochCoordinator is requested to commitEpoch

  • MicroBatchWriter is requested to commit

Table 2. StreamWriters
StreamWriter Description

ForeachWriterProvider

foreachWriter data source

ConsoleWriter

console data source

KafkaStreamWriter

kafka data source

MemoryStreamWriter

memory data source

results matching ""

    No results matching ""