apply(
batchWatermarkMs: Long,
batchTimestampMs: Long,
sessionConf: RuntimeConfig): OffsetSeqMetadata
OffsetSeqMetadata — Metadata of Streaming Batch
OffsetSeqMetadata holds the metadata for the current streaming batch:
-
Event-time watermark threshold
-
Batch timestamp (in millis)
-
Streaming configuration with
spark.sql.shuffle.partitionsand spark.sql.streaming.stateStore.providerClass Spark properties
|
Note
|
OffsetSeqMetadata is used mainly when IncrementalExecution is created.
|
OffsetSeqMetadata considers some configuration properties as relevantSQLConfs:
relevantSQLConfs are used when OffsetSeqMetadata is created and is requested to setSessionConf.