apply(
batchWatermarkMs: Long,
batchTimestampMs: Long,
sessionConf: RuntimeConfig): OffsetSeqMetadata
OffsetSeqMetadata — Metadata of Streaming Batch
OffsetSeqMetadata
holds the metadata for the current streaming batch:
-
Event-time watermark threshold
-
Batch timestamp (in millis)
-
Streaming configuration with
spark.sql.shuffle.partitions
and spark.sql.streaming.stateStore.providerClass Spark properties
Note
|
OffsetSeqMetadata is used mainly when IncrementalExecution is created.
|
OffsetSeqMetadata
considers some configuration properties as relevantSQLConfs:
relevantSQLConfs
are used when OffsetSeqMetadata
is created and is requested to setSessionConf.