Powered by GitBook

StreamingJoinHelper Utility

StreamingJoinHelper is a Scala object with the following utility methods:

getStateValueWatermark

Tip	Enable `ALL` logging level for `org.apache.spark.sql.catalyst.analysis.StreamingJoinHelper` to see what happens inside. Add the following line to `conf/log4j.properties`: `log4j.logger.org.apache.spark.sql.catalyst.analysis.StreamingJoinHelper=ALL` Refer to Logging.

State Value Watermark — `getStateValueWatermark` Object Method

getStateValueWatermark(
  attributesToFindStateWatermarkFor: AttributeSet,
  attributesWithEventWatermark: AttributeSet,
  joinCondition: Option[Expression],
  eventWatermark: Option[Long]): Option[Long]

getStateValueWatermark…FIXME

Note	`getStateValueWatermark` is used when: `UnsupportedOperationChecker` utility is used to checkForStreaming `StreamingSymmetricHashJoinHelper` utility is used to create a JoinStateWatermarkPredicates

results matching ""

No results matching ""