Trigger is used to define how often a streaming query should be executed to produce results.

Trigger is a sealed trait so all available implementations are in the same file Trigger.scala.
A trigger can also be considered a batch (as in Spark Streaming).

Import org.apache.spark.sql to work with Trigger and the only implementation ProcessingTime.

import org.apache.spark.sql._


ProcessingTime is the only available implementation of Trigger sealed trait. It assumes that milliseconds is the minimum time unit.

You can create an instance of ProcessingTime using the following constructors:

  • ProcessingTime(Long) that accepts non-negative values that represent milliseconds.

  • ProcessingTime(interval: String) or ProcessingTime.create(interval: String) that accept CalendarInterval instances with or without leading interval string.

    ProcessingTime("10 milliseconds")
    ProcessingTime("interval 10 milliseconds")
  • ProcessingTime(Duration) that accepts scala.concurrent.duration.Duration instances.

  • ProcessingTime.create(interval: Long, unit: TimeUnit) for Long and java.util.concurrent.TimeUnit instances.

    ProcessingTime.create(10, TimeUnit.SECONDS)

results matching ""

    No results matching ""