scala> spark.version
res0: String = 2.3.0-SNAPSHOT
val rates = spark.
readStream.
format("rate").
load
// StreamingRelation logical operator
scala> println(rates.queryExecution.logical.numberedTreeString)
00 StreamingRelation DataSource(org.apache.spark.sql.SparkSession@31ba0af0,rate,List(),None,List(),None,Map(),None), rate, [timestamp#0, value#1L]
// StreamingRelationExec physical operator (shown without "Exec" suffix)
scala> rates.explain
== Physical Plan ==
StreamingRelation rate, [timestamp#0, value#1L]
StreamingRelationExec Leaf Physical Operator
StreamingRelationExec
is a leaf physical operator (i.e. LeafExecNode
) that…FIXME
StreamingRelationExec
is created when StreamingRelationStrategy
plans StreamingRelation
and StreamingExecutionRelation
logical operators.
StreamingRelationExec
is not supposed to be executed and is used…FIXME
Creating StreamingRelationExec Instance
StreamingRelationExec
takes the following when created:
-
The name of a streaming data source