doExecute(): RDD[InternalRow]StreamingGlobalLimitExec Unary Physical Operator
StreamingGlobalLimitExec is a unary physical operator that represents a Limit logical operator of a streaming query at execution time.
|
Note
|
A unary physical operator ( Read up on UnaryExecNode (and physical operators in general) in The Internals of Spark SQL book. |
StreamingGlobalLimitExec is created exclusively when StreamingGlobalLimitStrategy execution planning strategy is requested to plan a Limit logical operator (in the logical plan of a streaming query) for execution.
|
Note
|
Read up on Limit Logical Operator in The Internals of Spark SQL book. |
StreamingGlobalLimitExec is a stateful physical operator that can write to a state store.
StreamingGlobalLimitExec supports Append output mode only.
The optional properties, i.e. the StatefulOperatorStateInfo and the output mode, are initially undefined when StreamingGlobalLimitExec is created. StreamingGlobalLimitExec is updated to hold execution-specific configuration when IncrementalExecution is requested to prepare the logical plan (of a streaming query) for execution (when the state preparation rule is executed).
Creating StreamingGlobalLimitExec Instance
StreamingGlobalLimitExec takes the following to be created:
-
StatefulOperatorStateInfo (default:
None) -
OutputMode (default:
None)
StreamingGlobalLimitExec initializes the internal properties.
StreamingGlobalLimitExec as StateStoreWriter
StreamingGlobalLimitExec is a stateful physical operator that can write to a state store.
Performance Metrics
StreamingGlobalLimitExec uses the performance metrics of the parent StateStoreWriter.
Executing Physical Operator (Generating RDD[InternalRow]) — doExecute Method
|
Note
|
doExecute is part of SparkPlan Contract to generate the runtime representation of an physical operator as a recipe for distributed computation over internal binary rows on Apache Spark (RDD[InternalRow]).
|
doExecute…FIXME