stateStoreAwareZipPartitions[U: ClassTag, V: ClassTag](
dataRDD2: RDD[U],
stateInfo: StatefulOperatorStateInfo,
storeNames: Seq[String],
storeCoordinator: StateStoreCoordinatorRef
)(f: (Iterator[T], Iterator[U]) => Iterator[V]): RDD[V]StateStoreAwareZipPartitionsHelper — Extension Methods for Creating StateStoreAwareZipPartitionsRDD
StateStoreAwareZipPartitionsHelper is a Scala implicit class of a data RDD (of type RDD[T]) to create a StateStoreAwareZipPartitionsRDD for StreamingSymmetricHashJoinExec physical operator.
|
Note
|
Implicit Classes are a language feature in Scala for implicit conversions with extension methods for existing types. |
Creating StateStoreAwareZipPartitionsRDD — stateStoreAwareZipPartitions Method
stateStoreAwareZipPartitions simply creates a new StateStoreAwareZipPartitionsRDD.
|
Note
|
stateStoreAwareZipPartitions is used exclusively when StreamingSymmetricHashJoinExec physical operator is requested to execute and generate a recipe for a distributed computation (as an RDD[InternalRow]).
|