OneSideHashJoiner

OneSideHashJoiner is created exclusively when StreamingSymmetricHashJoinExec physical operator is requested to execute and generate a recipe for a distributed computation (as an RDD[InternalRow]) (through process partitions).

Note
OneSideHashJoiner is a Scala private internal class of StreamingSymmetricHashJoinExec.

storeAndJoinWithOtherSide Method

storeAndJoinWithOtherSide(
  otherSideJoiner: OneSideHashJoiner)(
  generateJoinedRow: (InternalRow, InternalRow) => JoinedRow): Iterator[InternalRow]

storeAndJoinWithOtherSide…​FIXME

Note
storeAndJoinWithOtherSide is used when StreamingSymmetricHashJoinExec physical operator is requested to process partitions (when requested to execute and generate a recipe for a distributed computation (as an RDD[InternalRow])).

Creating OneSideHashJoiner Instance

OneSideHashJoiner takes the following to be created:

  • JoinSide

  • Input attributes (Seq[Attribute])

  • Join keys (Seq[Expression])

  • Input internal rows (Iterator[InternalRow])

  • Optional pre-join filter Catalyst expression

  • Post-join filter ((InternalRow) ⇒ Boolean)

  • Optional JoinStateWatermarkPredicate

OneSideHashJoiner initializes the internal registries and counters.

removeOldState Method

removeOldState(): Iterator[UnsafeRowPair]

removeOldState…​FIXME

Note
removeOldState is used when…​FIXME

Getting Values For Key — get Method

get(key: UnsafeRow): Iterator[UnsafeRow]

get…​FIXME

Note
get is used when…​FIXME

commitStateAndGetMetrics Method

commitStateAndGetMetrics(): StateStoreMetrics

commitStateAndGetMetrics simply requests the SymmetricHashJoinStateManager to commit and then for the metrics.

Note
commitStateAndGetMetrics is used exclusively when StreamingSymmetricHashJoinExec physical operator is requested to processPartitions (when requested to execute and generate a recipe for a distributed computation (as an RDD[InternalRow])).

Internal Properties

Name Description

joinStateManager

SymmetricHashJoinStateManager

Used when…​FIXME

keyGenerator

UnsafeProjection to generate join keys

Used when…​FIXME

preJoinFilter

InternalRow ⇒ Boolean

Used when…​FIXME

stateKeyWatermarkPredicateFunc

InternalRow ⇒ Boolean

Used when…​FIXME

stateValueWatermarkPredicateFunc

InternalRow ⇒ Boolean

Used when…​FIXME

updatedStateRowsCount

Counter

Used exclusively when requested to storeAndJoinWithOtherSide

results matching ""

    No results matching ""