ExchangeCoordinator

ExchangeCoordinator is created when EnsureRequirements physical query optimization is requested to add an ExchangeCoordinator for Adaptive Query Execution.

ExchangeCoordinator takes the following to be created:

ExchangeCoordinator keeps track of ShuffleExchangeExec unary physical operators that were registered (when ShuffleExchangeExec unary physical operator was requested to prepare itself for execution).

ExchangeCoordinator uses the following text representation (i.e. toString):

coordinator[target post-shuffle partition size: [advisoryTargetPostShuffleInputSize]]

postShuffleRDD Method

postShuffleRDD(exchange: ShuffleExchangeExec): ShuffledRowRDD

postShuffleRDD…​FIXME

Note
postShuffleRDD is used exclusively when ShuffleExchangeExec unary physical operator is requested to execute.

doEstimationIfNecessary Internal Method

doEstimationIfNecessary(): Unit

doEstimationIfNecessary…​FIXME

Note
doEstimationIfNecessary is used exclusively when ExchangeCoordinator is requested for a post-shuffle RDD (ShuffledRowRDD).

estimatePartitionStartIndices Method

estimatePartitionStartIndices(
  mapOutputStatistics: Array[MapOutputStatistics]): Array[Int]

estimatePartitionStartIndices…​FIXME

Note
estimatePartitionStartIndices is used exclusively when ExchangeCoordinator is requested for a doEstimationIfNecessary.

registerExchange Method

registerExchange(exchange: ShuffleExchangeExec): Unit

registerExchange simply adds the ShuffleExchangeExec unary physical operator to the exchanges internal registry.

Note
registerExchange is used exclusively when ShuffleExchangeExec unary physical operator is requested to prepare itself for execution.

results matching ""

    No results matching ""