PullupCorrelatedPredicates Logical Optimization

PullupCorrelatedPredicates is a base logical optimization that transforms logical plans with the following operators:

  1. Filter operators with an Aggregate child operator

  2. UnaryNode operators

PullupCorrelatedPredicates is part of the Pullup Correlated Expressions once-executed batch in the standard batches of the Catalyst Optimizer.

PullupCorrelatedPredicates is simply a Catalyst rule for transforming logical plans, i.e. Rule[LogicalPlan].

import org.apache.spark.sql.catalyst.optimizer.PullupCorrelatedPredicates

// FIXME
// Demo: Filter + Aggregate
// Demo: Filter + UnaryNode

val plan = ???
val optimizedPlan = PullupCorrelatedPredicates(plan)

PullupCorrelatedPredicates uses PredicateHelper for…​FIXME

pullOutCorrelatedPredicates Internal Method

pullOutCorrelatedPredicates(
  sub: LogicalPlan,
  outer: Seq[LogicalPlan]): (LogicalPlan, Seq[Expression])

pullOutCorrelatedPredicates…​FIXME

Note
 pullOutCorrelatedPredicates is used exclusively when PullupCorrelatedPredicates is requested to rewriteSubQueries.

rewriteSubQueries Internal Method

rewriteSubQueries(plan: LogicalPlan, outerPlans: Seq[LogicalPlan]): LogicalPlan

rewriteSubQueries…​FIXME

Note
 rewriteSubQueries is used exclusively when PullupCorrelatedPredicates is executed (i.e. applied to a logical plan).

Executing Rule — apply Method

apply(plan: LogicalPlan): LogicalPlan
Note
 apply is part of the Rule Contract to execute (apply) a rule on a TreeNode (e.g. LogicalPlan).

apply transforms the input logical plan as follows:

  1. For Filter operators with an Aggregate child operator, apply rewriteSubQueries with the Filter and the Aggregate and its child as the outer plans

  2. For UnaryNode operators, apply rewriteSubQueries with the operator and its children as the outer plans

results matching ""

    No results matching ""