QueryPlanner — Converting Logical Plan to Physical Trees

QueryPlanner plans a logical plan for execution, i.e. converts a logical plan to one or more physical plans using strategies.

Note
QueryPlanner generates at least one physical plan.

QueryPlanner's main method is plan that defines the extension points, i.e. strategies, collectPlaceholders and prunePlans.

QueryPlanner is part of Catalyst Framework.

QueryPlanner Contract

abstract class QueryPlanner[PhysicalPlan <: TreeNode[PhysicalPlan]] {
  def collectPlaceholders(plan: PhysicalPlan): Seq[(PhysicalPlan, LogicalPlan)]
  def prunePlans(plans: Iterator[PhysicalPlan]): Iterator[PhysicalPlan]
  def strategies: Seq[GenericStrategy[PhysicalPlan]]
}
Table 1. QueryPlanner Contract
Method Description

strategies

Collection of GenericStrategy planning strategies.

Used exclusively as an extension point in plan.

collectPlaceholders

Collection of "placeholder" physical plans and the corresponding logical plans.

Used exclusively as an extension point in plan.

Overriden in SparkPlanner

prunePlans

Prunes physical plans (e.g. bad or somehow incorrect plans).

Used exclusively as an extension point in plan.

Planning Logical Plan — plan Method

plan(plan: LogicalPlan): Iterator[PhysicalPlan]

plan converts the input plan logical plan to zero or more PhysicalPlan plans.

Internally, plan applies planning strategies to the input plan (one by one collecting all as the plan candidates).

plan then walks over the plan candidates to collect placeholders.

If a plan does not contain a placeholder, the plan is returned as is. Otherwise, plan walks over placeholders (as pairs of PhysicalPlan and unplanned logical plan) and (recursively) plans the child logical plan. plan then replaces the placeholders with the planned child logical plan.

In the end, plan prunes "bad" physical plans.

Note
plan is used exclusively (through the concrete SparkPlanner) when a QueryExecution is requested for a physical plan.

results matching ""

    No results matching ""