PhysicalOperation — Scala Extractor for Destructuring Logical Query Plans

PhysicalOperation is a Scala extractor to destructure a logical query plan into a tuple with the following elements:

  1. Named expressions (aka projects)

  2. Expressions (aka filters)

  3. Logical operator (aka leaf node)

ReturnType
(Seq[NamedExpression], Seq[Expression], LogicalPlan)

The following idiom is often used in Strategy implementations (e.g. HiveTableScans, InMemoryScans, DataSourceStrategy, FileSourceStrategy):

def apply(plan: LogicalPlan): Seq[SparkPlan] = plan match {
  case PhysicalOperation(projections, predicates, plan) =>
    // do something
  case _ => Nil
}

Whenever used to pattern match to a LogicalPlan, PhysicalOperation's unapply is called.

unapply Method

type ReturnType = (Seq[NamedExpression], Seq[Expression], LogicalPlan)

unapply(plan: LogicalPlan): Option[ReturnType]

unapply…​FIXME

Note
 unapply is almost collectProjectsAndFilters method itself (with some manipulations of the return value).
Note

unapply is used when…​FIXME

results matching ""

    No results matching ""