SubExprUtils Helper Object

SubExprUtils is a Scala object that is used for…​FIXME

SubExprUtils uses PredicateHelper for…​FIXME

Checking If Condition Expression Has Any Null-Aware Predicate Subqueries Inside Not — hasNullAwarePredicateWithinNot Method

hasNullAwarePredicateWithinNot(condition: Expression): Boolean

hasNullAwarePredicateWithinNot splits conjunctive predicates (i.e. expressions separated by And expression).

hasNullAwarePredicateWithinNot is positive (i.e. true) and is considered to have a null-aware predicate subquery inside a Not expression when conjuctive predicate expressions include a Not expression with an In predicate expression with a ListQuery subquery expression.

import org.apache.spark.sql.catalyst.plans.logical._
import org.apache.spark.sql.catalyst.dsl.plans._
val plan = LocalRelation('key.int, 'value.string).analyze

import org.apache.spark.sql.catalyst.expressions._
val in = In(value = Literal.create(1), Seq(ListQuery(plan)))
val condition = Not(child = Or(left = Literal.create(false), right = in))

import org.apache.spark.sql.catalyst.expressions.SubExprUtils
val positive = SubExprUtils.hasNullAwarePredicateWithinNot(condition)
assert(positive)

hasNullAwarePredicateWithinNot is negative (i.e. false) for all the other expressions and in particular the following expressions:

  1. Exists predicate subquery expressions

  2. Not expressions with a Exists predicate subquery expression as the child expression

  3. In expressions with a ListQuery subquery expression as the list expression

  4. Not expressions with a In expression (with a ListQuery subquery expression as the list expression)

import org.apache.spark.sql.catalyst.plans.logical._
import org.apache.spark.sql.catalyst.dsl.plans._
val plan = LocalRelation('key.int, 'value.string).analyze

import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.SubExprUtils

// Exists
val condition = Exists(plan)
val negative = SubExprUtils.hasNullAwarePredicateWithinNot(condition)
assert(!negative)

// Not Exists
val condition = Not(child = Exists(plan))
val negative = SubExprUtils.hasNullAwarePredicateWithinNot(condition)
assert(!negative)

// In with ListQuery
val condition = In(value = Literal.create(1), Seq(ListQuery(plan)))
val negative = SubExprUtils.hasNullAwarePredicateWithinNot(condition)
assert(!negative)

// Not In with ListQuery
val in = In(value = Literal.create(1), Seq(ListQuery(plan)))
val condition = Not(child = in)
val negative = SubExprUtils.hasNullAwarePredicateWithinNot(condition)
assert(!negative)
Note
hasNullAwarePredicateWithinNot is used exclusively when CheckAnalysis analysis validation is requested to validate analysis of a logical plan (with Filter logical operators).

results matching ""

    No results matching ""