WindowsSubstitution Logical Evaluation Rule

WindowsSubstitution is a logical evaluation rule that Spark SQL’s logical query plan analyzer uses to resolve (aka substitute) WithWindowDefinition unary logical operators with UnresolvedWindowExpression to their corresponding WindowExpression with resolved WindowSpecDefinition.

WindowsSubstitution is a part of Substitution fixed-point batch of rules.

It appears that WindowsSubstitution is exclusively used for pure SQL queries because WithWindowDefinition unary logical operator is created exclusively when AstBuilder parses window definitions.

If a window specification is not found, WindowsSubstitution fails analysis with the following error:

Window specification [windowName] is not defined in the WINDOW clause.
The analysis failure is unlikely to happen given AstBuilder builds a lookup table of all the named window specifications defined in a SQL text and reports a ParseException when a WindowSpecReference is not available earlier.

For every WithWindowDefinition, WindowsSubstitution takes the child logical plan and transforms its UnresolvedWindowExpression expressions to be a WindowExpression with a window specification from the WINDOW clause (see WithWindowDefinition Example).

results matching ""

    No results matching ""