BasicStatsPlanVisitor — Computing Statistics for Cost-Based Optimization

BasicStatsPlanVisitor is a LogicalPlanVisitor that computes the statistics of a logical query plan for cost-based optimization (i.e. when cost-based optimization is enabled).

Note
 Cost-based optimization is enabled when spark.sql.cbo.enabled configuration property is on, i.e. true, and is disabled by default.

BasicStatsPlanVisitor is used exclusively when a logical operator is requested for the statistics with cost-based optimization enabled.

BasicStatsPlanVisitor comes with custom handlers for a few logical operators and falls back to SizeInBytesOnlyStatsPlanVisitor for the others.

Table 1. BasicStatsPlanVisitor’s Visitor Handlers
Logical Operator Handler Behaviour

Aggregate

visitAggregate

Requests AggregateEstimation for statistics estimates and query hints or falls back to SizeInBytesOnlyStatsPlanVisitor

Filter

visitFilter

Requests FilterEstimation for statistics estimates and query hints or falls back to SizeInBytesOnlyStatsPlanVisitor

Join

visitJoin

Requests JoinEstimation for statistics estimates and query hints or falls back to SizeInBytesOnlyStatsPlanVisitor

Project

visitProject

Requests ProjectEstimation for statistics estimates and query hints or falls back to SizeInBytesOnlyStatsPlanVisitor

results matching ""

    No results matching ""