CollectionGenerator Generator Expression Contract

CollectionGenerator is the contract in Spark SQL for Generator expressions that generate a collection object (i.e. an array or map) and (at execution time) use a different path for whole-stage Java code generation (while executing GenerateExec physical operator with Whole-Stage Java Code Generation enabled).

package org.apache.spark.sql.catalyst.expressions

trait CollectionGenerator extends Generator {
  def collectionType: DataType = dataType
  def inline: Boolean
  def position: Boolean
}
Table 1. CollectionGenerator Contract
Method Description

collectionType

The type of the returned collection object.

Used when…​

inline

Flag whether to inline rows during whole-stage Java code generation.

Used when…​

position

Flag whether to include the positions of elements within the result collection.

Used when…​
Table 2. CollectionGenerators
CollectionGenerator Description

Inline

ExplodeBase

Explode

PosExplode

results matching ""

    No results matching ""