BoundReference Leaf Expression — Reference to Value in Internal Binary Row

BoundReference is a leaf expression that evaluates to a value in an internal binary row at a specified position and of a given data type.

BoundReference takes the following when created:

  • Ordinal, i.e. the position

  • Data type of the value

  • nullable flag that controls whether the value can be null or not

import org.apache.spark.sql.catalyst.expressions.BoundReference
import org.apache.spark.sql.types.LongType
val boundRef = BoundReference(ordinal = 0, dataType = LongType, nullable = true)

scala> println(boundRef.toString)
input[0, bigint, true]

import org.apache.spark.sql.catalyst.InternalRow
val row = InternalRow(1L, "hello")

val value = boundRef.eval(row).asInstanceOf[Long]

You can also create a BoundReference using Catalyst DSL’s at method.

import org.apache.spark.sql.catalyst.dsl.expressions._
val boundRef = 'hello.string.at(4)
scala> println(boundRef)
input[4, string, true]

Evaluating Expression — eval Method

eval(input: InternalRow): Any
Note
 eval is part of Expression Contract for the interpreted (non-code-generated) expression evaluation, i.e. evaluating a Catalyst expression to a JVM object for a given internal binary row.

eval gives the value at position from the input internal binary row that is of a correct type.

Internally, eval returns null if the value at the position is null.

Otherwise, eval uses the methods of InternalRow per the defined data type to access the value.

Table 1. eval’s DataType to InternalRow’s Methods Mapping (in execution order)
DataType InternalRow’s Method

BooleanType

getBoolean

ByteType

getByte

ShortType

getShort

IntegerType or DateType

getInt

LongType or TimestampType

getLong

FloatType

getFloat

DoubleType

getDouble

StringType

getUTF8String

BinaryType

getBinary

CalendarIntervalType

getInterval

DecimalType

getDecimal

StructType

getStruct

ArrayType

getArray

MapType

getMap

others

get(ordinal, dataType)

Generating Java Source Code (ExprCode) For Code-Generated Expression Evaluation — doGenCode Method

doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode
Note
 doGenCode is part of Expression Contract to generate a Java source code (ExprCode) for code-generated expression evaluation.

doGenCode…​FIXME

BindReferences.bindReference Method

bindReference[A <: Expression](
  expression: A,
  input: AttributeSeq,
  allowFailures: Boolean = false): A

bindReference…​FIXME

Note
 bindReference is used when…​FIXME

results matching ""

    No results matching ""