CatalystSerde Helper Object

CatalystSerde is a Scala object that consists of three utility methods:

  1. deserialize to create a new logical plan with the input logical plan wrapped inside DeserializeToObject logical operator.

  2. serialize

  3. generateObjAttr

CatalystSerde and belongs to org.apache.spark.sql.catalyst.plans.logical package.

Creating Logical Plan with DeserializeToObject Logical Operator for Logical Plan — deserialize Method

deserialize[T : Encoder](child: LogicalPlan): DeserializeToObject

deserialize creates a DeserializeToObject logical operator for the input child logical plan.

Internally, deserialize creates a UnresolvedDeserializer for the deserializer for the type T first and passes it on to a DeserializeToObject with a AttributeReference (being the result of generateObjAttr).

serialize Method

serialize[T : Encoder](child: LogicalPlan): SerializeFromObject

generateObjAttr Method

generateObjAttr[T : Encoder]: Attribute

results matching ""

    No results matching ""