SpecificParquetRecordReaderBase — Hadoop RecordReader

SpecificParquetRecordReaderBase is the base Hadoop RecordReader for parquet format readers that directly materialize to T.

Note
RecordReader reads <key, value> pairs from an Hadoop InputSplit.
Note
VectorizedParquetRecordReader is the one and only SpecificParquetRecordReaderBase that directly materialize to Java Objects.
Table 1. SpecificParquetRecordReaderBase’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description

sparkSchema

Spark schema

Initialized when SpecificParquetRecordReaderBase is requested to initialize (from the value of org.apache.spark.sql.parquet.row.requested_schema configuration as set when ParquetFileFormat is requested to build a data reader with partition column values appended)

initialize Method

void initialize(InputSplit inputSplit, TaskAttemptContext taskAttemptContext)
Note
initialize is part of RecordReader Contract to initialize a RecordReader.

initialize…​FIXME

results matching ""

    No results matching ""