SpecificParquetRecordReaderBase — Hadoop RecordReader

SpecificParquetRecordReaderBase is the base Hadoop RecordReader for parquet format readers that directly materialize to T.

RecordReader reads <key, value> pairs from an Hadoop InputSplit.
VectorizedParquetRecordReader is the one and only SpecificParquetRecordReaderBase that directly materialize to Java Objects.
Table 1. SpecificParquetRecordReaderBase’s Internal Properties (e.g. Registries, Counters and Flags)
Name Description


Spark schema

Initialized when SpecificParquetRecordReaderBase is requested to initialize (from the value of org.apache.spark.sql.parquet.row.requested_schema configuration as set when ParquetFileFormat is requested to build a data reader with partition column values appended)

initialize Method

void initialize(InputSplit inputSplit, TaskAttemptContext taskAttemptContext)
initialize is part of RecordReader Contract to initialize a RecordReader.


results matching ""

    No results matching ""