parquet.avro

Class AvroReadSupport<T extends org.apache.avro.generic.IndexedRecord>

java.lang.Object

parquet.hadoop.api.ReadSupport<T>

parquet.avro.AvroReadSupport<T>

public class AvroReadSupport<T extends org.apache.avro.generic.IndexedRecord>
extends ReadSupport<T>

Avro implementation of ReadSupport for Avro IndexedRecords which cover both Avro Specific and Generic. Users should use AvroParquetReader or AvroParquetInputFormat rather than using this class directly.

Nested Class Summary

Nested classes/interfaces inherited from class parquet.hadoop.api.ReadSupport

ReadSupport.ReadContext

Field Summary

Fields

Modifier and Type	Field and Description
static String	AVRO_REQUESTED_PROJECTION

Fields inherited from class parquet.hadoop.api.ReadSupport

PARQUET_READ_SCHEMA

Constructor Summary

Constructors

Constructor and Description
AvroReadSupport()

Method Summary

Modifier and Type	Method and Description
ReadSupport.ReadContext	init(org.apache.hadoop.conf.Configuration configuration, Map<String,String> keyValueMetaData, MessageType fileSchema) called in InputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext) in the front end
RecordMaterializer<T>	prepareForRead(org.apache.hadoop.conf.Configuration configuration, Map<String,String> keyValueMetaData, MessageType fileSchema, ReadSupport.ReadContext readContext) called in RecordReader.initialize(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext) in the back end the returned RecordConsumer will materialize the records and add them to the destination
static void	setAvroReadSchema(org.apache.hadoop.conf.Configuration configuration, org.apache.avro.Schema avroReadSchema)
static void	setRequestedProjection(org.apache.hadoop.conf.Configuration configuration, org.apache.avro.Schema requestedProjection)

Methods inherited from class parquet.hadoop.api.ReadSupport

getSchemaForRead, getSchemaForRead, init

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

AVRO_REQUESTED_PROJECTION

public static String AVRO_REQUESTED_PROJECTION

Constructor Detail

AvroReadSupport

public AvroReadSupport()

Method Detail

setRequestedProjection

public static void setRequestedProjection(org.apache.hadoop.conf.Configuration configuration,
                                          org.apache.avro.Schema requestedProjection)

See Also:

AvroParquetInputFormat.setRequestedProjection(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)

setAvroReadSchema

public static void setAvroReadSchema(org.apache.hadoop.conf.Configuration configuration,
                                     org.apache.avro.Schema avroReadSchema)

See Also:

AvroParquetInputFormat.setAvroReadSchema(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)

init

public ReadSupport.ReadContext init(org.apache.hadoop.conf.Configuration configuration,
                                    Map<String,String> keyValueMetaData,
                                    MessageType fileSchema)

Description copied from class: ReadSupport

called in InputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext) in the front end

Overrides:

init in class ReadSupport<T extends org.apache.avro.generic.IndexedRecord>

Parameters:

configuration - the job configuration

keyValueMetaData - the app specific metadata from the file

fileSchema - the schema of the file

Returns:

the readContext that defines how to read the file

prepareForRead

public RecordMaterializer<T> prepareForRead(org.apache.hadoop.conf.Configuration configuration,
                                            Map<String,String> keyValueMetaData,
                                            MessageType fileSchema,
                                            ReadSupport.ReadContext readContext)

Description copied from class: ReadSupport

called in RecordReader.initialize(org.apache.hadoop.mapreduce.InputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext) in the back end the returned RecordConsumer will materialize the records and add them to the destination

Specified by:

prepareForRead in class ReadSupport<T extends org.apache.avro.generic.IndexedRecord>

Parameters:

configuration - the job configuration

keyValueMetaData - the app specific metadata from the file

fileSchema - the schema of the file

readContext - returned by the init method

Returns:

the recordConsumer that will receive the events