org.apache.hadoop.hive.ql.io

Class CombineHiveInputFormat<K extends WritableComparable,V extends Writable>

java.lang.Object

org.apache.hadoop.hive.ql.io.HiveInputFormat<K,V>

org.apache.hadoop.hive.ql.io.CombineHiveInputFormat<K,V>

public class CombineHiveInputFormat<K extends WritableComparable,V extends Writable>
extends HiveInputFormat<K,V>

CombineHiveInputFormat is a parameterized InputFormat which looks at the path name and determine the correct InputFormat for that path name from mapredPlan.pathToPartitionInfo(). It can be used to read files with different input format in the same map-reduce job.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CombineHiveInputFormat.CombineHiveInputSplit CombineHiveInputSplit encapsulates an InputSplit with its corresponding inputFormatClassName.

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

HiveInputFormat.HiveInputSplit

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.commons.logging.Log	LOG

Fields inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

inputFormats, pathToPartitionInfo

Constructor Summary

Constructors

Constructor and Description
CombineHiveInputFormat()

Method Summary

Modifier and Type	Method and Description
RecordReader	getRecordReader(InputSplit split, JobConf job, Reporter reporter) Create a generic Hive RecordReader than can iterate over all chunks in a CombinedFileSplit.
InputSplit[]	getSplits(JobConf job, int numSplits) Create Hive splits based on CombineFileSplit.

Methods inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

configure, getInputFormatFromCache, getPartitionDescFromPath, init, pushFilters, pushProjectionsAndFilters, pushProjectionsAndFilters, validateInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Constructor Detail

CombineHiveInputFormat

public CombineHiveInputFormat()

Method Detail

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws java.io.IOException

Create Hive splits based on CombineFileSplit.

Overrides:

getSplits in class HiveInputFormat<K extends WritableComparable,V extends Writable>

Throws:

java.io.IOException

getRecordReader

public RecordReader getRecordReader(InputSplit split,
                                    JobConf job,
                                    Reporter reporter)
                             throws java.io.IOException

Create a generic Hive RecordReader than can iterate over all chunks in a CombinedFileSplit.

Overrides:

getRecordReader in class HiveInputFormat<K extends WritableComparable,V extends Writable>

Throws:

java.io.IOException