org.apache.hadoop.hive.ql.io

Class BucketizedHiveInputFormat<K extends WritableComparable,V extends Writable>

java.lang.Object

org.apache.hadoop.hive.ql.io.HiveInputFormat<K,V>

org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat<K,V>

public class BucketizedHiveInputFormat<K extends WritableComparable,V extends Writable>
extends HiveInputFormat<K,V>

BucketizedHiveInputFormat serves the similar function as hiveInputFormat but its getSplits() always group splits from one input file into one wrapper split. It is useful for the applications that requires input files to fit in one mapper.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

HiveInputFormat.HiveInputSplit

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.commons.logging.Log	LOG

Fields inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

inputFormats, pathToPartitionInfo

Constructor Summary

Constructors

Constructor and Description
BucketizedHiveInputFormat()

Method Summary

Modifier and Type	Method and Description
RecordReader	getRecordReader(InputSplit split, JobConf job, Reporter reporter)
InputSplit[]	getSplits(JobConf job, int numSplits)
protected FileStatus[]	listStatus(JobConf job, Path path)

Methods inherited from class org.apache.hadoop.hive.ql.io.HiveInputFormat

configure, getInputFormatFromCache, getPartitionDescFromPath, init, pushFilters, pushProjectionsAndFilters, pushProjectionsAndFilters, validateInput

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

Constructor Detail

BucketizedHiveInputFormat

public BucketizedHiveInputFormat()

Method Detail

getRecordReader

public RecordReader getRecordReader(InputSplit split,
                                    JobConf job,
                                    Reporter reporter)
                             throws java.io.IOException

Overrides:

getRecordReader in class HiveInputFormat<K extends WritableComparable,V extends Writable>

Throws:

java.io.IOException

listStatus

protected FileStatus[] listStatus(JobConf job,
                                  Path path)
                           throws java.io.IOException

Throws:

java.io.IOException

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws java.io.IOException

Overrides:

getSplits in class HiveInputFormat<K extends WritableComparable,V extends Writable>

Throws:

java.io.IOException