org.apache.hadoop.hive.ql.io

Class HiveContextAwareRecordReader<K,V>

java.lang.Object

org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader<K,V>

Direct Known Subclasses:

BucketizedHiveRecordReader, CombineHiveRecordReader, HiveRecordReader

public abstract class HiveContextAwareRecordReader<K,V>
extends java.lang.Object

This class prepares an IOContext, and provides the ability to perform a binary search on the data. The binary search can be used by setting the value of inputFormatSorted in the MapreduceWork to true, but it should only be used if the data is going to a FilterOperator, which filters by comparing a value in the data with a constant, using one of the comparisons =, <, >, <=, >=. If the RecordReader's underlying format is an RCFile, this object can perform a binary search to find the block to begin reading from, and stop reading once it can be determined no other entries will match the filter.

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	isSorted
protected JobConf	jobConf
protected RecordReader	recordReader

Constructor Summary

Constructors

Constructor and Description
HiveContextAwareRecordReader(JobConf conf)
HiveContextAwareRecordReader(RecordReader recordReader)
HiveContextAwareRecordReader(RecordReader recordReader, JobConf conf)

Method Summary

Modifier and Type	Method and Description
void	close()
abstract void	doClose() Close this InputSplit to future operations.
boolean	doNext(K key, V value)
boolean	foundAllTargets() Returns true if the current comparison is in the list of stop comparisons, i.e.
IOContext	getIOContext()
float	getProgress()
void	initIOContext(FileSplit split, JobConf job, java.lang.Class inputFormatClass)
void	initIOContext(FileSplit split, JobConf job, java.lang.Class inputFormatClass, RecordReader recordReader)
void	initIOContext(long startPos, boolean isBlockPointer, java.lang.String inputFile)
void	initIOContextSortedProps(FileSplit split, RecordReader recordReader, JobConf job)
boolean	next(K key, V value)
void	setRecordReader(RecordReader recordReader)
protected void	updateIOContext()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

recordReader

protected RecordReader recordReader

jobConf

protected JobConf jobConf

isSorted

protected boolean isSorted

Constructor Detail

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(JobConf conf)
                             throws java.io.IOException

Throws:

java.io.IOException

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(RecordReader recordReader)

HiveContextAwareRecordReader

public HiveContextAwareRecordReader(RecordReader recordReader,
                                    JobConf conf)
                             throws java.io.IOException

Throws:

java.io.IOException

Method Detail

setRecordReader

public void setRecordReader(RecordReader recordReader)

doClose

public abstract void doClose()
                      throws java.io.IOException

Close this InputSplit to future operations.

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Throws:

java.io.IOException

next

public boolean next(K key,
                    V value)
             throws java.io.IOException

Throws:

java.io.IOException

updateIOContext

protected void updateIOContext()
                        throws java.io.IOException

Throws:

java.io.IOException

getIOContext

public IOContext getIOContext()

initIOContext

public void initIOContext(long startPos,
                          boolean isBlockPointer,
                          java.lang.String inputFile)

initIOContext

public void initIOContext(FileSplit split,
                          JobConf job,
                          java.lang.Class inputFormatClass)
                   throws java.io.IOException

Throws:

java.io.IOException

initIOContext

public void initIOContext(FileSplit split,
                          JobConf job,
                          java.lang.Class inputFormatClass,
                          RecordReader recordReader)
                   throws java.io.IOException

Throws:

java.io.IOException

initIOContextSortedProps

public void initIOContextSortedProps(FileSplit split,
                                     RecordReader recordReader,
                                     JobConf job)

getProgress

public float getProgress()
                  throws java.io.IOException

Throws:

java.io.IOException

doNext

public boolean doNext(K key,
                      V value)
               throws java.io.IOException

Throws:

java.io.IOException

foundAllTargets

public boolean foundAllTargets()

Returns true if the current comparison is in the list of stop comparisons, i.e. we've found all records which won't be filtered

Returns:

true if the current comparison is found