org.apache.hadoop.hive.ql.exec.persistence

Class RowContainer<ROW extends java.util.List<java.lang.Object>>

java.lang.Object

org.apache.hadoop.hive.ql.exec.persistence.AbstractRowContainer<ROW>

org.apache.hadoop.hive.ql.exec.persistence.RowContainer<ROW>

Direct Known Subclasses:

PTFRowContainer

public class RowContainer<ROW extends java.util.List<java.lang.Object>>
extends AbstractRowContainer<ROW>

Simple persistent container for rows. This container interface only accepts adding or appending new rows and iterating through the rows in the order of their insertions. The iterator interface is a lightweight first()/next() API rather than the Java Iterator interface. This way we do not need to create an Iterator object every time we want to start a new iteration. Below is simple example of how to convert a typical Java's Iterator code to the LW iterator interface. Iterator itr = rowContainer.iterator(); while (itr.hasNext()) { v = itr.next(); // do anything with v } can be rewritten to: for ( v = rowContainer.first(); v != null; v = rowContainer.next()) { // do anything with v } Once the first is called, it will not be able to write again. So there can not be any writes after read. It can be read multiple times, but it does not support multiple reader interleaving reading.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.commons.logging.Log	LOG

Constructor Summary

Constructors

Constructor and Description
RowContainer(Configuration jc, Reporter reporter)
RowContainer(int bs, Configuration jc, Reporter reporter)

Method Summary

Modifier and Type	Method and Description
void	add(ROW t)
void	clear() Remove all elements in the RowContainer.
protected void	close()
void	copyToDFSDirecory(FileSystem destFs, Path destPath)
protected boolean	endOfCurrentReadBlock()
ROW	first()
protected int	getAddCursor()
protected int	getBlockSize()
protected int	getCurrentReadBlockSize()
protected InputSplit[]	getInputSplits()
protected ROW	getReadBlockRow(int rowOffset)
protected FileSinkOperator.RecordWriter	getRecordWriter()
ROW	next()
protected boolean	nextBlock(int readIntoOffset)
protected void	resetCurrentReadBlockToFirstReadBlock()
protected void	resetReadBlocks()
void	setKeyObject(java.util.List<java.lang.Object> dummyKey)
protected org.apache.hadoop.mapred.RecordReader	setReaderAtSplit(int splitNum)
void	setSerDe(SerDe sd, ObjectInspector oi)
void	setTableDesc(TableDesc tblDesc)
protected void	setupWriter()
protected void	setWriteBlockAsReadBlock()
long	size() Get the number of elements in the RowContainer.
protected boolean	willSpill()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static org.apache.commons.logging.Log LOG

Constructor Detail

RowContainer

public RowContainer(Configuration jc,
                    Reporter reporter)
             throws HiveException

Throws:

HiveException

RowContainer

public RowContainer(int bs,
                    Configuration jc,
                    Reporter reporter)
             throws HiveException

Throws:

HiveException

Method Detail

setSerDe

public void setSerDe(SerDe sd,
                     ObjectInspector oi)

add

public void add(ROW t)
         throws HiveException

Specified by:

add in class AbstractRowContainer<ROW extends java.util.List<java.lang.Object>>

Throws:

HiveException

first

public ROW first()
          throws HiveException

Specified by:

first in class AbstractRowContainer<ROW extends java.util.List<java.lang.Object>>

Throws:

HiveException

next

public ROW next()
         throws HiveException

Specified by:

next in class AbstractRowContainer<ROW extends java.util.List<java.lang.Object>>

Throws:

HiveException

size

public long size()

Get the number of elements in the RowContainer.

Specified by:

size in class AbstractRowContainer<ROW extends java.util.List<java.lang.Object>>

Returns:

number of elements in the RowContainer

nextBlock

protected boolean nextBlock(int readIntoOffset)
                     throws HiveException

Throws:

HiveException

copyToDFSDirecory

public void copyToDFSDirecory(FileSystem destFs,
                              Path destPath)
                       throws java.io.IOException,
                              HiveException

Throws:

java.io.IOException

HiveException

clear

public void clear()
           throws HiveException

Remove all elements in the RowContainer.

Specified by:

clear in class AbstractRowContainer<ROW extends java.util.List<java.lang.Object>>

Throws:

HiveException

setKeyObject

public void setKeyObject(java.util.List<java.lang.Object> dummyKey)

setTableDesc

public void setTableDesc(TableDesc tblDesc)

getAddCursor

protected int getAddCursor()

willSpill

protected final boolean willSpill()

getBlockSize

protected int getBlockSize()

setupWriter

protected void setupWriter()
                    throws HiveException

Throws:

HiveException

getRecordWriter

protected FileSinkOperator.RecordWriter getRecordWriter()

getInputSplits

protected InputSplit[] getInputSplits()

endOfCurrentReadBlock

protected boolean endOfCurrentReadBlock()

getCurrentReadBlockSize

protected int getCurrentReadBlockSize()

setWriteBlockAsReadBlock

protected void setWriteBlockAsReadBlock()

setReaderAtSplit

protected org.apache.hadoop.mapred.RecordReader setReaderAtSplit(int splitNum)
                                                          throws java.io.IOException

Throws:

java.io.IOException

getReadBlockRow

protected ROW getReadBlockRow(int rowOffset)

resetCurrentReadBlockToFirstReadBlock

protected void resetCurrentReadBlockToFirstReadBlock()

resetReadBlocks

protected void resetReadBlocks()

close

protected void close()
              throws HiveException

Throws:

HiveException