org.apache.hive.hcatalog.pig

Class HCatLoader

java.lang.Object

LoadFunc

org.apache.hive.hcatalog.pig.HCatLoader

public class HCatLoader
extends LoadFunc

Pig org.apache.pig.LoadFunc to read data from HCat

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	INNER_SIGNATURE
static java.lang.String	INNER_SIGNATURE_PREFIX
protected static java.lang.String	PRUNE_PROJECTION_INFO
protected java.lang.String	signature

Constructor Summary

Constructors

Constructor and Description
HCatLoader()

Method Summary

Modifier and Type	Method and Description
java.util.List<OperatorSet>	getFeatures()
<any>	getInputFormat()
Tuple	getNext()
java.lang.String[]	getPartitionKeys(java.lang.String location, Job job)
ResourceSchema	getSchema(java.lang.String location, Job job)
protected static long	getSizeInBytes(InputJobInfo inputJobInfo) A utility method to get the size of inputs.
ResourceStatistics	getStatistics(java.lang.String location, Job job) Get statistics about the data to be loaded.
void	prepareToRead(RecordReader reader, PigSplit arg1)
RequiredFieldResponse	pushProjection(RequiredFieldList requiredFieldsInfo)
java.lang.String	relativeToAbsolutePath(java.lang.String location, Path curDir)
void	setLocation(java.lang.String location, Job job)
void	setPartitionFilter(Expression partitionFilter)
void	setUDFContextSignature(java.lang.String signature)
protected void	storeInUDFContext(java.lang.String signature, java.lang.String key, java.lang.Object value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INNER_SIGNATURE

public static final java.lang.String INNER_SIGNATURE

See Also:

Constant Field Values

INNER_SIGNATURE_PREFIX

public static final java.lang.String INNER_SIGNATURE_PREFIX

See Also:

Constant Field Values

PRUNE_PROJECTION_INFO

protected static final java.lang.String PRUNE_PROJECTION_INFO

See Also:

Constant Field Values

signature

protected java.lang.String signature

Constructor Detail

HCatLoader

public HCatLoader()

Method Detail

getInputFormat

public <any> getInputFormat()
                     throws java.io.IOException

Throws:

java.io.IOException

relativeToAbsolutePath

public java.lang.String relativeToAbsolutePath(java.lang.String location,
                                               Path curDir)
                                        throws java.io.IOException

Throws:

java.io.IOException

setLocation

public void setLocation(java.lang.String location,
                        Job job)
                 throws java.io.IOException

Throws:

java.io.IOException

getPartitionKeys

public java.lang.String[] getPartitionKeys(java.lang.String location,
                                           Job job)
                                    throws java.io.IOException

Throws:

java.io.IOException

getSchema

public ResourceSchema getSchema(java.lang.String location,
                                Job job)
                         throws java.io.IOException

Throws:

java.io.IOException

setPartitionFilter

public void setPartitionFilter(Expression partitionFilter)
                        throws java.io.IOException

Throws:

java.io.IOException

getStatistics

public ResourceStatistics getStatistics(java.lang.String location,
                                        Job job)
                                 throws java.io.IOException

Get statistics about the data to be loaded. Only input data size is implemented at this time.

Throws:

java.io.IOException

getNext

public Tuple getNext()
              throws java.io.IOException

Throws:

java.io.IOException

prepareToRead

public void prepareToRead(RecordReader reader,
                          PigSplit arg1)
                   throws java.io.IOException

Throws:

java.io.IOException

getFeatures

public java.util.List<OperatorSet> getFeatures()

pushProjection

public RequiredFieldResponse pushProjection(RequiredFieldList requiredFieldsInfo)
                                     throws FrontendException

Throws:

FrontendException

setUDFContextSignature

public void setUDFContextSignature(java.lang.String signature)

storeInUDFContext

protected void storeInUDFContext(java.lang.String signature,
                                 java.lang.String key,
                                 java.lang.Object value)

getSizeInBytes

protected static long getSizeInBytes(InputJobInfo inputJobInfo)
                              throws java.io.IOException

A utility method to get the size of inputs. This is accomplished by summing the size of all input paths on supported FileSystems. Locations whose size cannot be determined are ignored. Note non-FileSystem and unpartitioned locations will not report their input size by default.

Throws:

java.io.IOException