org.apache.hive.hcatalog.mapreduce

Class HCatBaseInputFormat

java.lang.Object

<any>

org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat

Direct Known Subclasses:

HCatInputFormat

public abstract class HCatBaseInputFormat
extends <any>

Constructor Summary

Constructors

Constructor and Description
HCatBaseInputFormat()

Method Summary

Modifier and Type	Method and Description
<any>	createRecordReader(InputSplit split, TaskAttemptContext taskContext) Create the RecordReader for the given InputSplit.
protected static <any>	getMapRedInputFormat(JobConf job, java.lang.Class inputFormatClass)
java.util.List<InputSplit>	getSplits(JobContext jobContext) Logically split the set of input files for the job.
static HCatSchema	getTableSchema(Configuration conf) Gets the HCatTable schema for the table specified in the HCatInputFormat.setInput call on the specified job context.
static HCatSchema	getTableSchema(JobContext context) Deprecated. Use #getTableSchema(org.apache.hadoop.conf.Configuration)
static void	setOutputSchema(Job job, HCatSchema hcatSchema) Set the schema for the HCatRecord data returned by HCatInputFormat.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HCatBaseInputFormat

public HCatBaseInputFormat()

Method Detail

setOutputSchema

public static void setOutputSchema(Job job,
                                   HCatSchema hcatSchema)
                            throws java.io.IOException

Set the schema for the HCatRecord data returned by HCatInputFormat.

Parameters:

job - the job object

hcatSchema - the schema to use as the consolidated schema

Throws:

java.io.IOException

getMapRedInputFormat

protected static <any> getMapRedInputFormat(JobConf job,
                                            java.lang.Class inputFormatClass)
                                     throws java.io.IOException

Throws:

java.io.IOException

getSplits

public java.util.List<InputSplit> getSplits(JobContext jobContext)
                                     throws java.io.IOException,
                                            java.lang.InterruptedException

Logically split the set of input files for the job. Returns the underlying InputFormat's splits

Parameters:

jobContext - the job context object

Returns:

the splits, an HCatInputSplit wrapper over the storage handler InputSplits

Throws:

java.io.IOException - or InterruptedException

java.lang.InterruptedException

createRecordReader

public <any> createRecordReader(InputSplit split,
                                TaskAttemptContext taskContext)
                         throws java.io.IOException,
                                java.lang.InterruptedException

Create the RecordReader for the given InputSplit. Returns the underlying RecordReader if the required operations are supported and schema matches with HCatTable schema. Returns an HCatRecordReader if operations need to be implemented in HCat.

Parameters:

split - the split

taskContext - the task attempt context

Returns:

the record reader instance, either an HCatRecordReader(later) or the underlying storage handler's RecordReader

Throws:

java.io.IOException - or InterruptedException

java.lang.InterruptedException

getTableSchema

public static HCatSchema getTableSchema(JobContext context)
                                 throws java.io.IOException

Deprecated. Use #getTableSchema(org.apache.hadoop.conf.Configuration)

Throws:

java.io.IOException

See Also:

org.apache.hive.hcatalog.mapreduce.HCatBaseInputFormat#getTableSchema(org.apache.hadoop.conf.Configuration)

getTableSchema

public static HCatSchema getTableSchema(Configuration conf)
                                 throws java.io.IOException

Gets the HCatTable schema for the table specified in the HCatInputFormat.setInput call on the specified job context. This information is available only after HCatInputFormat.setInput has been called for a JobContext.

Parameters:

conf - the Configuration object

Returns:

the table schema

Throws:

java.io.IOException - if HCatInputFormat.setInput has not been called for the current context