org.apache.hadoop.hive.ql.exec.mr

Class MapRedTask

java.lang.Object

org.apache.hadoop.hive.ql.exec.Task<MapredWork>

org.apache.hadoop.hive.ql.exec.mr.ExecDriver

org.apache.hadoop.hive.ql.exec.mr.MapRedTask

All Implemented Interfaces:

java.io.Serializable, HadoopJobExecHook, Node

public class MapRedTask
extends ExecDriver
implements java.io.Serializable

Extension of ExecDriver: - can optionally spawn a map-reduce task from a separate jvm - will make last minute adjustments to map-reduce job parameters, viz: * estimating number of reducers * estimating whether job should run locally

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Task

Task.FeedType

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.exec.mr.ExecDriver

job, jobExecHelper, LOG, memoryMXBean

Fields inherited from class org.apache.hadoop.hive.ql.exec.Task

BACKUP_COMMON_JOIN, backupChildrenTasks, backupTask, childTasks, clonedConf, COMMON_JOIN, conf, console, CONVERTED_LOCAL_MAPJOIN, CONVERTED_MAPJOIN, CONVERTED_SORTMERGEJOIN, db, driverContext, feedSubscribers, id, initialized, isdone, jobID, LOCAL_MAPJOIN, MAPJOIN_ONLY_NOBACKUP, NO_TAG, parentTasks, queryPlan, queued, started, taskCounters, taskHandle, taskTag, work

Constructor Summary

Constructors

Constructor and Description
MapRedTask()
MapRedTask(MapredWork plan, JobConf job, boolean isSilent)

Method Summary

Modifier and Type	Method and Description
int	execute(DriverContext driverContext) Execute a query plan using Hadoop.
Operator<? extends OperatorDesc>	getReducer()
static java.lang.String	isEligibleForLocalMode(HiveConf conf, int numReducers, long inputLength, long inputFileCount) Find out if a job can be run in local mode based on it's characteristics
boolean	mapDone()
boolean	mapStarted()
boolean	reduceDone()
boolean	reduceStarted()
void	shutdown()

Methods inherited from class org.apache.hadoop.hive.ql.exec.mr.ExecDriver

checkFatalErrors, generateCmdLine, getName, getTopOperators, getType, hasReduce, initialize, isMapRedTask, logPlanProgress, main, requireLock, setInputAttributes, updateCounters

Methods inherited from class org.apache.hadoop.hive.ql.exec.Task

addDependentTask, cloneConf, done, executeTask, fetch, getAndInitBackupTask, getBackupChildrenTasks, getBackupTask, getChildren, getChildTasks, getCounters, getDependentTasks, getFeedSubscribers, getId, getInitialized, getJobID, getParentTasks, getQueryPlan, getQueued, getResultSchema, getTaskHandle, getTaskTag, getWork, ifRetryCmdWhenFail, isLocalMode, isMapRedLocalTask, isRunnable, pushFeed, receiveFeed, removeDependentTask, removeFromChildrenTasks, setBackupChildrenTasks, setBackupTask, setChildTasks, setDone, setFeedSubscribers, setId, setInitialized, setLocalMode, setParentTasks, setQueryPlan, setQueued, setRetryCmdWhenFail, setStarted, setTaskTag, setWork, started, subscribeFeed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapRedTask

public MapRedTask()

MapRedTask

public MapRedTask(MapredWork plan,
                  JobConf job,
                  boolean isSilent)
           throws HiveException

Throws:

HiveException

Method Detail

execute

public int execute(DriverContext driverContext)

Description copied from class: ExecDriver

Execute a query plan using Hadoop.

Overrides:

execute in class ExecDriver

Returns:

status of executing the task

mapStarted

public boolean mapStarted()

Overrides:

mapStarted in class ExecDriver

reduceStarted

public boolean reduceStarted()

Overrides:

reduceStarted in class ExecDriver

mapDone

public boolean mapDone()

Overrides:

mapDone in class ExecDriver

reduceDone

public boolean reduceDone()

Overrides:

reduceDone in class ExecDriver

isEligibleForLocalMode

public static java.lang.String isEligibleForLocalMode(HiveConf conf,
                                                      int numReducers,
                                                      long inputLength,
                                                      long inputFileCount)

Find out if a job can be run in local mode based on it's characteristics

Parameters:

conf - Hive Configuration

numReducers - total number of reducers for this job

inputLength - the size of the input

inputFileCount - the number of files of input

Returns:

String null if job is eligible for local mode, reason otherwise

getReducer

public Operator<? extends OperatorDesc> getReducer()

Overrides:

getReducer in class Task<MapredWork>

shutdown

public void shutdown()

Overrides:

shutdown in class ExecDriver