org.apache.hadoop.hive.ql.io.rcfile.stats

Class PartialScanTask

java.lang.Object

org.apache.hadoop.hive.ql.exec.Task<PartialScanWork>

org.apache.hadoop.hive.ql.io.rcfile.stats.PartialScanTask

All Implemented Interfaces:

java.io.Serializable, HadoopJobExecHook, Node

public class PartialScanTask
extends Task<PartialScanWork>
implements java.io.Serializable, HadoopJobExecHook

PartialScanTask. This task handles RCFile statics calculation via partial scan. Instead of reading all bytes, it reads block header and aggregates result. https://issues.apache.org/jira/browse/HIVE-3958

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Task

Task.FeedType

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	INPUT_SEPERATOR
protected JobConf	job
protected HadoopJobExecHelper	jobExecHelper

Fields inherited from class org.apache.hadoop.hive.ql.exec.Task

BACKUP_COMMON_JOIN, backupChildrenTasks, backupTask, childTasks, clonedConf, COMMON_JOIN, conf, console, CONVERTED_LOCAL_MAPJOIN, CONVERTED_MAPJOIN, CONVERTED_SORTMERGEJOIN, db, driverContext, feedSubscribers, id, initialized, isdone, jobID, LOCAL_MAPJOIN, LOG, MAPJOIN_ONLY_NOBACKUP, NO_TAG, parentTasks, queryPlan, queued, started, taskCounters, taskHandle, taskTag, work

Constructor Summary

Constructors

Constructor and Description
PartialScanTask()

Method Summary

Modifier and Type	Method and Description
boolean	checkFatalErrors(Counters ctrs, java.lang.StringBuilder errMsg)
int	execute(DriverContext driverContext) This method is overridden in each Task.
java.lang.String	getName() Gets the name of the node.
StageType	getType() Should be overridden to return the type of the specific task among the types in StageType.
void	initialize(HiveConf conf, QueryPlan queryPlan, DriverContext driverContext)
void	logPlanProgress(SessionState ss)
static void	main(java.lang.String[] args)
boolean	requireLock()
void	updateCounters(Counters ctrs, RunningJob rj)

Methods inherited from class org.apache.hadoop.hive.ql.exec.Task

addDependentTask, cloneConf, done, executeTask, fetch, getAndInitBackupTask, getBackupChildrenTasks, getBackupTask, getChildren, getChildTasks, getCounters, getDependentTasks, getFeedSubscribers, getId, getInitialized, getJobID, getParentTasks, getQueryPlan, getQueued, getReducer, getResultSchema, getTaskHandle, getTaskTag, getTopOperators, getWork, hasReduce, ifRetryCmdWhenFail, isLocalMode, isMapRedLocalTask, isMapRedTask, isRunnable, pushFeed, receiveFeed, removeDependentTask, removeFromChildrenTasks, setBackupChildrenTasks, setBackupTask, setChildTasks, setDone, setFeedSubscribers, setId, setInitialized, setLocalMode, setParentTasks, setQueryPlan, setQueued, setRetryCmdWhenFail, setStarted, setTaskTag, setWork, shutdown, started, subscribeFeed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

job

protected transient JobConf job

jobExecHelper

protected HadoopJobExecHelper jobExecHelper

INPUT_SEPERATOR

public static java.lang.String INPUT_SEPERATOR

Constructor Detail

PartialScanTask

public PartialScanTask()

Method Detail

initialize

public void initialize(HiveConf conf,
                       QueryPlan queryPlan,
                       DriverContext driverContext)

Overrides:

initialize in class Task<PartialScanWork>

requireLock

public boolean requireLock()

Overrides:

requireLock in class Task<PartialScanWork>

execute

public int execute(DriverContext driverContext)

Description copied from class: Task

This method is overridden in each Task. TODO execute should return a TaskHandle.

Specified by:

execute in class Task<PartialScanWork>

Returns:

status of executing the task

getName

public java.lang.String getName()

Description copied from interface: Node

Gets the name of the node. This is used in the rule dispatchers.

Specified by:

getName in interface Node

Returns:

String

main

public static void main(java.lang.String[] args)

getType

public StageType getType()

Description copied from class: Task

Should be overridden to return the type of the specific task among the types in StageType.

Specified by:

getType in class Task<PartialScanWork>

Returns:

StageType.* or null if not overridden

checkFatalErrors

public boolean checkFatalErrors(Counters ctrs,
                                java.lang.StringBuilder errMsg)

Specified by:

checkFatalErrors in interface HadoopJobExecHook

logPlanProgress

public void logPlanProgress(SessionState ss)
                     throws java.io.IOException

Specified by:

logPlanProgress in interface HadoopJobExecHook

Throws:

java.io.IOException

updateCounters

public void updateCounters(Counters ctrs,
                           RunningJob rj)
                    throws java.io.IOException

Specified by:

updateCounters in interface HadoopJobExecHook

Throws:

java.io.IOException