org.apache.hadoop.hive.ql.exec

Class ReduceSinkOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<T>

org.apache.hadoop.hive.ql.exec.TerminalOperator<ReduceSinkDesc>

org.apache.hadoop.hive.ql.exec.ReduceSinkOperator

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, TopNHash.BinaryCollector, Node

public class ReduceSinkOperator
extends TerminalOperator<ReduceSinkDesc>
implements java.io.Serializable, TopNHash.BinaryCollector

Reduce Sink Operator sends output to the reduce stage.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.OperatorFunc, Operator.ProgressCounter, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected ExprNodeEvaluator[]	keyEval The evaluators for the key columns.
protected int	numDistinctExprs
protected int	numDistributionKeys
protected ExprNodeEvaluator[]	partitionEval The evaluators for the partition columns (CLUSTER BY or DISTRIBUTE BY in Hive language).
protected ExprNodeEvaluator[]	valueEval The evaluators for the value columns.

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

alias, beginTime, childOperators, childOperatorsArray, childOperatorsTag, colExprMap, conf, counterNames, counterNameToEnum, counters, done, fatalErrorCntr, groupKeyObject, id, inputObjInspectors, inputRows, isLogInfoEnabled, LOG, numInputRowsCntr, numOutputRowsCntr, operatorId, out, outputObjInspector, outputRows, parentOperators, reporter, state, statsMap, timeTakenCntr, totalTime

Constructor Summary

Constructors

Constructor and Description
ReduceSinkOperator()

Method Summary

Modifier and Type	Method and Description
protected void	closeOp(boolean abort) Operator specific close routine.
void	collect(BytesWritable key, BytesWritable value)
java.lang.String	getInputAlias()
java.lang.String	getName() Implements the getName function for the Node Interface.
static java.lang.String	getOperatorName()
OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
protected static StructObjectInspector	initEvaluatorsAndReturnStruct(ExprNodeEvaluator[] evals, java.util.List<java.util.List<java.lang.Integer>> distinctColIndices, java.util.List<java.lang.String> outputColNames, int length, ObjectInspector rowInspector) Initializes array of ExprNodeEvaluator.
protected void	initializeOp(Configuration hconf) Operator specific initialization.
boolean	opAllowedBeforeMapJoin()
void	processOp(java.lang.Object row, int tag) Process the row.
void	setInputAlias(java.lang.String inputAlias)

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, assignCounterNameToEnum, augmentPlan, checkFatalErrors, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, close, columnNamesRowResolvedCanBeObtained, dump, dump, endGroup, fatalErrorMessage, flush, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getConf, getConfiguration, getCounterNames, getCounterNameToEnum, getCounters, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getNextCntr, getNumChild, getNumParent, getOperatorId, getParentOperators, getSchema, getStats, getWrappedCounterName, incrCounter, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeCounters, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logStats, opAllowedAfterMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, process, processGroup, removeChild, removeChildAndAdoptItsChildren, removeChildren, removeParent, replaceChild, replaceParent, reset, resetId, resetLastEnumUsed, resetStats, setAlias, setChildOperators, setColumnExprMap, setConf, setCounterNames, setCounterNameToEnum, setDone, setExecContext, setGroupKeyObject, setId, setInputObjInspectors, setOperatorId, setOutputCollector, setParentOperators, setReporter, setSchema, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString, updateCounters

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

keyEval

protected transient ExprNodeEvaluator[] keyEval

The evaluators for the key columns. Key columns decide the sort order on the reducer side. Key columns are passed to the reducer in the "key".

valueEval

protected transient ExprNodeEvaluator[] valueEval

The evaluators for the value columns. Value columns are passed to reducer in the "value".

partitionEval

protected transient ExprNodeEvaluator[] partitionEval

The evaluators for the partition columns (CLUSTER BY or DISTRIBUTE BY in Hive language). Partition columns decide the reducer that the current row goes to. Partition columns are not passed to reducer.

numDistributionKeys

protected transient int numDistributionKeys

numDistinctExprs

protected transient int numDistinctExprs

Constructor Detail

ReduceSinkOperator

public ReduceSinkOperator()

Method Detail

setInputAlias

public void setInputAlias(java.lang.String inputAlias)

getInputAlias

public java.lang.String getInputAlias()

initializeOp

protected void initializeOp(Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<ReduceSinkDesc>

Throws:

HiveException

initEvaluatorsAndReturnStruct

protected static StructObjectInspector initEvaluatorsAndReturnStruct(ExprNodeEvaluator[] evals,
                                                                     java.util.List<java.util.List<java.lang.Integer>> distinctColIndices,
                                                                     java.util.List<java.lang.String> outputColNames,
                                                                     int length,
                                                                     ObjectInspector rowInspector)
                                                              throws HiveException

Initializes array of ExprNodeEvaluator. Adds Union field for distinct column indices for group by. Puts the return values into a StructObjectInspector with output column names. If distinctColIndices is empty, the object inspector is same as Operator.initEvaluatorsAndReturnStruct(ExprNodeEvaluator[], List, ObjectInspector)

Throws:

HiveException

processOp

public void processOp(java.lang.Object row,
                      int tag)
               throws HiveException

Description copied from class: Operator

Process the row.

Specified by:

processOp in class Operator<ReduceSinkDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException

collect

public void collect(BytesWritable key,
                    BytesWritable value)
             throws java.io.IOException

Throws:

java.io.IOException

closeOp

protected void closeOp(boolean abort)
                throws HiveException

Description copied from class: Operator

Operator specific close routine. Operators which inherents this class should overwrite this funtion for their specific cleanup routine.

Overrides:

closeOp in class Operator<ReduceSinkDesc>

Throws:

HiveException

getName

public java.lang.String getName()

Description copied from class: Operator

Implements the getName function for the Node Interface.

Specified by:

getName in interface Node

Overrides:

getName in class Operator<ReduceSinkDesc>

Returns:

the name of the operator

getOperatorName

public static java.lang.String getOperatorName()

getType

public OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<ReduceSinkDesc>

Returns:

OperatorType.*

opAllowedBeforeMapJoin

public boolean opAllowedBeforeMapJoin()

Overrides:

opAllowedBeforeMapJoin in class Operator<ReduceSinkDesc>