org.apache.hadoop.mapreduce.v2.app.rm

Class RMContainerAllocator

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator

org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor

org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator

All Implemented Interfaces:

Closeable, AutoCloseable, ContainerAllocator, RMHeartbeatHandler, Service, EventHandler<ContainerAllocatorEvent>

public class RMContainerAllocator
extends RMContainerRequestor
implements ContainerAllocator

Allocates the container from the ResourceManager scheduler.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator

ContainerAllocator.EventType

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
static float	DEFAULT_COMPLETED_MAPS_PERCENT_FOR_REDUCE_SLOWSTART

Fields inherited from class org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator

allocatorThread, applicationACLs, applicationId, eventHandler, isSignalled, scheduler

Constructor Summary

Constructors

Constructor and Description
RMContainerAllocator(ClientService clientService, AppContext context)

Method Summary

Methods

Modifier and Type	Method and Description
TaskAttemptEvent	createContainerFinishedEvent(ContainerStatus cont, TaskAttemptId attemptID)
boolean	getIsReduceStarted()
int	getMemLimit()
void	handle(ContainerAllocatorEvent event)
protected void	handleEvent(ContainerAllocatorEvent event)
protected void	heartbeat()
void	rampDownReduces(int rampDown)
void	rampUpReduces(int rampUp)
void	scheduleAllReduces()
void	scheduleReduces(int totalMaps, int completedMaps, int scheduledMaps, int scheduledReduces, int assignedMaps, int assignedReduces, int mapResourceReqt, int reduceResourceReqt, int numPendingReduces, float maxReduceRampupLimit, float reduceSlowStart)
protected void	serviceInit(Configuration conf) All initialization code needed by a service.
protected void	serviceStart() Actions called during the INITED to STARTED transition.
protected void	serviceStop() Actions called during the transition to the STOPPED state.
void	setIsReduceStarted(boolean reduceStarted)

Methods inherited from class org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor

addContainerReq, computeIgnoreBlacklisting, containerFailedOnHost, decContainerReq, getAvailableResources, getBlacklistedNodes, getFilteredContainerRequest, isNodeBlacklisted, makeRemoteRequest, release

Methods inherited from class org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator

createSchedulerProxy, doUnregistration, getApplicationProgress, getContext, getJob, getLastHeartbeatTime, getMaxContainerCapability, register, runOnNextHeartbeat, setShouldUnregister, setSignalled, startAllocatorThread, unregister

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_COMPLETED_MAPS_PERCENT_FOR_REDUCE_SLOWSTART

public static final float DEFAULT_COMPLETED_MAPS_PERCENT_FOR_REDUCE_SLOWSTART

See Also:

Constant Field Values

Constructor Detail

RMContainerAllocator

public RMContainerAllocator(ClientService clientService,
                    AppContext context)

Method Detail

serviceInit

protected void serviceInit(Configuration conf)
                    throws Exception

Description copied from class: AbstractService

All initialization code needed by a service. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.init(Configuration) prevents re-entrancy. The base implementation checks to see if the subclass has created a new configuration instance, and if so, updates the base class value

Overrides:

serviceInit in class RMContainerRequestor

Parameters:

conf - configuration

Throws:

Exception - on a failure -these will be caught, possibly wrapped, and wil; trigger a service stop

serviceStart

protected void serviceStart()
                     throws Exception

Description copied from class: AbstractService

Actions called during the INITED to STARTED transition. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.start() prevents re-entrancy.

Overrides:

serviceStart in class RMCommunicator

Throws:

Exception - if needed -these will be caught, wrapped, and trigger a service stop

heartbeat

protected void heartbeat()
                  throws Exception

Specified by:

heartbeat in class RMCommunicator

Throws:

Exception

serviceStop

protected void serviceStop()
                    throws Exception

Description copied from class: AbstractService

Actions called during the transition to the STOPPED state. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.stop() prevents re-entrancy. Implementations MUST write this to be robust against failures, including checks for null references -and for the first failure to not stop other attempts to shut down parts of the service.

Overrides:

serviceStop in class RMCommunicator

Throws:

Exception - if needed -these will be caught and logged.

getIsReduceStarted

public boolean getIsReduceStarted()

setIsReduceStarted

public void setIsReduceStarted(boolean reduceStarted)

handle

public void handle(ContainerAllocatorEvent event)

Specified by:

handle in interface EventHandler<ContainerAllocatorEvent>

handleEvent

protected void handleEvent(ContainerAllocatorEvent event)

scheduleReduces

@InterfaceAudience.Private
public void scheduleReduces(int totalMaps,
                                             int completedMaps,
                                             int scheduledMaps,
                                             int scheduledReduces,
                                             int assignedMaps,
                                             int assignedReduces,
                                             int mapResourceReqt,
                                             int reduceResourceReqt,
                                             int numPendingReduces,
                                             float maxReduceRampupLimit,
                                             float reduceSlowStart)

scheduleAllReduces

@InterfaceAudience.Private
public void scheduleAllReduces()

rampUpReduces

@InterfaceAudience.Private
public void rampUpReduces(int rampUp)

rampDownReduces

@InterfaceAudience.Private
public void rampDownReduces(int rampDown)

createContainerFinishedEvent

public TaskAttemptEvent createContainerFinishedEvent(ContainerStatus cont,
                                            TaskAttemptId attemptID)

getMemLimit

@InterfaceAudience.Private
public int getMemLimit()