org.apache.hadoop.yarn.client.api.impl

Class NMClientImpl

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.client.api.NMClient

org.apache.hadoop.yarn.client.api.impl.NMClientImpl

All Implemented Interfaces:

Closeable, AutoCloseable, Service

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class NMClientImpl
extends NMClient

This class implements NMClient. All the APIs are blocking.

By default, this client stops all the running containers that are started by it when it stops. It can be disabled via cleanupRunningContainersOnStop(boolean), in which case containers will continue to run even after this client is stopped and till the application runs at which point ResourceManager will forcefully kill them.

Note that the blocking APIs ensure the RPC calls to NodeManager are executed immediately, and the responses are received before these APIs return. However, when startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) returns, NodeManager may still need some time to either start or stop the container because of its asynchronous implementation. Therefore, getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) is likely to return a transit container status if it is executed immediately after startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	NMClientImpl.StartedContainer

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
protected ConcurrentMap<ContainerId,NMClientImpl.StartedContainer>	startedContainers

Constructor Summary

Constructors

Constructor and Description
NMClientImpl()
NMClientImpl(String name)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	cleanupRunningContainers()
void	cleanupRunningContainersOnStop(boolean enabled) Set whether the containers that are started by this client, and are still running should be stopped when the client stops.
protected NMClientImpl.StartedContainer	createStartedContainer(Container container)
AtomicBoolean	getCleanupRunningContainers()
ContainerStatus	getContainerStatus(ContainerId containerId, NodeId nodeId) Query the status of a container.
protected NMClientImpl.StartedContainer	getStartedContainer(ContainerId containerId)
protected void	removeStartedContainer(NMClientImpl.StartedContainer container)
protected void	serviceInit(Configuration conf) All initialization code needed by a service.
protected void	serviceStop() Actions called during the transition to the STOPPED state.
Map<String,ByteBuffer>	startContainer(Container container, ContainerLaunchContext containerLaunchContext) Start an allocated container.
void	stopContainer(ContainerId containerId, NodeId nodeId) Stop an started container.

Methods inherited from class org.apache.hadoop.yarn.client.api.NMClient

createNMClient, createNMClient

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

startedContainers

protected ConcurrentMap<ContainerId,NMClientImpl.StartedContainer> startedContainers

Constructor Detail

NMClientImpl

public NMClientImpl()

NMClientImpl

public NMClientImpl(String name)

Method Detail

serviceStop

protected void serviceStop()
                    throws Exception

Description copied from class: AbstractService

Actions called during the transition to the STOPPED state. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.stop() prevents re-entrancy. Implementations MUST write this to be robust against failures, including checks for null references -and for the first failure to not stop other attempts to shut down parts of the service.

Overrides:

serviceStop in class AbstractService

Throws:

Exception - if needed -these will be caught and logged.

cleanupRunningContainers

protected void cleanupRunningContainers()

serviceInit

protected void serviceInit(Configuration conf)
                    throws Exception

Description copied from class: AbstractService

All initialization code needed by a service. This method will only ever be called once during the lifecycle of a specific service instance. Implementations do not need to be synchronized as the logic in AbstractService.init(Configuration) prevents re-entrancy. The base implementation checks to see if the subclass has created a new configuration instance, and if so, updates the base class value

Overrides:

serviceInit in class AbstractService

Parameters:

conf - configuration

Throws:

Exception - on a failure -these will be caught, possibly wrapped, and wil; trigger a service stop

cleanupRunningContainersOnStop

public void cleanupRunningContainersOnStop(boolean enabled)

Description copied from class: NMClient

Set whether the containers that are started by this client, and are still running should be stopped when the client stops. By default, the feature should be enabled.

However, containers will be stopped only when service is stopped. i.e. after AbstractService.stop().

Specified by:

cleanupRunningContainersOnStop in class NMClient

Parameters:

enabled - whether the feature is enabled or not

startContainer

public Map<String,ByteBuffer> startContainer(Container container,
                                    ContainerLaunchContext containerLaunchContext)
                                      throws YarnException,
                                             IOException

Description copied from class: NMClient

Start an allocated container.

The ApplicationMaster or other applications that use the client must provide the details of the allocated container, including the Id, the assigned node's Id and the token via Container. In addition, the AM needs to provide the ContainerLaunchContext as well.

Specified by:

startContainer in class NMClient

Parameters:

container - the allocated container

containerLaunchContext - the context information needed by the NodeManager to launch the container

Returns:

a map between the auxiliary service names and their outputs

Throws:

YarnException

IOException

stopContainer

public void stopContainer(ContainerId containerId,
                 NodeId nodeId)
                   throws YarnException,
                          IOException

Description copied from class: NMClient

Stop an started container.

Specified by:

stopContainer in class NMClient

Parameters:

containerId - the Id of the started container

nodeId - the Id of the NodeManager

Throws:

YarnException

IOException

getContainerStatus

public ContainerStatus getContainerStatus(ContainerId containerId,
                                 NodeId nodeId)
                                   throws YarnException,
                                          IOException

Description copied from class: NMClient

Query the status of a container.

Specified by:

getContainerStatus in class NMClient

Parameters:

containerId - the Id of the started container

nodeId - the Id of the NodeManager

Returns:

the status of a container

Throws:

YarnException

IOException

createStartedContainer

protected NMClientImpl.StartedContainer createStartedContainer(Container container)
                                                        throws YarnException,
                                                               IOException

Throws:

YarnException

IOException

removeStartedContainer

protected void removeStartedContainer(NMClientImpl.StartedContainer container)

getStartedContainer

protected NMClientImpl.StartedContainer getStartedContainer(ContainerId containerId)

getCleanupRunningContainers

public AtomicBoolean getCleanupRunningContainers()