@InterfaceAudience.Private @InterfaceStability.Unstable public class NMClientImpl extends NMClient
This class implements NMClient. All the APIs are blocking.
By default, this client stops all the running containers that are started by
it when it stops. It can be disabled via
cleanupRunningContainersOnStop(boolean), in which case containers will
continue to run even after this client is stopped and till the application
runs at which point ResourceManager will forcefully kill them.
Note that the blocking APIs ensure the RPC calls to NodeManager
are executed immediately, and the responses are received before these APIs
return. However, when startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId)
returns, NodeManager may still need some time to either start
or stop the container because of its asynchronous implementation. Therefore,
getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId) is likely to return a transit container status
if it is executed immediately after startContainer(org.apache.hadoop.yarn.api.records.Container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext) or
stopContainer(org.apache.hadoop.yarn.api.records.ContainerId, org.apache.hadoop.yarn.api.records.NodeId).
| Modifier and Type | Class and Description |
|---|---|
protected static class |
NMClientImpl.StartedContainer |
NMClient.UpgradeOp| Modifier and Type | Field and Description |
|---|---|
protected ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer> |
startedContainers |
| Constructor and Description |
|---|
NMClientImpl() |
NMClientImpl(String name) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
cleanupRunningContainers() |
void |
cleanupRunningContainersOnStop(boolean enabled)
Set whether the containers that are started by this client, and are
still running should be stopped when the client stops.
|
void |
commitLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
Commit last reInitialization of the specified container.
|
AtomicBoolean |
getCleanupRunningContainers() |
org.apache.hadoop.yarn.api.records.ContainerStatus |
getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.NodeId nodeId)
Query the status of a container.
|
org.apache.hadoop.yarn.api.records.NodeId |
getNodeIdOfStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
Get the NodeId of the node on which container is running.
|
void |
increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container)
Deprecated.
|
void |
reInitializeContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContex,
boolean autoCommit)
Re-Initialize the Container.
|
void |
restartContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
Restart the specified container.
|
void |
rollbackLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
Rollback last reInitialization of the specified container.
|
protected void |
serviceInit(org.apache.hadoop.conf.Configuration conf) |
protected void |
serviceStop() |
Map<String,ByteBuffer> |
startContainer(org.apache.hadoop.yarn.api.records.Container container,
org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext)
Start an allocated container.
|
void |
stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.NodeId nodeId)
Stop an started container.
|
void |
updateContainerResource(org.apache.hadoop.yarn.api.records.Container container)
Update the resources of a container.
|
createNMClient, createNMClient, getNMTokenCache, setNMTokenCacheclose, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStopprotected ConcurrentMap<org.apache.hadoop.yarn.api.records.ContainerId,NMClientImpl.StartedContainer> startedContainers
public NMClientImpl()
public NMClientImpl(String name)
protected void serviceStop()
throws Exception
serviceStop in class org.apache.hadoop.service.AbstractServiceExceptionprotected void cleanupRunningContainers()
protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
throws Exception
serviceInit in class org.apache.hadoop.service.AbstractServiceExceptionpublic void cleanupRunningContainersOnStop(boolean enabled)
NMClientSet whether the containers that are started by this client, and are still running should be stopped when the client stops. By default, the feature should be enabled.
However, containers will be stopped only when service is stopped. i.e. afterAbstractService.stop().cleanupRunningContainersOnStop in class NMClientenabled - whether the feature is enabled or notpublic Map<String,ByteBuffer> startContainer(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContext) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
NMClientStart an allocated container.
The ApplicationMaster or other applications that use the
client must provide the details of the allocated container, including the
Id, the assigned node's Id and the token via Container. In
addition, the AM needs to provide the ContainerLaunchContext as
well.
startContainer in class NMClientcontainer - the allocated containercontainerLaunchContext - the context information needed by the
NodeManager to launch the
containerorg.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.@Deprecated public void increaseContainerResource(org.apache.hadoop.yarn.api.records.Container container) throws org.apache.hadoop.yarn.exceptions.YarnException, IOException
NMClientIncrease the resource of a container.
The ApplicationMaster or other applications that use the
client must provide the details of the container, including the Id and
the target resource encapsulated in the updated container token via
Container.
increaseContainerResource in class NMClientcontainer - the container with updated token.org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void updateContainerResource(org.apache.hadoop.yarn.api.records.Container container)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientUpdate the resources of a container.
The ApplicationMaster or other applications that use the
client must provide the details of the container, including the Id and
the target resource encapsulated in the updated container token via
Container.
updateContainerResource in class NMClientcontainer - the container with updated token.org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void stopContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.NodeId nodeId)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientStop an started container.
stopContainer in class NMClientcontainerId - the Id of the started containernodeId - the Id of the NodeManagerorg.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public org.apache.hadoop.yarn.api.records.ContainerStatus getContainerStatus(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.NodeId nodeId)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientQuery the status of a container.
getContainerStatus in class NMClientcontainerId - the Id of the started containernodeId - the Id of the NodeManagerorg.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void reInitializeContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
org.apache.hadoop.yarn.api.records.ContainerLaunchContext containerLaunchContex,
boolean autoCommit)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientRe-Initialize the Container.
reInitializeContainer in class NMClientcontainerId - the Id of the container to Re-Initialize.containerLaunchContex - the updated ContainerLaunchContext.autoCommit - commit re-initialization automatically ?org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void restartContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientRestart the specified container.
restartContainer in class NMClientcontainerId - the Id of the container to restart.org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void rollbackLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientRollback last reInitialization of the specified container.
rollbackLastReInitialization in class NMClientcontainerId - the Id of the container to restart.org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public void commitLastReInitialization(org.apache.hadoop.yarn.api.records.ContainerId containerId)
throws org.apache.hadoop.yarn.exceptions.YarnException,
IOException
NMClientCommit last reInitialization of the specified container.
commitLastReInitialization in class NMClientcontainerId - the Id of the container to commit reInitialize.org.apache.hadoop.yarn.exceptions.YarnException - YarnException.IOException - IOException.public AtomicBoolean getCleanupRunningContainers()
public org.apache.hadoop.yarn.api.records.NodeId getNodeIdOfStartedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
NMClientgetNodeIdOfStartedContainer in class NMClientcontainerId - Container Id of the container.Copyright © 2022 Apache Software Foundation. All rights reserved.