In YARN what is the difference between a managed and an unmanaged Application Manager

Question

I'm experimenting with the Distributed Shell example in YARN 2.2 and am hoping that someone can clarify what the difference between a managed and and an un-managed application manager is?

For example the following lines appear in the client code

// unmanaged AM
appContext.setUnmanagedAM(true);

but I am unable to find documentation explaining the difference this line makes to the execution behaviour.

Many thanks.

Answer 1

The setUnmanagedAM(true) is used for debugging purposes i.e. it runs an application manager in local mode and does not submit it to a cluster so it is easier to step into code and debug. You can see it in use in the hadoop-yarn-applications-unmanaged-am-launcher.jar that ships with yarn

Answer 2

Check the respective JIRA tickets: JIRA-420 and JIRA-419 (client side)

Currently, the RM itself manages the AM by allocating a container for it and negotiating the launch on the NodeManager and manages the AM lifecycle. Thereafter, the AM negotiates resources with the RM and launches tasks to do the real work.
It would be a useful improvement to enhance this model by allowing the AM to be launched independently by the client without requiring the RM. These AM's would be launched on a gateway machine that can talk to the cluster. This would open up new use cases such as the following
1) Easy debugging of AM, specially during initial development. Having the AM launched on an arbitrary cluster node makes it hard to looks at logs or attach a debugger to the AM. If it can be launched locally then these tasks would be easier.
2) Running AM's that need special privileges that may not be available on machines managed by the NodeManager

Blog post with more implementation details on unmanaged AM: click-me

Example of how Impala manages its resources with the help of unmanaged applications: Llama