What you get from the lookup operation (through the JNDI service) is an object called Stub and him is not fixed with any special EJB instance.
Once cached, every time you invoke an EJB service the stub is able to select a different EJB instance from the pool (this apply to stateless); even in a cluster environment, the stub object is able to select an EJB instance from different servers.
Therefore, caching the stub object should not be a problem.
Notice that I'm talking about stateless, I think that caching doesn't make sense to statefull session bean.
The EJB lookup is a time-consuming operation, so caching improves the client performance.
About your comment:
If you were using EntityManager inside a component that is used by multiple concurrent threads like a Servlet, yes, you will have to care about concurrency, but the EJB's Tread Model imply that will not be several thread using the same EntityManager instance at the same time, so the fact that EM is not thread safe doesn't matter.
What concerns me still is the fact that different EJBs are using the same injected (via @PersistenceContext) EntityManager instance
I think that for simple sceneries the best way to think about it, if as describe here:
The most common pattern in a multi-user client/server application is entitymanager-per-request. In this model, a request from the client is send to the server (where the JPA persistence layer runs), a new EntityManager is opened, and all database operations are executed in this unit of work. Once the work has been completed (and the response for the client has been prepared), the persistence context is flushed and closed, as well as the entity manager object.
This will be more difficult to check ;)