Which is the preferred approach for transaction management in stateless bean - container >managed or bean managed?
The typical and preferred approach is to use CMT. Transaction management is one of the useful services that a app server offers, it simplifies your development, therefore, you should use this approach (that also is the default one) the vast majority of the time.
However, BMT is still necessary in some special cases:
a) when you need reduce the transaction boundaries for improving performance.
b) when you have a statefull session bean and you need retain a transaction across several client calls. (it's hard to see when this can be useful).
Is it advisable to have both bean managed and container managed transaction beans in service layer?
Yes, If some services need the special requirement described above, you can use both bean transaction types as part of the service layer.
Is it possible to use both container managed and bean managed transactions in a single bean? If possible is it advisable?
No, it is not possible.