What I was looking for was Database Resident Connection Pooling in oracle. It is supported after 11g.
http://www.oracle.com/technetwork/topics/php/whatsnew/php-scalability-ha-twp-128842.pdf
Database Resident Connection Pooling (DRCP) provides a connection pool in the database server for typical Web application usage scenarios where the application acquires a database connection, works on it for a relatively short duration, and then releases it.