Thanks for the people who helped me with this issue. As I mentioned in the question, this issue does relate to our live server configuration. The PHP code:
session_start();
After we removed this line (we are using CodeIgniter's session library, so PHP session_start function is a redundant line in our code....), the application's performance was getting much better.
I'm new in PHP coding, and this one teaches me that if something slows the application performance, always look into the code you wrote first.
Thanks guys.