For a very long time, MySQL server-side prepared statements were a theoretical performance hit and source of annoyance. This MySQL Performance Blog post from 2006 outlines most of the reasons. The ones to pay attention to concern the query cache and how prepared statements bypass it, extra round-trips for the prepare-params-execute cycle, and a stricter standard for what can and can't be used as a placeholder value.
Starting with MySQL 5.1.17, some but not all prepared statements can now use the query cache. The other minor concerns are still present.
It's worth knowing that there are two controls for how PDO does prepared statements. The other is called PDO::ATTR_EMULATE_PREPARES
. See also: this not-quite-dupe question from earlier this year.