Yes, you are wrong in believing that POI will do that. Apache POI works with Microsoft Office file formats, which PDF isn't.
You'll either want to use Apache PDFBox directly, or us Apache Tika which will do both Microsoft Office and PDF file formats (amongst many others).