You'll want to use an external tool for this. PostgreSQL knows nothing about the XLSX format, nor about OpenDocument or any of that.
I suggest writing a script that exports a bunch of individual CSV files with copy
. Then using an external tool to convert them to xlsx and assemble them into sheets in the document.
It's possible that ETL tools like CloverETL, Pentaho Kettle, or Talend Studio may do what you want too. I haven't checked this specific functionality.