"In PDF/A-3, the data is added as a document-level attachment. That makes much more sense than to put it into an XMP stream.
The document-level attachment won't be visible on any page, but people will be able to select it in the attachment panel, just like they'll be able to see the contents of the XMP (it's easy to add a document-level attachment with iText). There are of course many other ways to add data to a PDF that isn't visible. For instance Adobe Illustrator adds proprietary artifacts as a /PieceInfo entry in the root dictionary of the PDF. That's also possible with iText. There are many solutions, all are better than abusing the XMP stream"
Attaching at the document level solved the problem.