I created empty placeholders in Layouts\ClientBin
for all the xaps to be copied.
This wasn't optimal since it required some manual dependency management, but it works.
The end result is that the package manager is aware of those files existing when you open the project in VS and that they need to be packaged up into the WSP. The copy operation works as before but now the new files are packaged.