I'm the author and maintainer of pg_search.
Unfortunately, it's not possible to traverse a polymorphic association in this direction in pure SQL, so doing that search is not possible with pg_search.
One thing you could do is calculate the text from the other records and cache it to a column on the Cloth table, then search against that instead. You'd have to be careful to update it whenever either the polymorphic foreign key changes on Cloth or the content changes on the Owner record.
Hopefully I can improve the error message so that it's not so confusing. Thanks for pointing this out.