Answered this on the Thinking Sphinx Google group too:
What you’re trying to do is not possible (while searching on Users, at least). Sphinx has no concept of hashes/dictionaries… in your index, each user has a field called answer, which contains all the answers joined together in a single string, and then an attribute that is an array of question ids. There’s no relationship between the question ids and each answer.
A better approach would be to define an index on ProfileAnswer, and search via that instead - and then display the user for each ProfileAnswer in the search results.