Efficient way for String to ID mapping for myrrix

Question

Myrrix takes in only long userId, itemId for processing which forces in generating an initial mapper for files having such items as string. For example file like :

(alice,refrigerator) (bob,refrigerator)

needs to change into:

<1,1> <2,1>.

This mapping takes lot of time if the files are big. Also creates problem if any more data points need to be added later. Any suggestion on making this process faster?

Answer 1

Yeah, really you want to generate the mapping one time and hold on to it for a long time. Obviously you have to re-create it each time you run the command-line tools, which is not good. For a long-running Java process which embeds the client, you can hold on to the client object with its mapping as long as you like.