Pergunta

I wrote a mapper like this:

@Override
protected void map(VarLongWritable userId, RecommendedItemsWritable recommendations, Context ctx) throws IOException,
        InterruptedException {

    List<RecommendedItem> items = recommendations.getRecommendedItems();
    for (RecommendedItem item : items) {
        long userID = Long.valueOf(userId.toString().trim());
        System.out.println(userID);
        ctx.write(new LongWritable(userID), new Text(item.getItemID() + "," + item.getValue()));
    }
}

and it generated the following content:

1   ,105,3.8652005
1   ,106,3.7047918
1   ,104,3.6800702
2   ,106,2.7261393
2   ,105,2.4083052
2   ,107,2.0
3   ,106,3.5539715
3   ,102,3.4113002
3   ,103,3.323024
4   ,107,4.674651
4   ,105,4.371781
4   ,102,4.0743575
5   ,107,3.754705

actually I just want to load the generated file into hive table. but with the tab space, the value in the hive table is wrong.

So is there any way to remove the tab space in the file?

Foi útil?

Solução

I don't think there is a way to remove the tab space between the key and the value. However, if you do not need a reducer, then, just put all of the info in the key of the mapper's output:

ctx.write(new Text(userID + "," + item.getItemID() + "," + item.getValue()), new Text(""));

Changing the type of map output key to Text.

Licenciado em: CC-BY-SA com atribuição
Não afiliado a StackOverflow
scroll top