I wrote a mapper like this:

@Override
protected void map(VarLongWritable userId, RecommendedItemsWritable recommendations, Context ctx) throws IOException,
        InterruptedException {

    List<RecommendedItem> items = recommendations.getRecommendedItems();
    for (RecommendedItem item : items) {
        long userID = Long.valueOf(userId.toString().trim());
        System.out.println(userID);
        ctx.write(new LongWritable(userID), new Text(item.getItemID() + "," + item.getValue()));
    }
}

and it generated the following content:

1   ,105,3.8652005
1   ,106,3.7047918
1   ,104,3.6800702
2   ,106,2.7261393
2   ,105,2.4083052
2   ,107,2.0
3   ,106,3.5539715
3   ,102,3.4113002
3   ,103,3.323024
4   ,107,4.674651
4   ,105,4.371781
4   ,102,4.0743575
5   ,107,3.754705

actually I just want to load the generated file into hive table. but with the tab space, the value in the hive table is wrong.

So is there any way to remove the tab space in the file?

有帮助吗?

解决方案

I don't think there is a way to remove the tab space between the key and the value. However, if you do not need a reducer, then, just put all of the info in the key of the mapper's output:

ctx.write(new Text(userID + "," + item.getItemID() + "," + item.getValue()), new Text(""));

Changing the type of map output key to Text.

许可以下: CC-BY-SA归因
不隶属于 StackOverflow
scroll top