Question

I wrote a mapper like this:

@Override
protected void map(VarLongWritable userId, RecommendedItemsWritable recommendations, Context ctx) throws IOException,
        InterruptedException {

    List<RecommendedItem> items = recommendations.getRecommendedItems();
    for (RecommendedItem item : items) {
        long userID = Long.valueOf(userId.toString().trim());
        System.out.println(userID);
        ctx.write(new LongWritable(userID), new Text(item.getItemID() + "," + item.getValue()));
    }
}

and it generated the following content:

1   ,105,3.8652005
1   ,106,3.7047918
1   ,104,3.6800702
2   ,106,2.7261393
2   ,105,2.4083052
2   ,107,2.0
3   ,106,3.5539715
3   ,102,3.4113002
3   ,103,3.323024
4   ,107,4.674651
4   ,105,4.371781
4   ,102,4.0743575
5   ,107,3.754705

actually I just want to load the generated file into hive table. but with the tab space, the value in the hive table is wrong.

So is there any way to remove the tab space in the file?

Was it helpful?

Solution

I don't think there is a way to remove the tab space between the key and the value. However, if you do not need a reducer, then, just put all of the info in the key of the mapper's output:

ctx.write(new Text(userID + "," + item.getItemID() + "," + item.getValue()), new Text(""));

Changing the type of map output key to Text.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top