you have to use the AvroSerDe described in
or you have to transform your output to the RowFormat that you are using in your defined hive table (again using another mapreduce job)
Regards
Martin
题
My mapreduce writes an avro file with the AvroKeyValueOutputFormat but I'm having some troubles to import this file into hive.
How I have to define my schema in hive to get it working?
解决方案
you have to use the AvroSerDe described in
or you have to transform your output to the RowFormat that you are using in your defined hive table (again using another mapreduce job)
Regards
Martin
其他提示
The format of the AvroSerDe can be a little tricky. As long as you know the Avro schema though it tends to work wonders. Hopefully this example helps.
CREATE EXTERNAL TABLE HIVEDATA
ROW FORMAT
SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
WITH SERDEPROPERTIES ('avro.schema.literal'='
{
"namespace": "originalname",
"name": "feature_value",
"type": "record",
"fields": [
{"name": "acct_id", "type": "long"},
{"name": "feature_name", "type": ["null","string"], "default": null},
{"name": "namespace", "type": ["null","string"], "default": null},
{"name": "feature_value", "type": ["null","double"], "default": null}
]
}
')
STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION '/hdfs/location'
;