you have to use the AvroSerDe described in
or you have to transform your output to the RowFormat that you are using in your defined hive table (again using another mapreduce job)
Regards
Martin
Domanda
My mapreduce writes an avro file with the AvroKeyValueOutputFormat but I'm having some troubles to import this file into hive.
How I have to define my schema in hive to get it working?
Soluzione
you have to use the AvroSerDe described in
or you have to transform your output to the RowFormat that you are using in your defined hive table (again using another mapreduce job)
Regards
Martin
Altri suggerimenti
The format of the AvroSerDe can be a little tricky. As long as you know the Avro schema though it tends to work wonders. Hopefully this example helps.
CREATE EXTERNAL TABLE HIVEDATA
ROW FORMAT
SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
WITH SERDEPROPERTIES ('avro.schema.literal'='
{
"namespace": "originalname",
"name": "feature_value",
"type": "record",
"fields": [
{"name": "acct_id", "type": "long"},
{"name": "feature_name", "type": ["null","string"], "default": null},
{"name": "namespace", "type": ["null","string"], "default": null},
{"name": "feature_value", "type": ["null","double"], "default": null}
]
}
')
STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION '/hdfs/location'
;