Reading a GZip compressed AVRO file in Ruby
Question
I have a bunch of AVRO files which I've compressed externally using GZip. I'm trying to read them in Ruby without decompressing them, but can't get it to work.
Solution
Solved it:
require 'avro'
def open_avro(file)
if file =~ /avro$/
Avro::DataFile.open(file)
elsif file =~ /avro\.gz$/
begin
$/ = ""
file = Zlib::GzipReader.open(file)
reader = Avro::IO::DatumReader.new(file, nil)
file.rewind # we need to rewind because DatumReader seeked thefile
avro = Avro::DataFile::Reader.new(StringIO.new(file.gets), reader)
end
return avro
end
end
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow