Storing binary Ruby Marshalled objects in Git. Use filters to convert to text (JSON or YAML) and back? [closed]

Question

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.

Closed 8 years ago.

Improve this question

I want to version control binary files that contain the data to run our project. The format used by the program is Marshalled Ruby objects, there is no options to change this in the program, Windows only, and it's closed source. Lovely right?

Here is some good news though. Most of the classes are well documented and for the most part are close to just being Structs, but some have custom Marshalling methods. I also plan to build tools for diffing and merging these files, but figuring out how to put them in to the repo is more important.

So, would using filters to smudge binary files into text (JSON or YAML) for storage in Git and clean them back out to binary for the working directory be a wise idea or just a waste of time?

Rough implementation of both filters, dropping imports, using YAML, and untested with Git:

puts Marshal.load(gets).to_yaml # Smudge

puts Marshal.dump(YAML.load(gets)) # Clean

Edit: Thought I should note that there is deflated Ruby scripts stored in one of these files. A clean project has about 133 KB of Zlib deflated script in it, about 800 KB when inflated.

Answer 1

I wouldn't get too caught up in the guideline of not storing binary files in Git.

The real challenge comes, as you suggested, in diffing and merging these files. If you store them as text, you likely don't need to do anything special here. YAML and JSON are both relatively easy to diff and merge manually.

If it is convenient, store text. This will let anybody diff the files using whatever tools they already have available.

On the other hand, if you are already planning to write your own diff and merge tools (which can be hooked into Git) you shouldn't have too much trouble storing the original binary files.

Storing binary files and using your custom diff / merge tools will require users to have those tools available for diffing and merging.