General purpose utility or library for compiling/decompiling binary data files?

Question

I have various binary file formats which I need to dump to some kind of text format, edit and then recompile (possibly to a slightly different version of the binary format). Of course I could write a bunch of utility code in C/C++ to do this kind of thing, and maybe leverage a library for the text side of things (XML or JSON or whatever), but this is a task pattern that keeps cropping up in my work and it seems to me that there probably ought to exist already some kind of general purpose tool for this kind of job.

Obviously there would need to be some way of describing the binary file formats - a schema or some such - this would need to support all the obvious data types (strings, ints of various sizes, bools, etc, floats would be nice too), and things like count fields, length fields, padding/alignment, etc, to handle all the typical stuff that you find in a data file. It would also be useful to have endianness support, as the binary file may have a different endianness than the platform we're running on.

I'd be very surprised if something like this doesn't exist already, but I have had no luck so far via Google (how do you even describe such a utility or library succinctly ?). The closest thing to this I ever remember seeing was Apple's resource compiler/decompiler utilities back in the days of "Classic" Mac OS. These used a C-like syntax, with resource template definitions in header files to describe the formats of the various binary resources, and you could compiled/decompile between this C-like source syntax and the binary resources.

Answer 1

BinX, the Binary XML Description Language, plus the corresponding library, seem to fit your purpose.

Also of interest might be Chapter 24. Practical: Parsing Binary Files of the Practical Common Lisp book, which demonstrates how to write a library for parsing ID3 tags in (binary) MP3 files in 150 lines of Lisp code. *

* nowadays, i probably wouldn't use Lisp, but a more modern functional language like Clojure. see Reading binary data in Clojure for an example.

P.S. binary data library or binary data file library are good keywords for Google.

Answer 2

"SynalyzeIt" in the Mac App Store might be worth a look...