Parse a large file of non-schematized json using Jackson?

Question

I have a very large .json file on disk. I want to instantiate this as a Java object using the Jackson parser.

The file looks like this:

[ { "prop1": "some_value",
    "prop2": "some_other_value",
    "something_random": [
      // ... arbitrary list of objects containing key/value 
      //     pairs of differing amounts and types ...
    ]
  },
  // ... repated many times ...
  {
  }
]

Basically it's a big array of objects and each object has two string properties that identify it, then another inner array of objects where each object is a random collection of properties and values which are mostly strings and ints, but may contain arrays as well.

Due to this object layout, there isn't a set schema I can use to easily instantiate these objects. Using the org.json processor requires attempting to allocate a string for the entire file, which often fails due to its size. So I'd like to use the streaming parser, but I am completely unfamiliar with it.

What I want in the end is a Map where the String is the value of prop1 and SomeObject is something that holds the data for the whole object (top-level array entry). Perhaps just the JSON which can then be parsed later on when it is needed?

Anyway, ideas on how to go about writing the code for this are welcome.

Answer 1

Since you do not want to bind the whole thing as single object, you probably want to use readValues() method of ObjectReader. And if structure of individual values is kind of generic, you may want to bind them either as java.util.Maps or JsonNodes (Jackson's tree model). So you would do something like:

ObjectMapper mapper = new ObjectMapper();
ObjectReader reader = mapper.reader(Map.class); // or JsonNode.class
MappingIterator<Map> it = reader.readValues(new File("stuff.json"));
while (it.hasNextValue()) {
   Map m = it.nextValue();
   // do something; like determine real type to use and:
   OtherType value = mapper.convertValue(OtherType.class);
}

to iterate over the whole thing.