Java XML vs SQL vs custom data storage

Question

I am writing a piece of software for my girlfriend and I'm struggling to choose between ways of permanent storage. Since the application revolves heavily around storing data, it's a fairly important issue.

My specific needs are:

• Small footprint for a constantly expanding dataset that could easily exceed 10,000 entries
• Easy to access from code, easy to port to other languages (in case I write a front-end in another language in the future)
• (Optional) Easy to access directly, so that manual editing can be performed
• File-based storage preferred, since it greatly increases the portability of the data.

I've considered XML, which fails mostly on the second and third points, but also a little on the first point (I'm not sure how scalable XML really is, I've not done any tests).

I've also considered SQLite, but it requires a third-party library to work properly with Java (as far as I'm aware) and also fails on the third point, since direct-access is impossible.

As far as bespoke is concerned, I'd really rather avoid it since re-inventing the wheel is generally considered a Bad Thing®, but if it really is the best option then I will consider it.

Note: I know this is a possible duplicate of Best data storage method for small personal application(SQL Database, XML, or other file type) but that one's favoured answer is SQLite, and there is a difference between C++ and Java where SQLite is concerned, so I figure this question is sufficiently different. Correct me if I'm wrong.

Answer 1

In the end, I opted for a bespoke JSON-like solution since it wasn't important to the project that the file use a standardised format.

I think any SQL-based solution would have been too heavyweight for this project, however the data is tabular in nature so I might consider switching storage modes in the future (the code is written such that the permanent storage method can easily be modified, which I would recommend as a modus operandi for other people who have a problem choosing between storage methods).

Answer 2

For something like relational data I'd prefer H2 database engine. HSQLDB could also make it. It will be able to query the data more easily than XML or JSON. Think of it as an advanced SQLite that can run in a server mode and be accessed by external tools easily (actualy delivered with H2).

Look at neo4j if your data is graph-strucutred.

Answer 3

Accessing a db through ODBC makes the code less db-technology dependent, as most databases have ODBC drivers.