Dealing with large data?

Question

I'm using WinForms and C# for my application and my data is mainly some strings, integers and many lists. Now I store them in xml and text files but I just found out that reading the data takes too long. I'm using XmlWriter and XmlReader. For example I have 4 xml files totalling 2-3 mbs which I parse and ~250 text which I read their contents all at once. The loading takes 3-4 minutes and I'm not using threads. Is this normal or is something else going on? Should I use some other way to store my data? Does it have to do that I only use one thread?

EDIT I found the problem. It had nothing to do with the reading (I think). Anyway though, assuming the my data won't get more than a couple of MBs, should I use a database or I'm fine with xml?

Answer 1

You are probably mostly I/O bound on this, so parallelizing will not help much - but it certainly should not take 3 to 4 minutes (more like 3 to 4 seconds) - something else is going on.

If you have large structured data, you should consider a database instead - then you can query for the data you need instead of having to load up everything at once. I.e. SQL Server Compact has a very small footprint for use in desktop applications.