Text File DataProvider in .NET

Question

I want to read/write my entities in flat text file, and due to huge amount of data, Serlization/Deserialization is not proper solution.

Is there any data provider to work with text file data source (like csv,...)?

Answer 1

You can use OleDB combined with the Jet-engine:

using (var connection = new OleDbConnection(
    "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\"" + directoryPath 
    + "\";Extended Properties='text;HDR=Yes;FMT=Delimited(,)';"))
using (var command = new OleDbCommand(
    "SELECT * FROM [" + fileName + "]", connection))
{
    connection.Open();
    using (var reader = command.ExecuteReader())
        while (reader.Read())
            ...
}

Answer 2

You could always look into a LINQ provider for the target file, obviously LINQ to XML is available. A quick search yields other things like LINQ to CSV, and LINQ to Text.

However, I don't know what support they offer with regards to what you are looking for, so you'll have to review them and not take my answer verbatim.

Answer 3

though not a Text File DataProvider... check protobuf out - http://code.google.com/p/protobuf-net/
It is written specifically to make serialization less time+space consuming...

Its author is here on Stackoverflow - Marc Gravell

Answer 4

Have you considered using SQLite? It should do what you want and is pretty fast in a lot of scenarios.