With your edit: here's a bug (not sure if it is "the" bug, though):
result = new byte[memoryStream.GetBuffer().Length];
memoryStream.GetBuffer().CopyTo(result, 0);
The length of the buffer is irrelevant. If is the memoryStream.Length
that matters. Frankly, this should just be result = memoryStream.ToArray();
- which would give you the correct result.
And another bug in the SQL:
SQLiteCommand command = new SQLiteCommand("INSERT into 'dump' ('DTime', 'DBinaryData') VALUES ('" + DateTime.Now.ToString() + "', '" + GetSerializedMessages() + "')", connection);
command.ExecuteNonQuery();
Concatenation is never a good idea, but here it is fatal; since GetSerializedMessages()
returns either null
(on failure - not a good idea; should have just thrown) or a byte[]
, this does simple concatenation. If you concatenate a byte[]
the output is not what you expect:
byte[] b = {1,2,3};
string s = "a " + b + " c";
// gives: "a System.Byte[] c"
that clearly doesn't contain the actual data you wanted, so is gibberish. Ideally you should be using parameters here for both the data and the date:
SQLiteCommand command = new SQLiteCommand("INSERT into 'dump' ('DTime', 'DBinaryData') VALUES (@when, @data)", connection);
// note: not sure if SQLiteCommand has an "AddWithValue", but the overall usage
// should be something like this
command.Parameters.AddWithValue("when", DateTime.Now);
command.Parameters.AddWithValue("data", GetSerializedMessages());
command.ExecuteNonQuery();
Finally: don't swallow problems; your serialization code should be (IMO) more like
public byte[] GetSerializedMessages()
{
try {
using(MemoryStream memoryStream = new MemoryStream())
{
BinaryFormatter formatter = new BinaryFormatter();
// skipped: serialize etc
return memoryStream.ToArray();
}
} catch(Exception ex) {
Logger.Log("Failed to serialize. Reason: " + ex.Message);
throw; // it doesn't stop being a problem just because we logged it
}
}
The first thing to look at is whether the byte[]
you get out (via reader["DBinaryData"]
), is 100% identical to the byte[]
you had when you originally serialized. If you don't have a test for that, all bets are off. From the error, it sounds like they're not identical - this could be because of:
- an error in the code that serializes and stores the data
- truncation inside the database storage
- truncation when reading the BLOB (some connections limit the amount fetched in one go)
- an error in the code that fetches and deserializes the data
The first two are totally fatal: if it is those - the data is toast.
A lazy way to compare two byte[]
in an integration test is to compare the hex:
// here expected should be the raw data just after serializing; actual should
// be what you get after storing it in the db and fetching it back, using
// your code
Assert.AreEqual(BitConverter.ToString(expected), BitConverter.ToString(actual));
which gives a nice hex output of any delta. You don't show how you serialize and store the messages, so I can't tell you whether there are any obvious issues there, but please see http://marcgravell.blogspot.com/2013/02/how-many-ways-can-you-mess-up-io.html for a list of common issues here.
Finally, I strongly advise: stop using BinaryFormatter
for this. See questions like this to see other people's pain: basically they can't get their data back after even minor changes (or sometimes just rebuilds). Contract-based serializers would be much safer - I lean towards protobuf-net, but I'm hugely biased.