Convert hex string to bytearray and write to file

Question

I wrote one app using C# to read data from a serial port and show the data on a textbox in hex string format. Finally, I saved all data to a binary file. If data is big (maybe > 20mb) it throws an out of memory error. How can I solve this? Here is my code:

private void btn_Save_Click(object sender, EventArgs e)
{
    SaveFileDialog save_log = new SaveFileDialog();
    save_log.DefaultExt = ".bin";
    save_log.Filter = "Binary File (*.bin)|*.bin";
    // Determine if the user selected a file name from the saveFileDialog.
    if (save_log.ShowDialog() == System.Windows.Forms.DialogResult.OK &&
        save_log.FileName.Length > 0)
    {
        try
        {
            string hexString = Content.Text.ToString();
            FileStream stream = new FileStream(save_log.FileName, FileMode.Create, FileAccess.ReadWrite);
            stream.Write(Hex_to_ByteArray(hexString), 0, Hex_to_ByteArray(hexString).Length);
            stream.Close();
        }
        catch (Exception ex)
        {
            MessageBox.Show(ex.Message);
        }
    }
}

private byte[] Hex_to_ByteArray(string s)
{
    s = s.Replace(" ", "");
    byte[] buffer = new byte[s.Length / 2];
    for (int i = 0; i < s.Length; i += 2)
    {
        buffer[i / 2] = (byte)Convert.ToByte(s.Substring(i, 2), 16);
    }
    return buffer;
}

Answer 1

You're creating the byte array twice. Also, the .Replace over such a long string doesn't help. You can avoid all this:

try
{
    var stream = new FileStream(
        save_log.FileName,
        FileMode.Create,
        FileAccess.ReadWrite);

    WriteHexStringToFile(Content.Text, stream);

    stream.Close();
}
catch (Exception ex)
{
    MessageBox.Show(ex.Message);
}

private void WriteHexStringToFile(string hexString, FileStream stream)
{
    var twoCharacterBuffer = new StringBuilder();
    var oneByte = new byte[1];
    foreach (var character in hexString.Where(c => c != ' '))
    {
        twoCharacterBuffer.Append(character);

        if (twoCharacterBuffer.Length == 2)
        {
            oneByte[0] = (byte)Convert.ToByte(twoCharacterBuffer.ToString(), 16);
            stream.Write(oneByte, 0, 1);
            twoCharacterBuffer.Clear();
        }
    }
}

Also, take a look at Encoding and/or BinaryFormatter which might do all of this for you.

Update:

First of all, please note that your whole idea of storing megabytes of data in a string is a nonsense, and you shouldn't do that. You should process your data in smaller parts. Because of this nonsense I'm unable to provide you with a working demo (on IDEONE for example), because of resource limitations of online compilers. I've tested the code on my machine, and as you can see I can even process 50 MB strings - but it all depends on the amount of memory you have available. If you do such things, then on every machine it will be easy to reach the limit of available memory. And the methods you ask about in this particular question are irrelevant - the problem is because you fill your memory with tons of data in your Content.Text string. When memory is almost full, the OutOfMemoryException can occur at almost any place in your code.

You can view the whole picture in your browser to see all the details.

Answer 2

Use IEnumerable. That will avoid the large byte array.

I don't know what is in Content.Text. If it's a byte array, maybe you can change

static internal IEnumerable<byte>Hex_to_Byte(string s)

into

static internal IEnumerable<byte>Hex_to_Byte(byte[] bytes)

and modify the code a bit

FileStream stream = new FileStream(save_log.FileName, FileMode.Create, FileAccess.ReadWrite);
foreach( byte b in Hex_to_Byte(hexString) )
    stream.WriteByte(b);
stream.Close();



    static internal IEnumerable<byte>Hex_to_Byte(string s)
    {
        bool haveFirstByte = false;
        int firstByteValue = 0;

        foreach( char c in s)
        {
            if( char.IsWhiteSpace(c))
                continue;

            if( !haveFirstByte)
            {
                haveFirstByte = true;
                firstByteValue = GetHexValue(c) << 4;
            }
            else
            {
                haveFirstByte = false;
                yield return unchecked((byte)(firstByteValue + GetHexValue(c)));
            }

        }

    }
    static string hexChars = "0123456789ABCDEF";
    static int GetHexValue(char c)
    {
        int v = hexChars.IndexOf(char.ToUpper(c));
        if (v < 0)
            throw new ArgumentOutOfRangeException("c", string.Format("not a hex char: {0}"));
        return v;
    }