Copying a text file line by line vb.net half the size?

Question

I'm trying to modify specific lines in a 6 gig text file (SQL script). So I read it in with IO.StreamReader.ReadLine and write to a new file with IO.StreamWriter.WriteLine. If the line matches a certain condition, I'm modifiying it before I write it.

The problem is, the resulting file is exactly half (1.999582...) the size of the original file...

I'm trying to make sure the encoding is the same using:

sw = New IO.StreamWriter(NewFilepath, False, sr.CurrentEncoding)

But it doesn't make a difference, the new file is half the size of the old...

Answer 1

Where are you setting the encoding for your StreamReader, sr? If you are not doing this explicitly, and if you are setting the encoding of the StreamWriter before you perform any reads of your file(my best guess), then the CurrentEncoding of the StreamReader may change (it autodetects from the source file).

From MSDN on StreamReader.CurrentEncoding

The current character encoding used by the current reader. The value can be different after the first call to any Read method of StreamReader, since encoding autodetection is not done until the first call to a Read method.

To determine the encoding you can read off the first line of the file with the StreamReader and then do :

sw = New IO.StreamWriter(NewFilepath, False, sr.CurrentEncoding)