EBCDIC to ASCII conversion, handling numeric values

Question

I’m attempting to convert files from ECDIC to ASCII format and have run into an interesting issue. The files contain fixed length records with some fields being signed binary integers (described as B4 in the record layout), and long-precision numeric values (described as L8 in the record layout). I’ve been able to convert character data with no problem, but I’m not sure how to go about converting these numeric values. From a reference manual for the original system (an IBM 5110), the fields are described below.

B indicates the length (2, 4, or 8 bytes) of numeric data items in fixed-point signed binary integer format that are to be converted to BASIC internal data format. For record I/O file input, the next 2, 4, or 8 bytes in the record contain a signed binary value to be converted by the system into internal data format and assigned to the variable(s) specified in the READ FILE or REREAD FILE statement using a FORM statement.

and

L indicates long-precision (8 characters) for numeric values. For input, this entry indicates that an eight-position, long-precision value in the record is to be assigned without conversion to a corresponding numeric variable specified in the READ FILE or REREAD FILE statement.

EDIT: Here's the code I'm using for the conversion

private void ConvertFile(EbcdicFile file)
{
    if (file == null) return;

    var filePath = Path.Combine(file.Path, file.FileName);
    if (!File.Exists(filePath))
    {
        this.Logger.Info(string.Format("Cannot convert file {0}. It does not exist.", filePath));
        return;
    }

    var ebcdic = Encoding.GetEncoding(37);
    string convertedFilepath = Path.Combine(file.Path, file.ConvertedFileName);
    byte[] fileData = File.ReadAllBytes(filePath);

    if (!file.HasNumericFields)
        File.WriteAllBytes(convertedFilepath, Encoding.Convert(ebcdic, Encoding.ASCII, fileData));
    else
    {
        var convertedFileData = new List<byte>();
        for (int position = 0; position < fileData.Length; position += file.RecordLength)
        {
            var segment = new ArraySegment<byte>(fileData, position, file.RecordLength);
            file.Fields.ForEach(field =>
                {
                    var fieldSegment = segment.Array.Skip(segment.Offset + field.Start - 1).Take(field.Length);
                    if (field.Type.Equals("string", StringComparison.OrdinalIgnoreCase))
                    {
                        convertedFileData.AddRange(
                            Encoding.Convert(ebcdic, Encoding.ASCII, fieldSegment.ToArray())
                            );
                    }
                    else if (field.Type.Equals("B4", StringComparison.OrdinalIgnoreCase))
                    {
                        // Not sure how to convert this field
                    }
                    else if (field.Type.Equals("L8", StringComparison.OrdinalIgnoreCase))
                    {
                        // Not sure how to convert this field
                    }
                });
        }

        File.WriteAllBytes(convertedFilepath, convertedFileData.ToArray());
    }
}

Answer 1

Okay, so I've figured out how to convert both fields. B4 fields are very straightforward. They are essentially a 4-byte array which can be converted to an integer.

//The IBM 5110 were big endian machines, so reverse the array 
if (BitConverter.IsLittleEndian)
    Array.Reverse(by);

int value = BitConverter.ToInt32(by, 0);

The L8 fields are 8-bytes arrays that represented an IBM Double Precision Float. There are many ways this can be converted to an IEEE 754 Float. A few examples can be found at:

• How To Read IBM 370 Data from a Binary File
• Transform between IEEE, IBM or VAX floating point number formats and bytes expressions

Here's the version I used based on guidance from the articles.

private double IbmFloatToDouble(byte[] value)
{
    if (ReferenceEquals(null, value))
        throw new ArgumentNullException("value");

    if (BitConverter.ToInt64(value, 0) == 0)
        return 0;

    int exponentBias = 64;
    int ibmBase = 16;
    double sign = 0.0D;

    int signValue = (value[0] & 0x80) >> 7;
    int exponentValue = (value[0] & 0x7f);
    double fraction1 = (value[1] << 16) + (value[2] << 8) + value[3];
    double fraction2 = (value[4] << 24) + (value[5] << 16) + (value[6] << 8) + value[7];
    double exponent24 = 16777216.0;             // 2^24
    double exponent56 = 72057594037927936.0;    // 2^56

    double mantissa1 = fraction1 / exponent24;
    double mantissa2 = fraction2 / exponent56;
    double mantissa = mantissa1 + mantissa2;
    double exponent = Math.Pow(ibmBase, exponentValue - exponentBias);

    if (signValue == 0) 
        sign = 1.0;
    else 
        sign = -1.0;

    return (sign * mantissa * exponent);
}

Answer 2

You must first know the fixed record size. Use FileStream.Read() to read one record worth of bytes. Then Encoding.GetString() to convert it to a string.

Then fish the fields out of the record using String.SubString(). A B4 is simply a SubString call with a length of 4, L8 with a length of 8. Further convert such a field to a number with Decimal.Parse(). You may have to divide the result, it wasn't clear what fixed-point multiplier is used. Good odds for 100.