longitude reading measured in degrees with a 1x10^-7 degree lsb, signed 2’s complement

Question

I am receiving data from a gps unit via a udp packet. Lat/Lng values are in hex.

Example Data
13BF71A8 = Latitude (33.1313576)
BA18A506 = Longitude (-117.2790010)

The documentation explains that longitude/latitude readings are measured in degrees with a 1x10^-7 degree lsb, signed 2’s complement.

For the Latitude I can convert using this formula:
13BF71A8 = 331313576 * 0.0000001 = 33.1313576

This code works for Lat but not for Lng:

function convertLat(h){
var latdec = parseInt(h,16);
    var lat = latdec * 0.0000001;
    return lat;
}

console.log("LAT: " + convertLat("13BF71A8"));

I am having trouble converting the Longitude value. Does anyone know how to convert the Longitude?

Answer 1

Because you are using signed numbers, you need to specify a point at which the hexadecimal code should flip to the bottom. This will be happening at 7FFFFFFF and up. Now update your code to check if the input is greater than this number, and if so, subtract it from the input.

function convert(h) {
    dec = parseInt(h, 16);
    return (dec < parseInt('7FFFFFFF', 16)) ?
        dec * 0.0000001 :
        0 - ((parseInt('FFFFFFFF', 16) - dec) * 0.0000001);
}

The only reason your example worked is because the output was expected to be positive.

---

As AlexWien mentioned in the comments: Since parsing 7FFFFFFF and FFFFFFFF are giving the same integers every time, you could store them as constants. Their values are 2147483647 and 4294967295 respectively.

Answer 2

Lat and Long use the same algorithm for conversion. Your latitude accidentaly wokred because it is positive (33.13)

The test Longitude is negative, which makes the error in the conversion algorithm visible, as usually with negative numbers.

Answer 3

use this for 2 compliment

private static Decimal ParseHexStringToDouble(string hexNumber) {
    long result = 0;
    result = int.Parse(hexNumber, System.Globalization.NumberStyles.HexNumber);
    return result;
}