How to shift the bytes using Big Endian format instead of Little Endian in C++?

Question

I am writing Byte Array value into a file using Java with Big Endian Byte Order format.. Now I need to read that file from C++ program...

That Byte Array which I am writing into a file is made up of three Byte Arrays as described below-

short employeeId = 32767;
long lastModifiedDate = "1379811105109L";
byte[] attributeValue = os.toByteArray();

I am writing employeeId , lastModifiedDate and attributeValue together into a single Byte Array and that resulting Byte Array I am writing into a file and then I will be having my C++ program which will retrieve that Byte Array data from file and then deserialize it to extract employeeId, lastModifiedDate and attributeValue from it.

Below is my working Java code, which writes Byte Array value into a file with Big Endian format:

public class ByteBufferTest {

    public static void main(String[] args) {

        String text = "Byte Array Test For Big Endian";
        byte[] attributeValue = text.getBytes();

        long lastModifiedDate = 1289811105109L;
        short employeeId = 32767;

        int size = 2 + 8 + 4 + attributeValue.length; // short is 2 bytes, long 8 and int 4

        ByteBuffer bbuf = ByteBuffer.allocate(size); 
        bbuf.order(ByteOrder.BIG_ENDIAN);

        bbuf.putShort(employeeId);
        bbuf.putLong(lastModifiedDate);
        bbuf.putInt(attributeValue.length);
        bbuf.put(attributeValue);

        bbuf.rewind();

        // best approach is copy the internal buffer
        byte[] bytesToStore = new byte[size];
        bbuf.get(bytesToStore);

        writeFile(bytesToStore);

    }

    /**
     * Write the file in Java
     * @param byteArray
     */
    public static void writeFile(byte[] byteArray) {

        try{
            File file = new File("bytebuffertest");

            FileOutputStream output = new FileOutputStream(file);
            IOUtils.write(byteArray, output);           

        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }
}

Now I need to retrieve Byte Array from that same file using the below C++ program and deserialize it to extract employeeId, lastModifiedDate and attributeValue from it. I am not sure what is the best way on the C++ side. Below is the code I have so far:

int main() {

    string line;

    std::ifstream myfile("bytebuffertest", std::ios::binary);

    if (myfile.is_open()) {

        uint16_t employeeId;
        uint64_t lastModifiedDate;
        uint32_t attributeLength;

        char buffer[8]; // sized for the biggest read we want to do

        // read two bytes (will be in the wrong order)
        myfile.read(buffer, 2);

        // swap the bytes
        std::swap(buffer[0], buffer[1]);

        // only now convert bytes to an integer
        employeeId = *reinterpret_cast<uint16_t*>(buffer);

        cout<< employeeId <<endl;

        // read eight bytes (will be in the wrong order)
        myfile.read(buffer, 8);

        // swap the bytes
        std::swap(buffer[0], buffer[7]);
        std::swap(buffer[1], buffer[6]);
        std::swap(buffer[2], buffer[5]);
        std::swap(buffer[3], buffer[4]);

        // only now convert bytes to an integer
        lastModifiedDate = *reinterpret_cast<uint64_t*>(buffer);

        cout<< lastModifiedDate <<endl;

        // read 4 bytes (will be in the wrong order)
        myfile.read(buffer, 4);

        // swap the bytes
        std::swap(buffer[0], buffer[3]);
        std::swap(buffer[1], buffer[2]);

        // only now convert bytes to an integer
        attributeLength = *reinterpret_cast<uint32_t*>(buffer);

        cout<< attributeLength <<endl;

        myfile.read(buffer, attributeLength);


        // now I am not sure how should I get the actual attribute value here?

        //close the stream:
        myfile.close();
    }

    else
        cout << "Unable to open file";

    return 0;
}

I have specifically set the storage Java-side to big-endian which means I know where each byte belongs I guess. So how do I code it while shifting the bytes into the proper position for each and every value? Right now I am coding it as little-endian I guess which is not what I want...

I was reading somewhere that I can use ntoh in C++ to deserialize the Byte Array.. Not sure whether htons will be much better solution as compared to what I have currently?..

If yes, then I am not sure how to use that in my current C++ code?

Can anybody take a look on C++ code and see what I can do to improve it, as I don't think it is looking much efficient? Any better way to deserialize the Byte Array and extract relevant information on the C++ side?

Answer 1

If the Java and the C++ Code is developed by you then it's perhaps better to use the Google Protocol Buffer (https://developers.google.com/protocol-buffers/docs/overview) instead to write your own serializer / deserializer.

If you really want to write your own implementation the best way is to write a buffer class which receives the byte-stream as parameter (e.g. as Constructor parameter) and make some access methodes readShort / readLong / readInt / readByte ... and only swap the needed bytes.

class ByteBuffer{
  explicit ByteBuffer(uint8_t* byteStream, uint16_t streamLength);
  uint8_t readUInt8(uint16_t readPos)const {return m_byteStream[readPos];} // no conversion needed
  uint16_t readUInt16(uint16_t readPos)const {
    const uint8_t byteCount = 2;
    union{
      uint16_t u16;
      uint8_t u8[byteCount];
    }tmp;
    for(uint8_t i=0; i<byteCount; ++i){
      tmp.u8[i] = readUInt8(readPos+i*8);
    }
    return ntohs(tmp.u16); // do conversion
  }
  ...
}

Checks for read behind the buffer is missing here. If your code should be portable then you have use ntohl / ntohs (see: http://forums.codeguru.com/showthread.php?298741-C-General-What-do-ntohl%28%29-and-htonl%28%29-actually-do). If you swap the bytes by our own then your code is not portable (runs only on Little-Endian machines). If you use ntoh then it would also run on a such machine.

For convenience I would also write a wrapper class where you can read and write your fields (e.g. employeeId) directly:

class MyBuffer{
  uint16_t readEmployeeId()const{return m_Buffer.readuint16(EmployeeId_Pos);}
  ....
  static const uint16_t EmployeeId_Pos = 0;
  ....
}