how to send a buffer or byte array in a MQTT using nodeJS and protobufjs

Question

I have a nodejs (node v0.10.26) MQTT code which publishes strings to a topic on a remote MQTT server. Now I am trying to publish a byte array and a buffer to the same topic, but I get an error The code :

var ProtoBuf = require("protobufjs");
var mqtt = require('mqtt');
var builder = ProtoBuf.loadProtoFile('event.proto'),
    as2 = builder.build('as2'),
    Message=as2.Message;

var message = new Message({
    "message_type": "EMOTICON"
});
console.log("message : "+message);
var buffer_message= message.encode();
client = mqtt.createClient(1883,'hostNameOfMQTT',{ encoding: 'binary' });
client.publish('NewTopic',buffer_message);
client.on('connect', function() {
    console.log("Message published : "+buffer_message);
     client.end();
});

and I get this error when I execute node.exe sampleMqtt.js

C:\node>node.exe SimpleMQTT.js message : .as2.Message

C:\node\node_modules\mqtt\lib\generate.js:178 length += Buffer.byteLength(payload);

                 ^
TypeError: Argument must be a string
    at Object.module.exports.publish (C:\node\node_modules\mqtt\lib\
generate.js:178:22)
    at Connection.eval [as publish] (eval at <anonymous> (C:\node\no
de_modules\mqtt\lib\connection.js:58:29), <anonymous>:2:26)
    at MqttClient._sendPacket (C:\node\node_modules\mqtt\lib\client.
js:430:20)
    at MqttClient.<anonymous> (C:\node\node_modules\mqtt\lib\client.
js:105:12)
    at MqttClient.EventEmitter.emit (events.js:117:20)
    at MqttClient._handleConnack (C:\node\node_modules\mqtt\lib\clie
nt.js:498:10)
    at Connection.<anonymous> (C:\node\node_modules\mqtt\lib\client.
js:191:10)
    at Connection.EventEmitter.emit (events.js:95:17)
    at Connection._write (C:\node\node_modules\mqtt\lib\connection.j
s:187:12)
    at doWrite (_stream_writable.js:226:10)

However When I try to publish buffer_message.tostring(), I get the object details as a string but not the actual ArrayBuffer?? client.publish('AnkitTopic',buffer_message.toString()); Output with toString() -> ByteBuffer(offser=0,markedOffSet=-1,length=2,capacity=16)

I dont want a to string, I want the actual byteBuffer to be transmitted via MQTT.

Someone, Please suggest how can I do this!! Im guessing its not too hard to get, It's possible in java, then why not in nodeJS?

Answer 1

I found the solution to this problem and hope it helps other :

We can attach a header with every payload which we send to MQ, encode it and the final payload should look something similar (this is how we convert it into byte buffer) :

    var headerLengthBuffer = new ByteBuffer();
    var headerBuffer = header.encode();
    var messageLengthBuffer = new ByteBuffer();
    var messageBuffer = event1.encode();

    headerLengthBuffer.writeVarint32(headerBuffer.length);
    messageLengthBuffer.writeVarint32(messageBuffer.length);

    //Create the payload object
    var payload = headerLengthBuffer.toBuffer() + headerBuffer.toBuffer()+ messageLengthBuffer.toBuffer() + messageBuffer.toBuffer() ;

    return payload;

The above payload created can be published easily and is accepted by MQ as well.

Answer 2

Can you try to send another kind of binary content like a file to see that it works?

Latest MQTT.js versions supports binary payloads https://github.com/adamvr/MQTT.js/issues/109

Answer 3

the protobufjs package supports native node.js Buffers.

According to protobufjs/message API you just have to call message.toBuffer().

Here is a working example:

var ProtoBuf = require("protobufjs");
var mqtt = require('mqtt');
var builder = ProtoBuf.loadProtoFile('company.proto'),
    Company = builder.build('Company'),
    Employee = Company.Employee;

var employee = new Employee("James Bond", 007);

var buffer_message = employee.toBuffer();

var client = mqtt.connect('mqtt://test.mosquitto.org');

client.on('connect', function() {
   client.publish('MyCompany', buffer_message);
   client.end();
});

Using the following company.proto file:

package Company;

message Employee{
    optional    string      name = 1; 
    optional    uint32      id = 2;
}