How are Node.js+Socket.io+MongoDB webapps truly asynchronous?

Question

I have a good old-style LAMP webapp. A week ago I needed to add a push notification mechanism to it.
Therefore, what I did was to add node.js+socket.io on the server and poll the MySQL database every 10 seconds using node.js to check whether there were new items: if so, I would have sent them to the client(s) with socket.io.
I was pretty happy with the result, even if that is not a proper realtime notification (as there is a lag of up to 10 secs).

Now, I am about to build a new webapp which will need push notifications, too. I am wondering whether to go with the same approach as the first one (that I believe is more stable and mature) or to go totally Node.js, without PHP and Apache. As for the database, I have already decided to go for MongoDB.

Finally, my question is: if I go for Node.js+Socket.io+MongoDB will I get a truly near-real-time webapp? I mean, as soon as a new record is inserted into MongoDB, will there be some sort of event triggered that I can catch via node.js, do some checking on it and, if relevant, send the notification to the client? Or will there be anyway some sort of polling on the db server-side and lag, as with my first LAMP webapp?

A related question: can you build a realtime webapp on MySQL without doing any polling as I did with my first app. Or do you need MongoDB (or Redis)?

I hope this question is not too silly - sorry, I am just starting with Node.js and co.

Thanks.

Answer 1

I understand your problem because I switched to node.js from php/apache/mysql too.

• Generally node.js is stable, modules and your scripts are the main reasons for errors

• Real-time has nothing to do with database, it's all about client and server, you can query as many data as you want in your requests and push it to the other client.

• Choosing node.js is very wise but it's harder to implement.

• When you insert a new record to your db, the event is the request itself, you will make a push event along with the database query something like:

  // Please note this is not real code, just an example of the idea
  app.get('/query', function(request, response){
      // Query your database
      db.query('SELECT * FROM users', function(rows){
  
           // Push notification to dan
           socket.emit('database_query_executed', 'to_dan', rows); 
  
           // End request
           response.end('success'); 
  
      })   
  })
  

• Of course you can use MySQL! And any database you want, as I said real-time has nothing to do with databases because the database is in the middle of the process and it's totally optional.

• If you want to use node.js for push notifications and php/apache for mysql then you will need to create 2 requests for each server something like:

  // this is javascript
  ajax('http://node.yoursite.com/push', node_options)
  ajax('http://php.yoursite.com/mysql_query', php_options)
  

  or if you want just one request, or you want to use a form, you can call your php and inside php you can create an http or net request to node.js from php, something like:

  // this is php
  new HttpRequest('http://node.youtsite.com/push', HttpRequest::METH_GET);
  

Answer 2

Using:

• A regular MongoDB Collection as the Store,
• A MongoDB Capped Collection with Tailable Cursors as the Queue,
• A Node worker with Socket.IO watching the Queue as the Worker,
• A Node server to serve the page with the Socket.IO client, and to receive POSTed data (or however else the data gets added) as the Server

It goes like:

1. The new data gets sent to the Server,
2. The Server puts the data in the Store,
3. The Server adds the data's ObjectID to the Queue,
4. The Queue will send the newly arrived ObjectID to the open Tailable Cursor on the Worker,
5. The Worker goes and gets the actual data in the ObjectID from the Store,
6. The Worker emits the data through the socket,
7. The client receives the data from the socket.

This is 'push' from the initial addition of the data all the way to receipt at the client - no polling, so as real-time as you can get given the processing time at each step.

Answer 3

Re: triggers in MongoDB - please see this answer: https://stackoverflow.com/a/12405093/1651408

There are much more convenient triggers in MySQL, but to call Node.js from them would require a bit of work with MySQL UDFs (user-defined functions), for instance pushing data through a Unix socket. Please note that this is necessary only when other applications (besides your Node.js process) are updating the database, and be sure to choose InnoDB as storage in this case (row- vs. table-level locking).

Can see no big problem with your technology choice of sockets.io, even if client-side web sockets aren't supported, you'll fall back (gracefully, I hope) to polling.

Finally, your question is not silly at all, since push technology is definitely superior to the flood of polling requests - it scales better. EDIT: However, would not describe either technology as real-time.

Another EDIT: for a quite well-known and successful setup of this kind please read this: http://blog.fogcreek.com/the-trello-tech-stack/

Answer 4

Have you discovered Chole? It works separately from your web sever and interfaces with it by using HTTP POSTs. That way you can code your web app any which way you want.

Answer 5

Actually Using Push Technology like Socket.IO helps you to use

the server's resource efficiently and also helps you to leverage old browsers to modern browsers making websocket or websocket-like connection.

10 sec polling is a HTTP request which is expensive especially when a lot of users present.

Unlike polling technology, push technology is relatively cheap. Users' client is opening a dedicated socket(ie. websocket) to listen to the server's push notification.

And usually your client-side JavaScript do some actions when the push notification is received.

Using your LAMP stack and Socket.IO with different port (other than 80) will be good enough to implement what you need.

But using Node.js + MongoDB + Socket.IO actually helps you to manage your server's resource much efficiently.

Because those three have non-blocking nature.

If you understand non-blocking concept correctly and implement your app appropriately,

your identical app, an app with same feature but with different language and different database, would be able to handle a lot more requests than general LAMP stack.

---

Above picture is a famous chart of comparing Non-blocking vs Thread way to handle concurrency

Apache(Thread) vs Nginx(Non-blocking)

---

MySQL is a great database. I believe you won't need join and transactions for realtime notification.

MongoDB does not have those two features unless you implement similar features by yourself.

Because of not having those two and some characteristics of its own, MongoDB can store and fetch data much faster than traditional SQL databases.

Switching from MySQL to MongoDB will decrease the time taking to insert and fetch data.

Answer 6

with JS you can open a socket to your server (not old browser), the server will have a ah-hoc program (on an ad-hoc port, so you need the permission to open door and run program on your server) that will send data (almost) realtime from and to the client, and without the HTTP's protocol overhead.old browser will just fall-back to polling mechanism.

I can't see other way to do this (probably there are already "coocked" framework that do this)