How significant is the performance difference between in-memory and database storage [closed]

Question

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 5 years ago.

I am creating a chat application in signalr. It currently has some functionality with storing to a database for permanent storage of messages. I want to implement user lists and have been re-reading this article:

http://www.asp.net/signalr/overview/signalr-20/hubs-api/mapping-users-to-connections#inmemory

How significant is this difference between in-memory storage? Is there likely to be more than a second difference between say, 1000 users and 10,000 users messaging each other at a normal rate of say 1msg per second (they've had lots of sugar)? And how useful is it? Any references to previous tests on performance would be hugely appreciated.

Secondly, I may be going about this the wrong way, but if I want to store a list of users and there are multiple chat-rooms, I assume I will need a separate dictionary per room? And if so, is this feasable since I want users to also be able to generate their own rooms too. I just don't see how in-memory can be achieved this way.

Thank you

Answer 1

I hate to provide a single link for an answer but this OSS fully functional chat room https://jabbr.net addresses everything you're worried about.

Source: https://github.com/jabbr/jabbr

To comment on your main concern in-memory vs db: DB adds the ability to persist data or even scale out your application BUT is FAR less performant than in-memory. Now don't get me wrong, even though it's FAR less performant it doesn't mean it'll take seconds to send messages depending on your app... It's all about tradeoffs.

As for applying a DB oriented architecture to 1000 vs 10,000 users it all depends on how you implement it. For instance scale out provider REDIS runs all in-memory but then persists to disk on an interval. This is obviously extremely fast but allows for "some" data loss.

Answer 2

This is not directly related to your original question.

I think you cannot save them in in-memory storage. The problem is if Application Pool recycles, all messages stored in-memory will be gone.

You cannot predict when App Pool will recycle (altogether you can manually set a specific time to recycle). It can happens in either few days or few hours.

Ideally, you want to store in persistent location like Database. If you worry about the performance of saving to Database, you can async the database saving task.

Answer 3

Check out this comparison:

http://ruturaj.net/redis-memcached-tokyo-tyrant-and-mysql-comparision/

Redis (in memory) outperforms MySQL (classic DB) Obviously, their fields of application are strongly different, so we can't say that, in general, Redis is better than MySQL. Every software project is his own world, and many different solutions can be applied. In some situations, Redis adn MySQL are used together for different parts of a platform (E.g. Magento eCommerce)

It's up to you to evaluate if you need an in-memory solution or standard database. I think that a throughput of 1 insert/second can be achieved easily through MySQL, but this is only an opinion. I suggest you to write an implementation of your product which is "Redis aware": start using MySQL, but use it through an abstraction layer which adapts both to MySQL and Redis (I don't know if such a layer is available open source, but you can be release your own...)