Setting up a reverse proxy, server specs?

Question

We currently have two servers in a 1+1 setup. 1 apache, 1 mysql. Both dedicated with 32 cores and 64GB of RAM. With a 300K+ catalog. Magento EE 1.11

Following Magento's 'Optimizing for peak performance' I was wondering what would be the optimal server specs if we were to add a reverse proxy such as Varnish or nginx based on the current 1+1 setup?

In the guide the test hardware specs for all the servers were:

• CPU: 2 x Intel® Xeon® CPU E5645 @ 2.40GHz - 12 cores, 24 threads
• HDD: RAID1 - LSI MegaRAID SAS 9260-4i; 2 x SAS 164GB 15,000rpm
• RAM: 24GB ECC
• Network Interface Card: Intel 1GB 82576

I feel like adding another server with the specs of the current setup is overkill. All the reverse proxy is doing is handing out cached content and if something is dynamic it gets send down to apache/PHP for processing. Any good guides or books on this subject?

Answer 1

Firstly, don't blindly read any guide (whether provided by Magento or not) and try to ascertain server configuration and server specification for your own deployment.

Their tests and results are entirely specific to their test conditions and not applicable to your own store.

Every Magento store is different

Beyond that, selecting hardware for your deployment isn't something anyone here can do with the information you have provided.

You haven't provided nearly enough information for a useful response.

• How many visitors do you have per day peak
• How many visitors do you have per hour peak
• Do you offer digital downloads
• What amount of transit are you currently peaking at (in Mbps)
• What proportion of web traffic is SSL and none SSL
• How long are you prepared to keep stale items in a cache

For a reverse proxy, you need ...

• Proportional amount of RAM for your cache stores (min 4GB)
• Standard disks (disk I/O is not relevant)
• High end network card (to reduce interrupts, improve latency & throughput)
• Proportional amount of CPU cores to traffic level
• Optional. SSL hardware decryption

Our advice

Don't bother. At the moment, your deployment already has flaws

• 2 single points of failure
• Bottlenecked MySQL throughput over network
• Too few cores in web server
• Too many cores in DB server
• Over specified HDDs in web server
• Under specified HDDs in DB server

Believe it or not, Magento runs better on a single machine up until you hit the limits of vertical scaling, because the bottleneck of MySQL being on a high latency remote machine constrained over whatever you network speed is - is far higher than having it on the same system and having RAM speed throughout.

Don't get me wrong, there's certainly a tipping point where multi-purposing 1 machine creates its own contention issues; but 32 cores is a fairly small ceiling of load.

If you still press ahead

A Varnish cache for a low traffic site (anything under 200k unique visitors per day) doesn't usually need more than 2GB RAM and a 1 core CPU - with low end HDDs. But for you, it may as well just be on your web server anyway - it wouldn't draw any resources, and by putting it on a third machine - you'll now have 3 single points of failure and even more network activity bottlenecking your throughout.

Take that paper with a pinch of salt - it is a marketing exercise from Peer 1 and the majority of it is not correct.

Answer 2

In my experience any VPS (not OpenVZ) with 512MB of RAM and 1 or 2 cores should be enough for an nginx proxy for most users. Step up to 1GB and the proxy will easily handle 10Mbit/s of mixed http/https cached/dynamic content traffic. Unless there is a specific reason not to you can/should just run it on the same server as Apache and if anything it will be a small net decrease in resource usage.