Magento with Turpentine - Per session/connection speed issue

Question

Magento CE 1.9.0.1 with Varnish 4.0.3 & Turpentine 0.6.8

When I load the page or inner pages for the first time those are slow(as First bite is always slow) and on second attempt it becomes quicker coming from Varnish, now since page has been stored in Varnish cache is should be served faster(coming from Varnish) no matter which connection or network I use, but in my case if at same time I go to another PC/network(or sometimes even different browsers on same PC) and try to load the same page it becomes slow. I am using all the default settings of Turpentine in my VCL config.

Here is my /etc/varnish/varnish.params

# Varnish environment configuration description. This was derived from
# the old style sysconfig/defaults settings

# Set this to 1 to make systemd reload try to switch vcl without restart.
RELOAD_VCL=1

# Main configuration file. You probably want to change it.
VARNISH_VCL_CONF=/home/www/vhosts/example.org/httpdocs/var/default.vcl

# Default address and port to bind to. Blank address means all IPv4
# and IPv6 interfaces, otherwise specify a host name, an IPv4 dotted
# quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=99.25.55.148
VARNISH_LISTEN_PORT=80

# Admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082

# Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret

# Backend storage specification, see Storage Types in the varnishd(5)
# man page for details.
VARNISH_STORAGE="malloc,12G"

# Default TTL used when the backend does not specify one
VARNISH_TTL=120

# User and group for the varnishd worker processes
VARNISH_USER=varnish
VARNISH_GROUP=varnish

# Other options, see the man page varnishd(1)
DAEMON_OPTS="-p thread_pool_min=5 -p thread_pool_max=500 -p thread_pool_timeout=300 -p cli_buffer=16384 -p feature=+esi_ignore_other_elements -p vcc_allow_inline_c=on"

1. Could it be because I am storing everything in RAM as VARNISH_STORAGE="malloc,12G" and there might be a default behavior of memory to just wipe out after every minute or second and I also need to save in SSD? or there could be another issue?

2. If you say "First bite is always slow" than does it mean it will be slower for each different browser, and each session/connection? this is what is actually happening with me.

3. Almost all the stores I have seen which are providing page in like milliseconds(having Varnish) does not encounter the same issue. When I browse those sites from various connections it all looks same with pretty much same speed. How can I achieve this?

Please also look at Admin > System > Configuration > Caching Options which may help to answer

Answer 1

Could it be because I am storing everything in RAM as VARNISH_STORAGE="malloc,12G" and there might be a default behavior of memory to just wipe out after every minute or second and I also need to save in SSD? or there could be another issue?

No, if your Varnish is set up correctly this should not happen.

If you say "First bite is always slow" than does it mean it will be slower for each different browser, and each session/connection? this is what is actually happening with me

Yes, if you are using the VCL Fix option (which is the default), then the first request is always passed through Varnish to Magento to get a proper session ID. This means, if a request come to Varnish that does not have a frontend cookie set, then it will pass that request through to Magento - which generally means a slower page load. The second request for that same page should be much faster, as well as for other pages that are already in the Varnish cache. This page from our wiki gives more information about ways to make sure Varnish is set up and working properly.

... but in my case if at same time I go to another PC/network(or sometimes even different browsers on same PC) and try to load the same page it becomes slow.

In this case, when you go to another PC/network you're starting a new session, which means the first page request is getting passed through to Magento get a good session ID.

Almost all the stores I have seen which are providing page in like milliseconds(having Varnish) does not encounter the same issue. When I browse those sites from various connections it all looks same with pretty much same speed. How can I achieve this?

They may not be using the VCL fix, or they may just have a very fast Magento site even w/out Varnish - and only be using Varnish to improve concurrency.

Varnish/Turpentine is a very useful tool, especially to improve concurrency - but it's not a replacement for properly optimizing your site/environment as much as possible.

Answer 2

I know that turpentine varnish does not serve a site from cache, if the user visiting it, has no frontend session id. I think this is a needed step since magento1 does not support varnish out of the box.

So any user getting on a Site without a valid session will not get the content served from varnish which explains why you have the higher loading time when switching the browser / PC.

This can be looked up here under troubleshooting: https://github.com/nexcess/magento-turpentine/wiki/FAQ