Magento 2 Multiple Nodes Clear non-FP caches

Question

Considering a multi-node setup where there is a single Varnish server for load balancing and multiple Magento servers (Apache and Magento on each), what is the recommended setup so that using Admin (which runs on it's own server again) and clearing a non-FP cache (ie Blocks Output Cache) so that all other nodes also clear their local respective cache in /var folder?

Read somewhere that a shared NFS folder for /var is one option, but wouldn't that cause collisions (write I/O)?

Also read somewhere you can have Redis setup for this but we understand that Redis is an "or" case with Varnish. We are already using Varnish.

Answer 1

in multi server setup you need to keep your media, caches and sessions in centralized storage. all separate dedicated nodes:

1. fpc cache + load balancer = nginx + varnish + nginx
2. frontend master node = nginx + php-fpm
3. auto-scaling nodes = nginx + php-fpm
4. admin server = cron
5. sessions/cache = redis
6. nfs storage = media + logs (you can use syslog to pull messages)
7. database = mysql data

when all your nodes connected to centralized redis and varnish, you can easily clean all caches, then all nodes pull fresh data at once.

so for example you can go down from big multi dns setup:

             -> ASN1
           /          \
DNS1 -> LB1 -> FMN1 -> \
        \               \
         | -> ADMIN ->   -> NFS -> REDIS -> DATABASE
        /               /
DNS2 -> LB2 -> FMN2 -> /
           \          /
             -> ASN2 

you need to use also centralized control system, to interact with each node and send commands to all of them at once or control specific services, etc

like ansible, or you build on specific platform, any orchestration tools you like, etc

Answer 2

Redis is definitely not an "or" case with Varnish. Varnish acts as your frontend cache, being a reverse proxy in front of your sites. However Redis acts as a key-value store for backend data caching.

Setting up a Redis instance for your caches would resolve this, even if you had put the FP cache key as Redis too since it wouldn't stop Varnish being in front of the site. I'd also recommend that you use the Redis instance for Session storage as well as cache storage.

Resource: https://devdocs.magento.com/guides/v2.3/config-guide/redis/config-redis.html