inotify and rsync on large number of files

Question

I am using inotify to watch a directory and sync files between servers using rsync. Syncing works perfectly, and memory usage is mostly not an issue. However, recently a large number of files were added (350k) and this has impacted performance, specifically on CPU. Now when rsync runs, CPU usage spikes to 90%/100% and rsync takes long to complete, there are 650k files being watched/synced.

Is there any way to speed up rsync and only rsync the directory that has been changed? Or alternatively to set up multiple inotifywaits on separate directories. Script being used is below.

UPDATE: I have added the --update flag and usage seems mostly unchanged

#! /bin/bash

EVENTS="CREATE,DELETE,MODIFY,MOVED_FROM,MOVED_TO"

inotifywait -e "$EVENTS" -m -r --format '%:e %f' /var/www/ --exclude '/var/www/.*cache.*' | (
WAITING="";
while true; do
    LINE="";
    read -t 1 LINE;
    if test -z "$LINE"; then
        if test ! -z "$WAITING"; then
                echo "CHANGE";
                WAITING="";
                rsync --update -alvzr --exclude '*cache*' --exclude '*.git*' /var/www/* root@secondwebserver:/var/www/
        fi;
    else
        WAITING=1;
    fi;
done)

Answer 1

I ended up removing the compression option (z) and upping the WAITING var to 10 (seconds). This seems to have helped, rsync still spikes CPU load but it is shorter lived. Credit goes to an answer on unix stackexchange

Answer 2

You're using rsync to synchronize the root directory of a large tree, so I'm not surprised at the performance loss.
One possible solution is to only synchronize the changed files/directories, instead of the whole root directory.
For instance, file1, file2 and file3 lay under from/dir. When changes are made to these 3 files, use

rsync --update -alvzr from/dir/file1 from/dir/file2 from/dir/file3 to/dir

rather than

rsync --update -alvzr from/dir/* to/dir

But this has a potential bug: rsync won't create directories automatically if target folders don't exist. However, you can use ssh to execute remote command and create directories by yourself.
You may need to set SSH public-key authentication as well, but according to the rsync command line you paste, I assume you've already done this.

reference:
rsync - create all missing parent directories?
rsync: how can I configure it to create target directory on server?
How to use SSH to run a shell script on a remote machine?
SSH error when executing a remote command: "stdin: is not a tty"