Favicons are stored in a special cache. That cache lasts a long time (days to weeks) and can't be cleared by clearing the normal content cache. You could test favicons on a remote computer (like browserstack) or just wait 'till the cache expires.
Or, see the next answers for (ugly) workarounds: