Is there an easy way to manually force a caching HttpClient 4.3.x to bypass the cache?

Question

I am using CachingHttpClientBuilder to create a CloseableHttpClient in by an HttpServlet to fetch internal resources. In most cases I want it to use its cache, and that seems to be working fine. However, periodically the client is used to check for updates to a special remote file (basically it has configuration data). In that case, I would like to be able to tell the client to fetch the resource via HTTP even if the cache contains it and it isn't marked as stale yet.

When I've run into situations like this in JavaScript I have often appended a bogus query string parameter with a timestamp in it so that the URL would not match a cached entry. However, I would think there is a better solution in this case since we have direct, programmatic access to the HTTP client.

Lest someone suggest just changing the remote server so that it sets a Cache-Control: no-cache header to prevent this, please understand that this is outside of my control. The goal is to bypass the cache in this instance regardless of whether or not the remote server says it can/should be cached.

Someone might also suggest not using a caching HttpClient in this case, which would be OK, but it seems less than ideal to me because then I would need to make a second HttpClient (other portions of this application require caching of these HTTP resources in order to perform sufficiently well).

EDIT: user3360944 suggested using a HttpCacheInvalidator, which may be the right solution, but I am not sure yet how to do this. Could someone give an example of what to put in the flushInvalidatedCacheEntries method to remove a cached entry for a given URL? (I have particular URLs that I never want to cache)

new HttpCacheInvalidator() {
    @Override
    public void flushInvalidatedCacheEntries(HttpHost host, HttpRequest req) {
        // What do I need to do here to invalidate a cached entry for
        // say, http://www.example.com/path/file.txt?
    }

    @Override
    public void flushInvalidatedCacheEntries(HttpHost host, HttpRequest request, HttpResponse response) {        
        // Do nothing here since I don't need to invalidate anything
        // based on the response received
    }
}

Answer 1

It will involve a bit of custom code but one can bypass the caching layer entirely by slightly tweaking HttpClient's execution pipeline

class ConditionalCachingExec implements ClientExecChain {

    private final ClientExecChain mainExec;
    private final ClientExecChain cachingExec;

    public ConditionalCachingExec(final ClientExecChain mainExec, final ClientExecChain cachingExec) {
        this.mainExec = mainExec;
        this.cachingExec = cachingExec;
    }

    @Override
    public CloseableHttpResponse execute(
            final HttpRoute route,
            final HttpRequestWrapper request,
            final HttpClientContext clientContext,
            final HttpExecutionAware execAware) throws IOException, HttpException {
        URI uri = request.getURI();
        if ("/stuff".equals(uri.getPath())) {
            return mainExec.execute(route, request, clientContext, execAware);
        } else {
            return cachingExec.execute(route, request, clientContext, execAware);
        }
    }

};

class MyCachingHttpClientBuilder extends CachingHttpClientBuilder {

    @Override
    protected ClientExecChain decorateMainExec(final ClientExecChain mainExec) {
        ClientExecChain cachingExec = super.decorateMainExec(mainExec);
        return new ConditionalCachingExec(mainExec, cachingExec);
    }

};


CloseableHttpClient httpClient = new MyCachingHttpClientBuilder().build();

Answer 2

Since you have the CachingHttpClientBuilder, you can configure it to use a specific HttpCacheInvalidator with this method

setHttpCacheInvalidator(HttpCacheInvalidator cacheInvalidator) 

You can use the HttpCachedInvalidator to invalidate a request prior to issuing it if you want the uncached version of the response.