Limit Sitecore MediaCache Folder Size

Question

The Question

Is there a way to limit the size of the MediaCache folders that get created for each Sitecore site by default? The Sitecore 6.2 documentation says it's not possible, but we're on 6.4.1.

Background

We have a website that is deployed to three, load-balanced Sitecore servers. One of these servers is used exclusively by our search appliance to index our site content. Since it browses our site much faster and consumes much more data than a normal user would, our MediaCache folder for our website (/App_Data/MediaCache/website) fills up the disk on that web server. I suppose we could disable the media cache altogether on that server, but we prefer to keep the configurations identical between each web server, if possible.

Thoughts?

Answer 1

What's the issue with the MediaCache folder filling up? I presume you have a lot of media items and/or a small(ish) disk size on that server? I also assume you are not using Lucene for your indexing, instead a crawler with will visit each page and "scrap" the html, in much the same way as Google would? Why is the crawler requesting the images, and not just the HTML markup?

I don't know of any setting off hand, there may be something but it would mean looking through the Sitecore Kernel in dotPeek.

But, using what is there now, you would have to make a change to the config file on that one server.

You could set the MediaCache folder cleanup task to run sooner/more frequently, BUT you would need to set the parent agent interval to be the same, so it just means a lot more Sitecore chatter.

<agent type="Sitecore.Tasks.CleanupAgent" method="Run" interval="00:01:00">
  <files hint="raw:AddCommand">
    <remove folder="/App_Data/MediaCache" pattern="*.*" maxAge="00:01:00" recursive="true" />
  </files>
</agent>

Personally, I would just use a patch config file to disable the MediaCache for that folder. But, if there is truly that much data that you are running out of disk space then make sure you have your server architecture set up correctly otherwise there is going to be a lot of network traffic. If you have a single shared database between the 3 servers then this could be an issue.

Add the following to a .config file and place in /App_Config/include/ folder on just the indexing server.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <settings>
      <setting name="Media.CachingEnabled" value="false" />
    </settings>
  </sitecore>
</configuration>

Or for a particular website just patch <site name="website"> element and add cacheMedia="false".

If your index server vs front end servers have different URL's then you could consider having the index server output the full URL for the media items so they point at the front end servers. That way the images would get cached on those servers instead.