Cache Fluffer / Gorilla Caching / Cache Warmer

The relatively simple introduction of a cache fluffer can make a huge difference to performance, particularly at peak load. The idea is simple, keep the cache up to do date so you don't have to go and get data when the user requests the site.

        public T Get(Func<T> factory, string cacheKey)
        {
            if (cache.Contains(cacheKey))
            {
                Task.Factory.StartNew(() => FluffTheCache(factory, cacheKey));
                return cache.Get(cacheKey);
            }
            return Put(factory, cacheKey);
        }

        public void FluffTheCache(Func<T> factory, string cacheKey)
        {
            var expiry = cache.getExpiry(cacheKey);
            if (expiry.Subtract(new TimeSpan(0, 0, 10)).Second < new Random().Next(1, 60))
            {
                Put(factory, cacheKey);
            }
        }

        public T Put(Func<T> factory, string cacheKey)
        {
            var item = factory();
            cache.Set(item, cacheKey);
            return item;
        }

The beauty with this method is that popular items will always be in cache where as less popular items will not be cached for unnecessary lengths of time.

You also get some randomization mixed in for free meaning that items don't come out of sync at the same time.

Out of band caching

One of the things we try and do is always keep our site up, sounds simple right? One way to achieve this is to have a good caching policy.

Here's what we do:

• Cache a response from some external service (be it a db or web service, whatever)
• When a user request the resource ALWAYS serve it from cache if it's there
• Then "out of band" (i.e. on a different thread) make a call to get it from the external service and update the cache if you get a valid response.
• But only do this if the cache is stale

Simple right? That way if your external service goes down you can still serve it from cache. Your user will always get the fastest possible response (i.e. they wont be the sucker who has to go and get it). And your service remains up as much as possible.