BIG PIPE! The quest for a 6 hour cache lifetime.

When I first challenged my team to build one of the biggest news websites in the world with a 6 hour cache lifetime they laughed. A year later, we're serving millions of users with down-to-the-second real time breaking news, and >99% cache hit ratio.

How? The answer is simple: cache invalidation.

By caching content until it changes and then purging at just the right time, we avoid millions upon millions of needlessly uncached requests, and give users the "twitter-fast" live updates that they crave. By carefully designing each page to maximize cachability with big pipe architecture we ensured that our server never had to render the same content twice.

In this example heavy talk, I'll show how we did it, and how you can do the same. I'll show how our site took advantage of cutting edge APIs from our CDN partner to purge Drupal, Varnish, and CDN cache in real time. I'll argue that cache invalidation is the key to a Drupal site that scales up to limitless heights. I'll show how even small sites can achieve excellent cache invalidation with contributed modules such as expire.

Finally, I'll take a deep dive into Drupal 8's cache tag API and the contributed big pipe module. As you'll see, many of these concepts work out of the box with Drupal 8, and the fastest Drupal ever is here today!

Intermediate to advanced Drupal development knowledge is recommended for this talk, but non-technical users should be able to follow the major concepts.