Redis and Memcached are Dead: The Case for DB-Backed Hierarchical Caching
Despite my years of contributions to Drupal's support for Redis and Memcached, I think it's time to phase them out. We have several services we like run on these daemons, including caching, locking, and queues.
The database has gotten lots better:
- Internal lock optimization in MySQL and MariaDB
- Better HA options: MySQL MHA and MariaDB with Galera
- The near-universality of InnoDB
- Cheap, prevalent solid-state storage (SSD)
- Support for certificate authentication and TLS
Redis and memcached have not improved much:
- No coherency with Memcached failover
- The new Redis replication (Sentinel) isn't even as good as MySQL's in 2012
- Security based on iptables or a password (not great for cloud)
- No durability for queues or locks
There are better need-specific options:
- Hierarchical caching with apcu and coherency through the database
- Modern distributed lock/queue daemons for non-caching purposes
In short, we should stop using Redis and Memcached:
- Small sites should continue using the database backends for everything
- Big sites should deploy lock/queue daemons and using hierarchical caching
Why this matters for core:
- Certain cache items do not need cluster-wide consistency, but a consistent API is the only one we offer right now.
- Some cache items are content-addressed and can use lighter-weight consistency mechanisms.
- For some cache items, it may be better to hit a stale item (in the local cache) than miss or escalate to "L2" or "L3."