Quality-oriented Drupal DevOps

Continuous Integration. Everybody wants it. Few have made it work with Drupal, largely due to how difficult it is to orchestrate all the moving pieces -- database, code, and assets. Especially on sites already in production, where it's most needed.

Freelock has cracked this nut. We've built a system that runs Behat tests in a dev environment with every push to a development branch, runs visual regression tests on each release candidate, and manages deployments to stage and production environments.

This is the story of how we went from no automated testing and some basic manual deployment process to a fully automated platform that runs multiple tests on several dozen sites, ensuring we don't break things that once worked and giving us the confidence to make broader, riskier changes to client sites and know that they work correctly.

Our solution is driven by a chat bot. Leveraging some unique properties of Matrix.org chat rooms, our bot listens for messages from a variety of internal systems, and triggers jobs in a new Concourse CI server that handle the actual test runs and deployments.

We start by discussing DevOps in general, what that means and why you should run your business this way. Then we go over the parts that need to be put together to make a succesful continuous integration and continuous deployment system, and what choices we found worked (and a few that didn't). Most of this discussion is conceptual, not technical, and is suitable for people interested in learning the whys and what-fors of DevOps.

Towards the end we drill down into the specifics, and open the discussion up to questions and implementation details.

Technologies covered: Drupal 6, 7 and 8. Git branch strategies. Matrix.org chat, state handling, custom bots. Concourse CI (compared to Jenkins), CI/CD. Handling heterogenuos environments (self-hosted, Acquia, Pantheon, etc). SaltStack for server Configuration Management/distribution of configuration. Docker for tools/deployment scripts/test runners.