Migrating from Private Cloud to AWS: Troubles, Tricks, and Automation

Even 1 to 1 migrations are not as easy as they seem. Planning and role delegation is truly essential to make sure you have close to zero downtime. It’s possible to migrate certain parts of your infrastructure while both systems are live with no downtime(e.g. in our case our message broker system and all of its producers and consumers).

Simplify and automate as much as possible. While it’s very tempting to try and fix everything you know is wrong with your infrastructure during a large migration, it introduces an inordinate amount of unnecessary complexity, especially when you’re migrating on a deadline and trying to keep costs under control. Make sure you document everything you do, and use configuration management and AWS tools to capture states of your infrastructure as you go, so you don’t have wasted effort re-creating environments when you encounter migration problems.

Successful migrations are truly a DevOps practice. Without close collaboration between our fledgling Infrastructure team and established Dev teams, we never would have migrated in time, or even been aware of things to look for. Delegation of responsibilities is good, siloing is the enemy.

We used almost all open-source software projects for our migration, e.g. Varnish, HA Proxy, Ansible, Nginx, Rabbit MQ, etc. This is all great, but make sure you’re ready to deal with some interesting scenarios (and downtimes) usually arising from knowledge gaps and assumptions you made about what a software product actually does. We’ll go over how we hit various bottlenecks in our first week live, and how we quickly resolved the issues, and how we could have better planned for them.