How to roll out a site migration

Oct 26, 2014

One of the regular challenges for content-managed websites is how to manage the rollout of a new version of your site. I’ve worked on several large-scale web migration projects, and have identified a few key patterns that provide different trade-offs in terms of time to market, predictability and effectiveness.

The Big Bang

The Big Bang is the simplest and easiest to understand. It works like this: you have a public-facing website which you want to replace with a new, improved version, running on a different technology stack. You build your new version offline, improving it until it passes QA and receives sign-off from your stakeholders, at which point you put the new site live and decommission the old one.

This pattern works for small sites, where it is feasible to create a replacement site on a timeline of weeks rather than months. Side-by-side comparison of the new site with the old one can be a useful way of checking for bugs in the new site, and can help to verify that the migration process works. This pattern is also easy for non-technical stakeholders to understand, and can be preferred by people who see the project as a “re-launch”, which may be accompanied by associated marketing activity.

However, if development takes anything more than a few weeks, problems arise:

  • The top priority sections of the new site may be ready early in the project, but cannot be released until the whole project is complete.
  • Users of your site cannot access the new version until it is finished. This removes valuable opportunities for learning about how the new version may improve over the old one, and for large-scale testing in the real world.
  • Larger projects are more likely to fail than small ones, and the “big bang” project has to be a complete replacement for the existing system. This increases the risk of late delivery or cost overrun.
  • The lack of releases during the development process can create uncertainty. Pressure may build up to “just get something out there”, particularly if stakeholders believe there are important calendar dates by which the new site should be available. Because it’s all-or-nothing, you cannot reduce the scope and may be forced to trade off between time and quality.
  • Big Bang implies a change of process once your site is live. During the offline development phase, the consequences of broken code are limited to defect reports from your testing team, but once the site is live any future changes will be visible to the public. Big Bang can allow your team to get away without developing robust release management processes, which might leave you needing to make a painful adjustment to delivery processes after the initial launch.

This approach works best when you have a small and limited scope, where your main challenge is to create an improved version of something that already exists. If your migration is meant to introduce new features or innovations, the lack of exposure to external users and the waste involved in holding back completed work might outweigh the benefits of simplicity.

Most migration projects start out with the intention of delivering a “like-for-like” version of the old site, but very few end up that way. Unless the site in question is very small, you should assume that the new site is going to be different to the old one in more than just the technical sense. Releasing multiple new features simultaneously is a bad idea, and Big Bangs often end up doing exactly this.

Salami slice

The Salami Slice approach works by migrating sections of the site in a piecemeal fashion. This can be more technically complex - substantially more complex in some cases - but has the benefit of enabling code to be released to the real world when it’s ready, instead of being held back until everything is ready.

If your new code is better than the old code - and it should be, or you should not be doing the project at all - then getting it into production is good for your users and for your business. It’s the best way of ensuring that your project is moving in the right direction, and that any new ideas you’re trying out - responsive design, improved analytics or new advertising units - are delivering the value you expect. Regular releases on the new platform can validate (or invalidate!) your assumptions, allowing you to correct course in later releases if things aren’t working out as you expected.

Salami-slicing also allows you to defer decisions about areas of the site that have questionable value. Migrating higher-priority sections of the site may reveal new information that changes the decision about the rest, enabling you to re-think how they should work or whether they are required at all.

What are the down-sides? There are a few, mostly due to technical issues:

  • Where you have logged-in users, you’ll need to be able to maintain single sign-on between the old and new sections of the site. This can vary in complexity depending on how your authentication system works.
  • While both the old and new systems are operational, editorial users will need access to both, and will need to know which areas of the site are controlled by which system.
  • Other site-wide technologies may need to be kept in sync between the two systems - analytics and advertising code, for instance.
  • Managing URLs can be more complex. You’ll need some kind of traffic manager that can route requests to the appropriate back-end systems most CDNs such as Akamai or Fastly can do this, as can Varnish, nginx or Apache (Compoxure is a new and interesting alternative with some useful advantages). In the best case, your URLs will reflect the information architecture of the site; if you have a section called “News” that lives at the /news URL, and you want to migrate that to the new back-end system, your configuration change will be straightforward; otherwise, you might end up having to maintain a lengthy mapping of URLs to back-end systems.
  • If the visual design or information architecture of the new site is radically different to the old one, users may be confused in the transition between areas of the site that are “old” and “new”. This could be mitigated by deferring the more extreme elements of the design or IA changes until later, once more of the site has been migrated.

This approach is the fastest way to get code into production, provided that you can handle the technical setup issues. In my experience, Salami Slice is preferable to Big Bang for all except the smallest sites.

Mirror World

The Mirror World approach involves running your entire new system in parallel to your old one, allowing users to opt in or out of using it. This approach has been used by The Guardian, where for the last two years users have been able to switch at will between the legacy site and a new, mobile-friendly responsive version with a cleaner, modern design. However, the most attention-grabbing point in that last sentence might be the two years for which this project has been running!

Mirror World is similar to Big Bang - it involves the complete replacement of an old system with a new one, but instead of launching the new site in its “final” state, you “launch” to either a pre-selected or opt-in group of users as soon as you have something that meets minimum standards of usefulness and iterate from there.

The key advantage of Mirror World is that you have a public and visible “work in progress”, from which you can get the maximum possible feedback. Stakeholders and users can interact with it, and their reactions can guide future development.

Where Salami Slice encourages you to see a site as a collection of semi-independent sections, Mirror World encourages a holistic view of the site; this may or may not be a good thing for you. It works for The Guardian because, although their site is divided into sections, those sections are all doing similar jobs - presenting news content to the public. If your site’s sections are diverse - some e-commerce, some interactive, some news-driven, some premium content, and so on - then Salami Slice may help you to focus on what makes those sections work on their own terms. Mirror World is a better fit for publishing “platforms” which need to do a small number of core tasks well across a large site.

The down-sides?

  • Where Big Bang’s all-or-nothing approach can create arbitrary deadlines and time pressure, the open-ended nature of Mirror World means that there’s no obvious point at which it’s “finished”. This could be mitigated by taking a data-driven approach - if we can measure site performance in terms of user actions, a Mirror World can be judged ready to take over from the old site once it starts to produce better results (which also means that the focus on Mirror World development should be on getting to that point as quickly as possible, a valuable motivation).
  • Managing the invitation or opt-in/out process can be tricky. To gather a useful quantity of feedback requires a substantial number of opt-ins, but getting users to opt-in may be difficult if it requires them to make an affirmative choice.
  • The users who do opt-in may not be representative of your wider user-base, meaning that conclusions drawn on the basis of their behaviour may be skewed. This could be solved by randomly opting some people in to the new version, ensuring that the distribution of Mirror World users reflects the overall user-base, but this can only be done once you’re sure that the Mirror World site is of sufficient quality to avoid negative experiences.

Mirror World works best for heavily content-driven sites where the migration process is about improving the design of the site, and less well for sites with varied functionality where the migration is about functional changes more than design. The benefits of a public beta are substantial, but the problems of Big Bang - you can only make it the permanent replacement for the old site when everything is finished - still apply. Unlike Big Bang, you’re gathering feedback through the delivery process, but you still face an all-or-nothing decision about when to decommission the old site.

Selective Beta

The Selective Beta approach is a mix of the above. Like Big Bang, the plan is to release a complete replacement for the old site in one go. Borrowing from Mirror World, we want to get real-world feedback early and often and, as with Salami Slice, we can build things section-by-section, even if those sections are only released to beta testers and not to production.

Running beta tests of new features allows us to get some of the learning that Mirror World gives us, but with a more narrowly-defined scope. Rather than needing to release a beta version of the entire site, we can release beta versions of individual features.

There is no single “right” way to do beta tests of this kind. One option is to have invite-only beta tests, where the site being tested is un-connected to the production site. This avoids the technical complexity that comes with Salami Slice - you don’t need to worry about single sign-on, or site-wide features. This does shift some work on to the testers, as they have to understand the fact that this new feature doesn’t integrate with the existing site, and that this is intentional. The trade-off here is between accurately simulating the final product and getting early feedback with minimal technical setup costs.

If integration with the rest of the site is essential, Selective Betas can have high technical setup costs. As per Salami Slice, we need to worry about URLs and routing and site-wide features, as well as the question of how to run opt-in beta programs as per Mirror World. If you’re going to do that, you might as well do Salami Slice and get the pay-off from getting the feature into production, where it’s making your users happier and making you money, rather than waiting in a staging area until all of the other features are complete and tested.

Conclusion

The approach you take will depend on the nature of your site, the goals of your stakeholders and the needs of your users. There’s no universally-correct way of doing a complex site migration. In my experience, Salami Slice and Big Bang are the most common, the decision between them coming down to the size of the site being migrated; Salami Slice is better for larger sites, Big Bang acceptable for smaller ones. Mirror World offers an interesting alternative for large homogenous sites, and Selective Beta can be a way of dodging the technical complexities of the other approaches, at a cost of longer lead times.

The other key factor is what happens after the migration. Big Bang has a discontinuity in process between the offline “build” phase and the post-launch “maintain and improve” phase. The other approaches, by exposing work to real users, ensure that real-world quality standards are kept high from the beginning, and continuous delivery of high-quality code is essential for them to work. You can only establish a process of continuous improvement if you have a well-defined method of improving the site in small increments, and Big Bang allows you to ignore this. It’s not that you can’t do continuous improvement after a Big Bang release, but you will not necessarily have developed the process necessary to support this, as you would have been forced to do by the other approaches.

If you have any site migration experiences you’d like to share, give me a shout on Twitter.