The New Moz search engine optimisation Q&A: 100Okay URL Migration Case Research

The author’s views are entirely his or her own (with the exception of the unlikely event of hypnosis) and may not always reflect the views of Moz.

Should you always expect a decrease in traffic during a site / URL migration, even if it is a temporary migration?

In case you haven’t noticed, Moz recently launched a brand new SEO question and answer platform that the whole world can see, explore, and use to learn more about SEO.

Originally introduced many years ago as a private function for pro members, the Q&A was opened to the public – and the search engine – back in 2011.

In the years that followed, it grew to over 60,000 posts Covers every imaginable SEO topic and tens of millions of page views. For a long time, a significant portion of Moz’s organic traffic came from the Q&A.

Unfortunately, as is so often the case, the questions and answers have been seriously neglected over time. As a result:

  • The platform collected a ton of Technical debtwhich makes it almost impossible to update

  • Pages are loading so slowly many users have given up completely

  • Spam became more common

  • Moderation tools were out of date, and couldn’t keep up

Because of these reasons, two predictable things happened:

  1. The questions and answers became less useful and satisfactory to users

  2. In the course of time, the traffic decreased significantly

So Moz had a choice: improve the questions and answers right away, or stop them.

Fortunately, we decided to improve it.

In cooperation with the fantastic team at NodeBB (highly recommended by the way) we quickly created a new Q&A with our existing database, but with completely modern technology in the front and back end.

Why this migration was a challenge

We were under time pressure. What would normally take months we had to do in a few weeks. This presented unique challenges from an SEO perspective.

The biggest challenge? Our entire URL structure had to be changed. (If we had more time we could have avoided that, but it was a luxury we didn’t have.)

That meant we had to migrate thousands of URLs that looked like this:

The migration also included all Moz user profiles whose number was in the Hundreds of thousands. To be fair, most of the user profiles aren’t actually indexed.

Regardless, this was a huge migration!

The other potential red flag was that most of the questions and answers would use client-side rendering – not as best SEO practice! We could have implemented a solution for server-side rendering, but again we just didn’t have the time. We feared that Google might have problems rendering the content and that could hurt our rankings (more on that later).

How we did the migration

To do this huge migration while minimizing the risk of losing traffic, we followed basic SEO site migration best practices, along with some “special” extras for an extra boost.

1. 301 forwarding mapping

To put it simply, how you implement your 301 redirects will either make or break your migration implementation.

For us, this was actually the easiest and most straightforward part of the job as we have a lot of experience with site migrations! (Does anyone remember seomoz.org?)

  1. We have compiled a list of all possible URLs and URL paths. It’s amazing how many URLs and patterns you can overlook. A good crawler is essential to make sure you don’t forget anything. For Moz, we were able to do this with data from Google Analytics, the Search Console, and our own Moz Pro site crawling.

  2. We have mapped each URL to the corresponding URL on the new NodeBB platform. Although we found a lot of edge cases, it was relatively easy to do.

  3. We made sure that everything is redirected via 301. This is important as many platforms and developers use 302 by default. While Google has told us that they pass PageRank through 302s and 301s alike, Google has also indicated that 301s are a stronger canonicalization signal.

Speaking of canonicalization: we also did crawls of the new URL structures with the NodeBB platform. In cases where we found URL paths that didn’t match our old patterns or that we thought were irrelevant, the NodeBB team was able to easily set up canonicalization patterns to avoid Google over-indexing our URLs.

2. Maximum sitemap management

Sitemap management was an important part of our migration strategy. This involved two steps:

1. Old URLs: We already had sitemaps of all the old urls. It is important that we left these sitemaps live and registered them in the Search Console. That way, Google would continue to crawl the old URLs and “see” the redirects.

Often, webmasters make the mistake of removing sitemaps too early, which can reduce Google’s crawl rate. This means that it may take longer for Google to process the redirects.

Sitemaps aren’t a perfect guarantee that Google will visit all of your old urls, but they do provide a hint. In fact, after several months, we still had several thousand URLs that Google still hadn’t visited even with the sitemaps. Regardless, without the sitemaps of the old URLs, the problem could have taken a lot longer.

2. New URLs: Our old sitemaps were grouped into lists of 50,000 each – the maximum allowed by Google. There are some suggestions in the SEO community that grouping URLs into smaller sitemaps can actually improve crawling efficiency.

Fortunately, NodeBB allowed us to create smaller sitemaps by default, so we did just that. Instead of 2-3 sitemaps with tens of thousands of URLs, we now had 130 individual XML sitemaps, usually with no more than 500 URLs each.

3. Spam and Cruft Cleanup

As I mentioned earlier, the old Q&A ran out 60,000 individual posts built up over 10 years.

Inevitably, some of these posts were of very poor quality. We suspected that both the poor quality of the posts and the poor user experience could cause Google to rank us lower.

Again, time constraints meant we couldn’t do a full content pruning audit. Fortunately, NodeBB came back to the rescue (this is starting to sound like an advertorial – I swear it isn’t!) And ran all 60,000 posts through their spam plugin to remove the most obvious, inferior culprits.

In total, we removed over 10,000 posts.

We didn’t redirect these urls and just left them 404 after the migration. Nobody seemed to miss her.

For your information, another great resource for cropping content is this great webinar with Bernard Huang, Suganthan Mohanadasan, and Andy Chadwick.

4. Better internal linking & user experience

Although we carried over the same content and basic design, the migration was a great opportunity to improve the user experience. To achieve this, we made two small changes to the entire UX:

  1. Added breadcrumbs throughout the app

  2. Added highly relevant “related questions” in the sidebar

The old Q&A did not have any of these functions. Users who landed on one question had no opportunity to investigate other questions. As a result, we’ve suffered frustratingly high bounce rates and poor website engagement metrics for years.

Results: Before and after the migration

To be honest, I’ve never seen a migration like this before. After doing many migrations, I’ve done my best to prepare everyone for the most likely scenario: prepare for a 15-30% traffic drop for 1-3 months while Google processes all of the URLs.

In truth, nothing even remotely happened.

As you can see in the graph below, we’ve seen an increase in traffic almost from day one.

In fact, in the two months after the migration, Google organic traffic was on Q&A pages rose almost 19% compared to the traffic on all other sites.

What caused this instant increase in traffic? Was it the improved sitemap coverage, better internal linking, or something else?

We just don’t know for sure, but we have a clue.

No sooner had we started the new Q&A than the engagement numbers shot through the roof:

  • More time on site

  • Lower bounce rate

  • More pages per session

In short, the users seemed to be much happier and more committed with the new experience.

Could the improved user engagement have helped the rankings?

Again we don’t know. Google is pretty narrow-minded about how or not it can use user click signals for ranking purposes, but we have our suspicions.

Departure into the future

We keep improving the Q&A experience. Most of all, we’re working on prioritizing speed improvements, especially given Google’s work on Core Web Vitals.

Regardless, this was definitely an enjoyable migration where we didn’t see a drop in traffic – not even for a single day!

If you significantly improve your user experience, website architecture, and SEO best practices, migrations can actually turn out to be a quick net profit.

Comments are closed, but trackbacks and pingbacks are open.