My radio popper blog: How Should You Handle Expired Content?

Introduction

Handling expired content can be an overwhelming experience for any SEO in charge of a dynamic website, whether it be an e-commerce, a classified (example: job search, real estate listings), or a seasonal/promotional (example: New York Fashion Week) site. Even something as fundamental as glancing at the Google Webmaster Tools account for the site can evoke gut-wrenching emotions, especially if the site has amassed tens of thousands of 404 errors. How are you supposed to come up with a process to manage this? What should the process even look like?

What Qualifies as Expired Content?

There are a number of examples that would be considered "expired" content. Expired content is content on a website that is only relevant for a limited period of time. Below are examples of different scenarios that would need to be considered expired content.

Job Search/Real Estate Listings: Job listings routinely expire, especially when positions become filled. The same is true for real estate when property is sold.

What is the best way to handle expired listings, especially if the content is only available for a very limited amount of time?

E-commerce: Expired products can occur when products that are sold on the site routinely change for one reason or another, such as:

What happens when the site no longer sells a product?
What happens if the product becomes temporarily out-of-stock?
What about seasonal products that are only sold during limited times of the year?

Perhaps most importantly, sites that have to worry about expired content tend to be enormous - often comprised of hundreds of thousands of pages. Thus, recommendations need to be manageable and clear. Taking an individual look at all expired and out-of-stock products is unrealistic. Start thinking, is there a way we can build a process for these type of changes?

The Options

Like most SEO solutions, there isn't necessarily one right answer. We need to take a look at each individual situation on a case-by-case basis and take into consideration the current back-end of the site, as well as the resources and the technological capabilities of the site's team. There is a time and a place to use each of these options for expired content. Identifying the right scenario for each situation is very powerful.

I. The 404 Error

It makes sense for webmasters to think that 404ing expired content on the site is the approach to take. After all, isn't that the very definition of a 404 page?

(Distilled's 404 Page)

In most situations, a page on the site should not be 404ed. Why?

Disadvantages of 404 pages

404ing pages that used to be live on the site is just not beneficial for SEO because it alerts search engines that there are errors on the site. Essentially, you're wasting the site's crawl allowance on crawling/indexing pages that no longer exist.

Also another issue with 404 pages is that they tend to bounce - users land on the page, see that the page no longer exists, and quickly leave. Users are vital to the site and our goal as SEOs is to not only ensure that the site gains organic traffic, but that the users stay, browse through the site, and ultimately, convert.

Custom 404 Page

If you must 404 pages for one reason or another, consider creating a custom 404 page, so that in the chance that a visitor lands on the page, there is an opportunity for them to convert. A custom 404 page can also include keyword-rich links to other pages on the site (for instance: see Crate and Barrel's 404 page).

Determining the Right Approach for Expired Content

Now that we know the disadvantages of 404ing pages, what is the right approach in dealing with expired content? To determine this, multiple considerations need to be taken into account, such as:

Was there significant traffic (and not just organic, but also consider direct) coming to this page?
How can we provide the best user experience?
Has this page received external links? How is this page currently internally linked to?
Is there content/resources on the page that users would still find useful?

II. The 301 Permanent Redirect

Advantages of 301 Redirects

For the vast majority of scenarios, I'd suggest 301 redirecting your expired content to another page. This is usually the best option for SEO and can also be customized to enhance the user experience via dynamically-generated messages. For instance, if a product page had garnered external links, you're able to retain most of the link equity from those links via a 301 redirect (whereas with a 404, that link equity is lost). Why would you want to lose the link equity that you had worked so hard to obtain? Furthermore, it demonstrates to search engines that your site is well-maintained and up-to-date or "fresh".

(screenshot of infographic from Dr. Pete's epic status code post)

Where should you 301 redirect these pages?

Consider what would result in the best user experience. You want to redirect these pages to the most relevant page. A suggestion is to take a look at the breadcrumbs and redirect the page based on the internal navigation of the site. For instance, the product page can be redirected to the most relevant sub-category page. You want to be careful that you're redirecting the page to another page that is likely to stay on the site in the foreseeable future, otherwise you run the risk of having to deal with this issue again (not to mention that having a 301 redirect lead to another 301 redirect to another 301 redirect is not considered good SEO practice). A safe bet is to redirect these pages to the most relevant category page, as these are pages on the site that are least likely to change.

Dynamically-Generated Messages

You can customize and improve the user experience by implementing a dynamically-generated message via cookies during a 301 redirect. This would result in users who have landed on expired products receiving a message letting them know that the original product they were seeking is no longer available. This enhances the user experience because it informs users on why they are being redirected.

Disadvantages of 301 Redirects

For some sites, implementing multiple 301 redirects might affect server performance (though for a well-designed site, this should not be an issue). However, if it is true for your site, knowing that site speed is a search engine ranking factor, we want to be wary of the impact we may have by implementing this strategy. If this is the case for your site, consider only 301 redirecting the pages that have gained external links or have received significant amounts of traffic and directing the remaining pages to a customized 404 page. Please bear in mind that this is not an ideal scenario and is just a workaround.

III. Leave the Page on the Site

Advantages of Leaving the Page As Is

Sometimes product pages still garner significant amounts of traffic or are rich in unique content and contain information that is still useful to visitors. It would be worth leaving the original product there, especially if the page has unique, high-quality, evergreen content, but have a message that the product has been discontinued. This will likely provide the best user experience as it will provide a strong call-to-action.

How Could You Set Up the Message?

Implement a JS overlay that would include similar products as the one that has been discontinued and drive users to those new products. Consider incorporating keyword-targeted internal links to drive traffic to those sites. This provides a positive user experience and is especially important for repeat customers.

Example: Real Estate

For this niche, expired listings bring tons of traffic since people are curious about what has been sold and what the market looks like. Thus, consider leaving these pages on the site, but include additional information on the top of the page, such as "contact us to see similar listings" or "here are some other houses in the area that have similar selling prices."

Disadvantages of Leaving the Page As Is

You want to be wary of the practice of leaving old pages, especially if they aren't enhancing the value of the site. Why? Because this will require more bandwidth from search engine bots to crawl your site as you continue to add new product pages to the site. You don't want to risk wasting your crawl allowance having bots crawl pages that are thin in unique content and value. Also, having search engines crawl such pages indicates to them that the site is not "fresh."

Also often times, new products contain the same content as an older variation of the product. For instance, the names of new products may vary only slightly to their previous version and the product description can be a close duplicate. Having all these pages live on the site can result in massive duplicate content issues.

How to Deal with Out-of-Stock Products

If a product is out-of-stock and is expected to be restocked, the page should remain on the site, but an out-of-stock notice should be implemented on the page. However, please bear in mind that out-of-stock pages do tend to generate high bounce rates. To confront high bounce rates issues and improve the overall user experience, consider ensuring that users know similar products are still sold on the site or have users sign-up to be notified when the product becomes available again.

How to Deal with Seasonal Products - at the Category/Sub-Category Level

If a product is seasonal, such as the case for fashion products (example: swimsuits), you might want to leave the page on the site permanently. Why? Because overtime, these pages can retain their link equity year-after-year. If the swimsuit page garnered 3 links this year and 5 links the next, you can continue to accumulate those links. Overtime, you've developed a page that has retained a significant amount of link equity making it much more difficult for competitors to keep up. Thus, giving your site a huge advantage.

And if you don't want the page to be indexed in the off-season, add a meta tag to noindex/follow the page. Users will no longer be able to get to that link from search results (and hopefully from internal results as well), but only through direct links or bookmarks. Once the season starts again, remove the noindex/follow meta tag to an index/follow.

Building Processes/Checklists

Based on the specific needs of your site, it would be helpful to develop a checklist for your technical team. For example, if my site had seasonal products, I would compile a checklist that would include the following:

Remove noindex/follow tag from the [products] page in [month]
Update and resubmit XML site map
Submit this page to "Fetch as Googlebot" in Google Webmaster Tools

Consider creating separate checklists for the steps that you, as an SEO, would need to take to determine which pages to 301 redirect, which ones you need to 404 (if you absolutely need to), and which ones to leave as is. Checklists should also be created to help develop the framework for how your technical team would implement these changes. After awhile, an overall framework should emerge on how your site handles its expired content, which will help make the entire process run much more smoothly.

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!