Does Google Penalise Duplicate Content?

Duplicate Content & Multiple Site Issues

There has been intense debate among Webmasters whether duplicate content on a website would see it penalised by Google - the penalty being that Google would not give a high ranking to the website. There were some experts that felt Google even blacklisted sites with a heap of duplicate content.
My own understanding was that if you changed the content to a certain extent, Google would not penalise the website's rankings. This has now been confirmed by Google engineer, Greg Grothaus. Greg has confirmed that while having exactly the same duplicate content may have a negative impact on your rankings, Google is not penalizing you for it.

Greg says people see messages like the one below and think their content is being omitted from Google's results. In fact, it really is just being omitted for that particular search query. Greg stresses that duplicate content is simply a factor on a "by query" basis.

In order to show you the most relevant results, we have omitted some entries very similar to the 20 you already displayed.
If you like, you can repeat the search with the omitted results included.

"What's actually happening, is that we're looking at the query that the searcher is entering and we're saying that we want diversity in the results that we're going to show a searcher," says Grothaus. He says those who think their content is being omitted because it is duplicate, will likely find that if they adjust their query to more specifically reflect the missing piece, they may just find that it shows up in results after all.

Google recognises that most duplicate content is not created to be deceptive. Most webmasters are creating duplicate content through having the same content under several domain names e.g.

* example.com/
* example.com/?
* example.com/index.html
* example.com/Home.aspx
* www.example.com/
* www.example.com/?
* www.example.com/index.html
* www.example.com/Home.aspx

The above list, taken from Greg Grothaus's presentation, shows examples of URLs that are different, but that have the same content. Google will recognize that they're the same, and will try to pick the right one, (although sometimes they pick the wrong one). Greg says Webmasters are the best people to know which one is best, so it helps to only use one.

You will not be penalized for using more than one but there are some issues that can arise that may negatively affect your rankings. For one, your link popularity will be diluted. Backlinks pointing to several different URL versions of the same content, will make it harder to accumulate link juice for one URL. Greg says that user-unfriendly URLs in search results may offset branding efforts and decrease usability as well. Plus, with multiple versions of the same thing, Google will spend more time crawling the same content, meaning it will have less time to go deeper into your site, and you run the risk of having content not get indexed.

Multiple Domains

Greg Grothaus also discusses multiple domains. This is where you have content for different audiences, such as by country, language, etc. Zeald.com have many clients that replicate their original website when targeting a different country, like UK or Australia. There are some concerns here but they are relatively minor, especially if you take our advice and 'localise' the website. By that we mean loading local addresses and phone numbers; using the spelling variations of that country; producing copy that is appropriate for the intended market.

You do have to consider that your reputation will be distributed across multiple domains and Google will only show what it perceives to be the best page for a particular query.

One interesting factor of this to also consider is that with multiple domains, you are potentially losing the advantage Google's tabbed user interface. You know how sometimes search results are expandable and point you to different links within the site? If your content is spread out across multiple domains, you may be missing extra clicks, because Google can't link to another domain here.

Grothaus explains all of the above and elaborates on each point in the fifteen -minute video that is available here. The information is based on his presentation from the recent Search Engine Strategies conference in San Jose.