Duplicate Content & Multiple Site Issues #SESSJ

I will be attending the Geek Speak track today, which begins with a session centered around duplicate content. If you run mirror sites, will search engines ban you? If you have listings that are similar in nature, is that an issue? What happens if you syndicate content through RSS and feeds? Will other sites be considered the “real” site and rob you of a rightful place in the search results? This session looks at the issues and explores solutions.

Moderator:
PJ Fusco,Natural Search Director, Netconcepts

Speakers:
Shari Thurow,Founder & SEO Director, Omni Marketing Interactive
Greg Grothaus, Search Quality Team, Google
Marty Weintraub, President, aimClear
Sasi Parthasarathy, Program Manager, Bing
Ivan Davtchev, Lead Product Manager, Search Relevance, Yahoo! Search

Coverage: Tom Dressler, Senior Strategist, WebsiteBiz

Shari Thurow, Founder & SEO Director, Omni Marketing Interactive

Search Engine Filters

• Be aware of boilerplate links or templates when search engines determine a content fingerprint it focuses on the unique content only.
• Host Name Resolution – Who has control of the content?
• Shingle Comparison
  • Every web document has a document fingerprint
  • The more the shingles are the same, the search engine will post only what they think are the most relevant
  • Areas to review: Use of Boiler Plate, Host Resolution, Links…Content awareness
  • Solutions
    • Prefers robots exclusion – www.robotstext.org
    • Page Level exclusion
    • See PowerPoint (I will post as soon as available) for details on helpful URLs
    • The no follow tag is really a quick fix for poor site architecture

Marty Weintraub, President, aimClear

Canonicalization, 301’s and proof that it works

• Case Study Format
• Issues
  • Duplicate homepages indexed in SERPs
  • HTTP/HTTPS problems in SERPs
  • Secure Certificate Errors
  • Solutions
    • Custom Redirection Grid
    • Redirect Spreadsheet
    • Redirect to clean: versions of Interior pages
    • Clean page ends with a folder name /
    • Check the www Option on Google Webmaster Tools
    • Redirect all domains to “mother” domain
    • The unfortunate “You Suck” methodology
    • Cross reference the googlebot activity to search engine traffic
    • Canonicalization Facts
      • A clean URL is a Happy URL
      • www vs. non www
      • HTTPS vs. HTTP
      • Trailing slashes
      • The development server is indexing – To check search by the IP address
      • 301 is the only solution.
      • All activities occurred with in a 1 month period so it is measurable
      • Conduct Mini Audits

Sasi Parthasarathy, Program Manager, Bing

• The end user is the number 1 focus
• If there are spammers in your space please report them to Bing
• Geotargeting similar content is an issue
  • Please help Bing to understand what we want them to do
  • Use top-level domain names!
  • Content syndication
    • Ask your partners to use robots.txt to stop the Search Engine from
    • Don’t use dynamic URLs if you have static content
    • Follow a standard URL formation best practices
    • Canonical Tags
      • Not being actively used by webmasters
      • 50% us them incorrectly
        • 38% point to the same page
        • You cannot use across domains
        • Use only if content is similar
        • Use them as hints
        • The Search Engines will decide how to use the hint
        • Use with Caution
• Bing is looking to improve effectiveness
• Use the 301 as your best friend

Greg Grothaus, Search Quality Team, Google

• There is a myth about a duplicate content penalty
  • The omitted link is not a penalty
  • See duplicate content guidelines to clarify (Search: duplicate content guidelines)
  • Google try’s to show a variety on certain results to align with the user experience
  • New Options for Duplication Content Control
    • rel=canonical
    • Works on sub-domains not “other” domains
    • Both 301 and rel=canonical work the same
    • Multiple Domains
      • Okay especially in countries and languages
      • Content is similar Google will pick the most relevant to display
      • See Google.com/webmasters if you have additional questions or would like to learn more

Ivan Davtchev, Lead Product Manager, Search Relevance, Yahoo! Search

• 30% of the web is duplicate contents
• The search engines feel this is a wasted effort and would like to not have this issue
• Search Engines want SERPs to be diverse to create a great user experience
• Show the most relevant page based on the query
• Query time is taken into account on display impact
• Yahoo uses shingles; duplication does not have to be exact based on this method
• Some duplicate content is okay especially if you are a large publisher
• Aggressive duplication across domains is not recommended
• Remixing content can be detected through shingling algorithm

Tools & Links:

• Use Yahoo site explorer to inform the search engines what is best for your situation
  • Siteexplorer.search.yahoo.com
    • Dynamic URL Rewriting capability
• Ysearchblog.com

The session is now open to Q&A

How do you determine if this is a issue?

• Use site query URL to see if there are pages not expected
• Look into GWT

What about partner sites; How my “sony headphone” page be the primary choice?

• Google will pick 1 result based on what the SE sees as original & best source of content.
  • Add a footer stating the syndication root.
  • Add something specific content to make your syndicated content unique.
  • Under partner environments it’s okay and that the “local” algorithm will filter.
  • Make your site easy to use to help you be the “primary” solution for selection by the search engine.

What is the best way to redirect multiple domains to your “mother” domain?

• Use the 301 redirect as a permanent redirect

A plea to Bing: PLEASE keep Site Explorer as Shari indicates it is the best tool available today!

Tagged as: canonicalization, Duplicate Content, SEO, site architecture