BigCommerce SEO Basics: Block everything before doing anything!

To start off a series of articles on implementing some basic BigCommerce SEO techniques I’m going to recommend something that may seem a little counter intuitive.

The very first thing you MUST do after signing up for a BigCommerce account is block your entire site from the search engines.

Why the hell would you want to block the search engines?

While it is very nice that the people at BigCommerce provide every account holder with a free BigCommerce subdomain (yourname.BigCommerce.com), the truth of the matter is that most people never use their free subdomain once their online store is ready for public consumption.

Most BigCommerce store owners will (and should) ultimately end up using their own domain name (yourname.com), or a subdomain (shop.yourname.com) of their domain name to operate their online store once it is live and ready to use.

While it isn’t common for BigCommerce store owners to use their free subdomain  when their site is ready to go live, I would anticipate that it isn’t uncommon for them to use it in the early stages while they are first setting up their store or evaluating a free trial.

This is commonsense, there really is no point going that extra mile to set up your domain name until you’re certain that you will be remaining with the provider.

There is no problem with that.. right! Well… actually there is…

The Hidden Problem of the Free BigCommerce Subdomain

The problem with using the subdomain while you are setting up is that Google has become so bloody quick to discover and index new websites that before you’re even are aware of what’s going on you have pages starting to pop up in Google’s search results, all directed to your BigCommerce subdomain.

As you continue to add products, categories and information pages Google will continue to pick them up and add them to the search index, all unbeknown to you at this early stage of your shops development.

Ultimately all your hard work setting up will culminate to the point where you’re ready to point your domain name at your shiny new online store and start letting the world know about it.

You follow all the instructions and everything seems to be working perfectly, how could it have gone wrong!

Here Comes the Duplicate Content

After a short while a few pages from your site will start showing up in the search index and to help speed up the process you do the right thing and submit your sitemap to Google Webmaster Tools (more on that in a later lesson).

Everything is going along sweetly, or at least it is on the surface. Ticking away behind-the-scenes there is a hidden problem that you are totally unaware, one that continues to grow bigger by the day.

With your total consent and knowledge, you have happily let Google access your site and index your domain name, but behind-the-scenes, technically also with your consent, but this time without your knowledge Google has also continued to index your free BigCommerce subdomain.

With what is very much an innocent mistake you have allowed Google to place two exact replicas of your website in their search index.

This is what is known around SEO circles as a “duplicate content nightmare“.

What the #@!$% is Duplicate Content?

If you don’t have any SEO experience you’re probably wondering what the big fuss about having two exact replicas floating around Google is. Without going into the technicalities something that you need to learn very quickly is:

  1. Google only wants one version of your site, and
  2. Google (in fact any search engine) isn’t as smart as many think and has a hell of a lot of trouble initially determining which version they should keep and which version they should toss, ultimately favouring neither version.

Rule 1: avoid duplicate content like the plague!

So How Do You Prevent Creating This Duplicate Content Mess?

I am glad that you asked, because preventing this duplicate content mess is actually very easy… if you know beforehand.

Fortunately BigCommerce allows you to make changes to a very important little file called robots.txt. You can edit this file directly through your BigCommerce store administration panel by going to (Tools >> Edit Robots.txt File).

BigCommerce robots.txt file

First thing that you will want to do is make a backup copy of all the information that currently is inside the robots.txt file and paste it into a simple text document. You’re going to need this when you’re ready to launch your live site if the “Revert to Default” button doesn’t work for some reason (never hurts to make a backup).

BigCommerce robots.txt file

Once you have your backup copy safely saved away replace the information in the robots.txt with the following:

User-agent: *
Disallow: /

This instruction will tell any search engine that adheres to the robots.txt protocol (which any legitimate company will do) not to enter the site.

Disallowing the search engines from entering the site will prevent them from indexing your temporary subdomain… helping avoid duplicate content problems when you’re store is ready to go live.

You just need to remember to reinstate your backup copy when you’re ready to launch your real site.

What should you do next!

If you found this article helpful and want to learn more about optimising your BigCommerce store you’re in luck. I am currently in the middle of putting together a free short training course to help BigCommerce store owners get their head around using of the basic SEO tools that come with your BigCommerce account.

To obtain the free short course make sure you put your first name and e-mail address into the signup form, and I will notify you when the BigCommerce SEO training is released.

Anyone with questions, please feel free to us and below and I will answer them ASAP.

Comments

  1. I also go to Store Design > Browse Template Files and add the following line to HTMLHead.html on line 3 (it’s under Panels if you can’t find it)

    It ensures that not only with the engine no crawl your website they will not index it either. Just make sure you remove it before going live ;)

    • thanks Andrew – you can’t be too thorough when it comes to having the wrong site indexed.

      Unfortunately WordPress has removed the code that you recommended but I assume it would be the following:

      <meta name="robots" content="noindex, nofollow">
      

      Dave

  2. Hello Dave, first of all, amazing site you have here, thank you. We are having an issue with our bigcommerces store, the bots are trying to index search queries, duplicating products per currency available, in our case 3, and also category pagination. How can we avoid the bots to keep trying to index these types of pages? Is there a plug-in like the one available for interspire? Thank you!

    J

    • Hi Jenny

      unfortunately this is the dilemma that you face when using a hosted e-commerce platform such as BigCommerce. The beauty of a self hosted shopping cart like Interspire shopping cart, is that you are able to access the source code and make any necessary changes that facilitate your store (or create a third-party module such as my Interspire SEO module).

      In regard to what you can do with BigCommerce I suggest that you make sure that your robots.txt is blocking the pages that you do not want indexed and also make sure that your Google Webmaster account is set up to ignore certain parameters.

      • Thank you Dave, would you be so kind to guide me to a place where I can learn how to setup the webmaster parameters? We have thousands of pages and to block all the dups in robot file will be almost impossible, perhaps with the webmaster parameters it will be the way to fix it, as everyone here, we really appreciate your help and wonderful mountain of information for all ecommerce owners, thank you

        J

  3. Eeek Dave! thanks for this great article, I just wish I had found it sooner… I am in the process of moving my site from a different platform, and have imported a few products into my BigCommerce store, which I have been setting up over the past 1 1/2 months.
    I have not gone ‘live’ with the BC site yet.

    However I suspect that the damage has already been done, is there anything I can do to remedy the situation?

    • Hi Gill – I wouldn’t worry about what has already happened, just go ahead and apply the advice above ASAP and you should be okay.

      Just remember to remove the code when you’re ready to go live.

  4. This information would have been helpful awhile ago. So what is the solution when you have been running your store for awhile? My traffic just tanked.

Speak Your Mind

*

css.php