We all know search engine optimization is a tricky business. Sometimes we rank well on one engine for a particular keyphrase and assume that all search engines will like our pages, and hence we will rank well for that keyphrase on a number of engines. Unfortunately this is rarely the case. All the major search engines differ somewhat, so what's get you ranked high on one engine may actually help to lower your ranking on another engine.
It is for this reason that some people like to optimize pages for each particular search engine. Usually these pages would only be slightly different but this slight difference could make all the difference when it comes to ranking high.
However because search engine spiders crawl through sites indexing every page it can find, it might come across your search engine specific optimizes pages and because they are very similar, the spider may think you are spamming it and will do one of two things, ban your site altogether or severely punish you in the form of lower rankings.
So what can you do to say stop Google indexing pages that are meant for AltaVista, well the solution is really quite simple and I'm surprised that more webmaster's who do optimize for each search engine don't use it more. It's done using a robots.txt file which resides on your webspace.
A Robots.txt file is a vital part of any webmasters battle against getting banned or punished by the search engines if he or she designs different pages for different search engine's.
The robots.txt file is just a simple text file as the file extension suggests. It's created using a simple text editor like notepad or wordpad, complicated word processors such as Microsoft Word will only corrupt the file.
Here's the code you need to insert into the file to work:
Red text is compulsory and never changes, while the blue text you will have to change to suit the file and the engine which you want to avoid it.
User-Agent: (Spider Name)
Disallow: (File Name)
The User-Agent is the name of the search engines spider and Disallow is the name of the file that you don't want that spider to spider. I'm not entirely sure if the code is case sensitive or not (maybe someone can let me know) but I do know that the code above works, so to be sure to check that the U and A are in caps and likewise the D in disallow.
You have to start a new batch of code for each engine, but if you want to list multiply disallow files you can one under another. For example -
User-Agent: Slurp (Inktomi's spider)
Disallow: internet-marketing-gg.html
Disallow: internet-marketing-al.html
Disallow: advertising-secrets-gg.html
Disallow: advertising-secrets-al.html
In the above code, I have disallowed Inktomi to spider two pages optimized for Google (internet-marketing-gg.html & advertising-secrets-gg.html) and two pages optimized for Altavista (internet-marketing-al.html & advertising-secrets-al.html). If Inktomi were allowed to spider these pages as well as the pages specifically made for Inktomi, I run the risk of being banned or penalized, So it's always a good idea to use a robots.txt file.
I mentioned earlier that the robots.txt file resides on your webspace, but where on your webspace? The root directory that's where, if you upload your file to sub-directories it will not work. If you want to block certain engines from certain files that do not reside in your root directory you simply need to point to the right directory and then list the file as normal, For example -
User-Agent: Slurp (Inktomi's spider)
Disallow: folder/internet-marketing-gg.html
Disallow: folder/internet-marketing-al.html
If you wanted to disallow all engines from indexing a file, you simply use the * character where the engines name would usually be. However beaware that the * character won't work on the Disallow line.
Here's the names of a few of the big engines,
Excite - ArchitextSpider
Altavista - Scooter
Lycos - Lycos_Spider_(T-Rex)
Google - Googlebot
Alltheweb - FAST-WebCrawler/
Be sure to check over the file before uploading it, as you may have made a simple mistake, which could mean your pages are indexed by engines you don't want to index them, or even worse none of your pages might not be indexed.
A little note before I go, I have listed the User-Agent names of a few of the big search engines, but in reality, it's not worth creating different pages for more than 6-7 search engines. It's very time consuming and results would be similar to those if you created different pages for the only the top five. So more is not always best.
So now you know how to make a robots.txt file, to stop you from getting banned by the search engines. Wasn't that easy. Till next time!
Article by David Callan - admin@akamarketing.com
David is the webmaster of http://www.akamarketing.com.
Visit his site for free internet marketing articles, advice, ebooks, news and lots more.
Friday, January 11, 2008
Monday, January 7, 2008
7 Essential Steps to Web Site Promotion
Copyright 2003 by Herman Drost
Web Site promotion should be the major part of your web site marketing plan. It's not enough just to design a beautiful web site and put in on the Net. Promoting your web site has to be done constantly if you want to get a continuous stream of traffic to it. If you don't drive traffic to your web site, your online business venture will soon fail.
1. Create a web site marketing plan - if you fail to plan then you default to a plan for failure. When designing your web site, create a plan on how to market it as well. Try to put yourself in your customers shoes when doing this. It's too easy to just view your web site from your own perspective. You need to have a clear vision for your business, goals (short and long term) you wish to achieve and the strategies of how you will accomplish them.
2. Define your target market - many online marketers promote their web site aimlessly in a field which is too large i.e. they may use bulk mailing as one of their promotion strategies. Of course you may get some visitors but the chances are you will not make many sales. Why? You have not narrowed your field of customers and targeted them. If you sold Cadillacs online for instance, you are not randomly going to market to every person on the Net. Your target market may be previous owners of cadillacs, or people who routinely purchase luxury American cars.
Getting targeted customers to your web site, will increase sales, because they will have more interest in your product or service than marketing to a general audience.
3. Use several promotion strategies - don't just stick to one form of promotion because you never know which one will work the best for you. If you use many channels i.e. search engines, email marketing, forums, writing articles etc you have a greater chance of getting more traffic to your web site.
4. Promote continuously - if you want to drive constant traffic to your web site, you must promote it continuously. Web traffic is the lifeblood of your online business. For example, you could constantly optimize your web pages for the search engines, resubmit them if needed, visit discussion forums every day and write an article for online publications every two weeks.
5. Calculate your return on investment (ROI) - this means keeping track of how much money you invested in your marketing plan and what profit you received in return. This will avoid wasting large sums of money thrown aimlessly into advertising.
If you calculate it takes 100 visitors to your web site to generate one sale and with each sale you make $10, then you shouldn't spend more than 10 cents to get each targeted visitor and still make a profit.
In order to know which aspects of your marketing campaign bring the most visitors, you must be able to monitor visitor activity to your web pages.
6. Analyze your web site traffic - by analyzing your web site traffic, you can determine what marketing strategies are successful. You can then change them as necessary, to boost the sales or services from your site.
Your analysis may include what keywords your visitors are using to find your web site, which web pages are the most popular and where your visitors are coming from. These are essential statistics you should know, instead of blindly promoting your web site and hoping for the best.
7. Test, monitor and track your results - to run an effective online marketing campaign, you need to constantly test what works and what doesn't. Tracking the response to an ad may cause you to refine it until it generates a great return on the money you invested. Once your results are satisfactory, you can then roll out your ads on a larger scale and spend greater sums of money.
You should also monitor your site in the search engines, so you can optimize your web pages for higher search engine rankings. This will attract more visitors which leads to higher profits.
Web site promotion strategies will differ for everyone, and will depend a lot on your product, web site and your personality. Try everything you can, record the results, and see what works for you.
Herman Drost is the author of the new ebook "101 Highly Effective Strategies to Promote Your Web Site" a powerful guide for attracting 1000s of visitors to your web site. http://www.isitebuild.com/web-site-promotion
Web Site promotion should be the major part of your web site marketing plan. It's not enough just to design a beautiful web site and put in on the Net. Promoting your web site has to be done constantly if you want to get a continuous stream of traffic to it. If you don't drive traffic to your web site, your online business venture will soon fail.
1. Create a web site marketing plan - if you fail to plan then you default to a plan for failure. When designing your web site, create a plan on how to market it as well. Try to put yourself in your customers shoes when doing this. It's too easy to just view your web site from your own perspective. You need to have a clear vision for your business, goals (short and long term) you wish to achieve and the strategies of how you will accomplish them.
2. Define your target market - many online marketers promote their web site aimlessly in a field which is too large i.e. they may use bulk mailing as one of their promotion strategies. Of course you may get some visitors but the chances are you will not make many sales. Why? You have not narrowed your field of customers and targeted them. If you sold Cadillacs online for instance, you are not randomly going to market to every person on the Net. Your target market may be previous owners of cadillacs, or people who routinely purchase luxury American cars.
Getting targeted customers to your web site, will increase sales, because they will have more interest in your product or service than marketing to a general audience.
3. Use several promotion strategies - don't just stick to one form of promotion because you never know which one will work the best for you. If you use many channels i.e. search engines, email marketing, forums, writing articles etc you have a greater chance of getting more traffic to your web site.
4. Promote continuously - if you want to drive constant traffic to your web site, you must promote it continuously. Web traffic is the lifeblood of your online business. For example, you could constantly optimize your web pages for the search engines, resubmit them if needed, visit discussion forums every day and write an article for online publications every two weeks.
5. Calculate your return on investment (ROI) - this means keeping track of how much money you invested in your marketing plan and what profit you received in return. This will avoid wasting large sums of money thrown aimlessly into advertising.
If you calculate it takes 100 visitors to your web site to generate one sale and with each sale you make $10, then you shouldn't spend more than 10 cents to get each targeted visitor and still make a profit.
In order to know which aspects of your marketing campaign bring the most visitors, you must be able to monitor visitor activity to your web pages.
6. Analyze your web site traffic - by analyzing your web site traffic, you can determine what marketing strategies are successful. You can then change them as necessary, to boost the sales or services from your site.
Your analysis may include what keywords your visitors are using to find your web site, which web pages are the most popular and where your visitors are coming from. These are essential statistics you should know, instead of blindly promoting your web site and hoping for the best.
7. Test, monitor and track your results - to run an effective online marketing campaign, you need to constantly test what works and what doesn't. Tracking the response to an ad may cause you to refine it until it generates a great return on the money you invested. Once your results are satisfactory, you can then roll out your ads on a larger scale and spend greater sums of money.
You should also monitor your site in the search engines, so you can optimize your web pages for higher search engine rankings. This will attract more visitors which leads to higher profits.
Web site promotion strategies will differ for everyone, and will depend a lot on your product, web site and your personality. Try everything you can, record the results, and see what works for you.
Herman Drost is the author of the new ebook "101 Highly Effective Strategies to Promote Your Web Site" a powerful guide for attracting 1000s of visitors to your web site. http://www.isitebuild.com/web-site-promotion
Subscribe to:
Comments (Atom)
