| T O P I C R E V I E W |
| T00N |
Posted - 30 May 2006 : 15:33:43
In the metatags there used to be the following line for search engines:
<META NAME="robots" CONTENT="INDEX,FOLLOW">
DHE doesn't produce this?
Isn't this line necesasary nowadays? |
| 3 L A T E S T R E P L I E S (Newest First) |
| T00N |
Posted - 30 May 2006 : 19:30:16
Ok thanks, I was just wondering. |
| Martin1 |
Posted - 30 May 2006 : 17:04:31
As Davide says it's not necessary and the example you gave is the default setting. You are better of creating a seperate robots.txt file that contains the settings you want.
For example
User-agent: *
Disallow: /mytestfiles/
Disallow: /images/
Disallow: /cms/
This example shows you that all robots may crawl everything except for the folders named mytestfiles, images and cms.
For more information on robots.txt you can look here:
http://www.robotstxt.org/
For help with creating a robots.txt file you can go here:
http://www.webtoolcentral.com/webmaster/tools/robots_txt_file_generator/
Martin |
| s.dav |
Posted - 30 May 2006 : 16:37:20
I've inserted this line using the "Header Code" property of the page. D.H.E. cannot produce this code for now, maybe in future.
It is not necessary, this should be the default. |
