Spiders / Bots

Started by limitzero, September 12, 2012, 08:33:00 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

limitzero

Hey

So I was wondering if it is a built in feature or additional mod we have to install to show the Spiders / Bots who are 'online'  I have noticed on some forums like Logooff that it would say 'Google Adsense' or 'Baidu' on the Users Online list. Any info regarding this will be helpful.

Thanks

Skhilled

It's easy. Go to Admin -> Core Features and enable "Search Engine Tracking". Then go to Admin -> Members -> Search Engines -> Settings. There you can set how you want to see and log the them. You will only see the Search Engine menu if you enable it first.

The other menu items will give you stats and logs on what you have chosen. This is what I have here and what Rock has at logooff. You can also have it show to admin only like I have it here.

limitzero

Thanks! I couldn't find this on SMF 1.1.x , but have found it on a dummy 2.x I have installed, great feature.

Skhilled

Ah! I forget that you have SMF 1.1.x. That is only on SMF 2.x versions.

beast



  Is there any speacil reason for wanting to see them?

:dontknow:
Thanks [you] for reading my post

Skhilled

to learn how they are crawling your site. You will want them to see the important things about your site. But there may be things that you do not want them to see and you can block it or allow it via a robots.txt file or a .htaccess file.

This way when you look up your site in a search engine the things you want people to see will be seen and the things you don't will not.