Bing(bot) and Google(bot)

[starrxxfoxx]

Since when are these considered registered members? I see them at the bottom next to my name?

[Worker 11811]

Yes, when we did our last system update, a few months ago, we got a new capability treat search engine bots as "fake" users of the board.

 

It allows is to control how bots can access our site and it can keep them out of the parts of the site that we don't want them to see.

 

Here is some of the documentation from phpBB, the makers our our software:

 

3.10.2. Managing Search Robots

 

phpBB 3 introduced a new system for managing search indexer and bot accounts. It allows you to identify these automated bots by their IP or a part of their user-agent, which is a setting that normally identifies the browser of a user. After you add a bot and it is recognized, phpBB does not treat the session as anonymous, but uses the created bot account. Bots use permissions set by the predefined Bots group. Identifying bots is important so that phpBB can serve them content which is more appropriate for search engines - dead links to pages without content are omitted, e.g. posting pages, report post pages etc. Bots never receive a session ID in the URL, which should not appear in the search results. You can also assign a specific style and language to bots.

[fur999]

What to bots do? Is it just Google keeping tabs on what we as a site are doing, or is it every time a search result displays the site?

[Worker 11811]

That's how Google (or other search engines) work.

 

They send out crawlers (or bots) to search the web, automatically recording the web pages they find and categorizing them according to the content on each page. It's like a gigantic card catalog like you used to find in the library.

 

If you want to be able to find information on the internet, Google (et. al) has to index it.

 

FYI: The private parts of our website like the "Fur Freaks" section and the Photo Gallery are inaccessible to the crawlers. You might be able to go on Google, Bing or Yahoo and search for things on The Fur Den but you should NOT be able to find anything that has been posted in Fur Freaks, the Gallery or the private parts of the Wiki, such as the stories section.

 

We lock the bots out of those sections and we use those "fake" user accounts to make it easier to do.

[AKcoyote]

Another point about spiders and crawlers:

Search engine bots will follow any links they encounter. So the more sites that link to a particular site (or specific web page) generally results in a higher ranking of the site or page on the search service.