Skip to main content

New “Bingbot” to crawl through Non-optimized Sites more easily

New “Bingbot” to crawl through Non-optimized Sites more easily
Bing's New Web crawler Bingbot releasing coming October.

Microsoft made an announcement that it will be bringing the Bing web crawler out of beta on October 1st. It will be re branded as "the Bingbot" and will replace the existing msnbot. "It will still honor robots.txt directives written for msnbot, so no change is required to robots.txt file(s)," a Bing representative told WebProNews.

"Improvements to the bot enables more efficient crawling, and increases the ability to crawl content on sites not optimized for search," he says.

Rick DeJarnette has more to say about the change on the Bing Webmaster Blog:

Instead of the old msnbot 2.0 showing up in your server logs, the updated user agent will be:

Mozilla/5.0 (compatible; bingbot/2.0 +

The HTTP header from field will also change as shown below:

From: msnbot(at) will become

From: bingbot(at)

The company says; if Bing finds different sets of directives for Bingbot and for other crawlers, directives for bingbot will take precedence.

Also don't forget that optimizing for Bing is growing in importance. Not only is Facebook giving good reasons for people to search (where Bing provides the web results), but the Yahoo/Bing integration is expected (likely) before the holidays.