Sign up below to view device data and get your trial account.

We communicate via email to process your request in line with our privacy policy. Please check the box to give us your permission to do this.

Cancel

Tag archives: crawlers list

bots-and-crawlers

The most active crawlers and bots on the web

Allowing web crawlers to scan your site is vital if you want your web pages to appear in Google, Bing and other search results. Here, we list the most common crawlers alongside their User Agents.

bot-traffic-analysis-detection

The most active bots on the web - 2018

For our latest dive into web traffic data, we looked at the numbers for H1 2018 to investigate the most active bots and crawlers on the web.

Web crawlers user agents

List of most active web crawlers User-Agent strings in 2016

To help you understand web crawlers, bots and spiders visiting your site, here's a list of the top 20 bots and their user agents.