Tag archives: web crawlers

bot-traffic-analysis-detection

The most active bots on the web - 2018

For our latest dive into web traffic data, we looked at the numbers for H1 2018 to investigate the most active bots and crawlers on the web.

Device detection

6 not so obvious ways companies can use device detection

With so many devices connected to the internet most companies have understood the need for a reliable way to determine what devices their customers are using. However, not all businesses have a full understanding of what is possible with a device detection solution.

Hot to filter bot and crawler traffic

How DeviceAtlas can protect your site against bots

Non-human traffic is larger than most business owners are aware of. This is mainly due to the fact that Google Analytics doesn't report on bots and crawlers by default. Here's how device detection can help you identify bots to serve them different payloads or even block them from accessing the website.

Web crawlers user agents

List of most active web crawlers User-Agent strings

To help you understand web crawlers, bots and spiders visiting your site, here's a list of the top 20 bots and their user agents.

Bots report

Bots and Crawlers May Represent up to 50% of Web Traffic

Dublin – May 12, 2016 – DeviceAtlas, a leading supplier of intelligence on mobile devices, today released its quarterly Mobile Web Intelligence Report for Q1 2016, which analyzes mobile device traffic to hundreds of thousands of websites around the world.