Sign up below to view device data and get your trial account.

We communicate via email to process your request in line with our privacy policy. Please check the box to give us your permission to do this.

Cancel

Tag archives: web crawlers

bots-and-crawlers

The most active crawlers and bots on the web

Allowing web crawlers to scan your site is vital if you want your web pages to appear in Google, Bing and other search results. Here, we list the most common crawlers alongside their User Agents.

Dark Traffic and Misrepresentation - Analysing the Web Analysers (Part 2)

In part 1 of our Web Analytics Comparison: Analysing the Web Analysers post, we presented research findings of 4 different web analytics services. We noted a significant difference in traffic breakdown across the different solutions; in this post (part 2), we will cover this traffic breakdown discrepancy in more detail.

Website Analytics Comparison - Analysing the Web Analysers (Part 1)

Analysing visitor traffic is important to determine the health of a website. In this article, we present the findings from our research of four different analytics services

Bot Analytics Series Part 2: The Importance of Monitoring Bot Traffic

Analysing your bot traffic can help you understand how your site is being used (for example by preview bots) and lets you verify that it is being correctly indexed by search engines.

Introduction to Bot Traffic — Part One of our Bot Analytics Series

As much as 40% of internet traffic is from non-human sources. This traffic ranges from legitimate bots and crawlers to nefarious automated programs that may be controlled by hackers, fraudsters, or competitors.

bot-traffic-analysis-detection

The most active bots on the web - 2018

For our latest dive into web traffic data, we looked at the numbers for H1 2018 to investigate the most active bots and crawlers on the web.

Device detection

6 not so obvious ways companies can use device detection

With so many devices connected to the internet most companies have understood the need for a reliable way to determine what devices their customers are using. However, not all businesses have a full understanding of what is possible with a device detection solution.

Hot to filter bot and crawler traffic

How DeviceAtlas can protect your site against bots

Non-human traffic is larger than most business owners are aware of. This is mainly due to the fact that Google Analytics doesn't report on bots and crawlers by default. Here's how device detection can help you identify bots to serve them different payloads or even block them from accessing the website.

Bots report

Bots and Crawlers May Represent up to 50% of Web Traffic

Dublin – May 12, 2016 – DeviceAtlas, a leading supplier of intelligence on mobile devices, today released its quarterly Mobile Web Intelligence Report for Q1 2016, which analyzes mobile device traffic to hundreds of thousands of websites around the world.

Web crawlers user agents

List of most active web crawlers User-Agent strings in 2016

To help you understand web crawlers, bots and spiders visiting your site, here's a list of the top 20 bots and their user agents.