About our crawler
- Name in your logs:
IntelligenceOne- Full user agent:
IntelligenceOne/0.1 (+https://intelligenceone.co.uk/crawler)- Follows robots.txt:
- yes
- To stop it:
- add the robots.txt rule below, or email crawler@intelligenceone.co.uk
What it is
INTELLIGENCE ONE helps businesses find and contact other UK businesses. Our crawler visits company websites to confirm which website belongs to which registered company, and to find that company's general business contact details.
What it reads
- Only public pages. It never logs in, fills in forms or gets past a paywall.
- A small number of pages per site: at most ten pages, such as the home, contact and about pages, and only on that website's own domain.
- One request at a time, at least five seconds apart, and at most about once every 90 days per site unless we are asked to check sooner.
- It checks whether an email domain can receive email by looking up its public mail (MX) records. It never sends an email.
What it keeps
- Business contact details for the company itself: general email addresses at the company's own domain (such as info@ or sales@), switchboard numbers, the business address, the VAT number, the company number and the company's own social media pages.
- A short excerpt (at most 60 characters) showing where a company number or name was found, so the match can be checked.
- It does not keep personal details. Email addresses that name a person, mobile numbers and personal social media profiles are counted and then discarded, never stored.
How to stop it visiting your website
Add this to the robots.txt file at the root of your website. Our crawler reads your robots.txt and follows it.
User-agent: IntelligenceOne
Disallow: /To limit it to certain parts of your site, list only those paths under Disallow.
Ask us to stop or remove your details
Email crawler@intelligenceone.co.uk with your website address. We'll stop crawling it and deal with any request about the details we hold from it. You can also use our contact form.
Contact usRelated
Last updated: 22 November 2026

