About Our Crawler
Our crawler identifies itself as BookieNearMeBot. This page
exists so anyone who sees it in their logs can find out what it is and stop
it in one step.
How It Behaves
- One request at a time, no more often than once every 2.5 seconds.
- Reads your
robots.txtand your sitemap first. - Honours
Retry-After, and backs off on 429 and 503. - Stops completely on a 403. We treat that as you withdrawing access, and we do not retry, rotate addresses or work around it.
- Requests only ordinary public pages. No logins, no private APIs, no query strings where robots.txt disallows them, no bot challenges solved.
- Fetches each page once and caches the result, so a re-run costs you nothing.
What It Collects
Branch opening hours, address, and phone number from public shop-locator pages — so we can tell people whether a shop is open. We do not copy your page text, descriptions, photographs or logos, and we do not present ourselves as an official locator for any operator.
How to Stop It
Add this to your robots.txt and we will stop on our next run:
User-agent: BookieNearMeBot
Disallow: /Or email privacy@bookienearme.co.uk and we will remove the data as well as stop crawling. We keep every source separable precisely so that a removal request can be honoured completely rather than approximately.
Would You Rather Send Us the Data?
We would prefer that. A feed is more accurate than anything we can read off a page, and it lets us show bank-holiday and event-day hours, which a locator usually does not publish. What we need is here.