Business leads
What happens on every run
- Crawl politelyChecks robots.txt, waits between requests, retries with back-off and sends a clear user agent.
- Follow the structureCategory pages, then pagination, then each business page, collecting every listing once.
- CleanPhone numbers in any format become E.164; addresses split into street, city, state and ZIP.
- De-duplicateThe same business listed twice is merged by phone number and name, or name and ZIP when a copy has no phone, keeping the most complete record and all categories.
- Find contactsVisits each business's own website (home, contact and team pages) for e-mail addresses, LinkedIn, Facebook, Instagram and X, decoding "info [at] shop [dot] com" and skipping look-alikes such as image names and template placeholders.
- Check and scoreEach e-mail is marked as a named person, a shared inbox, free mail, disposable or invalid; each lead gets a 0–100 score for how reachable and established it is.
- Track changesCompares with yesterday's run: businesses that opened, closed or changed their phone, address, e-mail or rating.
- ExportCSV, a formatted Excel workbook, JSON and a CRM import file, published here by a scheduled GitHub Actions job.
The business directory is a demo site with invented businesses and their own small websites (phone numbers in the fictional 555-01xx range, domains under the reserved .example), built for this showcase; a few businesses open, close or change every day. The books dataset comes from books.toscrape.com, a public sandbox made for scraping practice.