While registering an account, you'll have to enter the email address. The password which is also the verification code will be sent to your inbox. You should change this password from the Website Crawler's settings page. In addition to the email ID, we collect the below details of the users of our platform:
The details displayed in the above list are saved to the log file by the web server. We reffer this file to monitor/fix errors, monitor the security, and improve the performance of the application.
When users run a crawl, our platform collects various information available on the website's page/pages for the purpose of generating reports or offering data in structured format (JSON or CSV). This information is stored as long as the user is using our platform. The data is exclusive to the crawl job starter only (the person who runs the crawl). The crawled data will be assigned to the user only if he/she registers an account after starting the crawl job or after signing in and running the crawl job. If the user doesn't sign in, the data is of no use for our platform and will be automatically deleted from our database. Like our platform, our API endpoints don't reveal data of users who is not the assigned the website data.
The email address of the users is used for sending alerts (crawl completion, downtimes, ssl expiry, etc) and for identification of the user. We don't sell or share your data with third-party companies. We might also send you emails when we release new features.
We use Google Analytics to track the user to gather user activity on a website. GA uses JavaScript to track the activities of the users. It gives us access to data which our webserver doesn't.
Our application supports integration with Google Search Console. To use this feature, you must allow WebsiteCrawler to access your Google Search Console account. We will have read only access to your account. Once you connect WebsiteCrawler to your GSC account, website crawler will sync the search console data to our database and you will be able to see the reports of the same in the project section of your dashboard. You can revoke the access anytime. Once you revoke the permission, our platform won't have access to your GSC account. We do not use your website's data for advertising, or resale. The data will deleted once you delete your WebsiteCrawler account.
Whether you've registered your account using your email address or signed in using Google, the settings page of WebsiteCrawler has an option to delete the account. Once you delete your account, your information will be immediately removed from our server.
As of now, we're using Paddle - one of the best payment service provider. Paddle is a secure and fast service that is available in over 100 countries.
The content available on WebsiteCrawler is for the consumption/reference of the user only. It must not be reproduced/copied anywhere/elsewhere without our permission. The use of our platform (WebsiteCrawler.org), plugins, SDKs, and our publicly available endpoints is at the own discretion of the user.
For further questions, please drop an email to [email protected]