Add custom User-Agent header to requests in website monitor script#1171
Conversation
Added user-agent header
WalkthroughAdds a User-Agent header ("NetAlertX") to the HTTP GET request in check_services_health within front/plugins/website_monitor/script.py. No other logic, parameters (verify=False, timeout), error handling, latency measurement, or return values are changed. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~6 minutes Pre-merge checks (2 passed, 1 warning)❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
Poem
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
front/plugins/website_monitor/script.py (2)
60-60: Make UA configurable and more descriptive (include plugin/name/version)Let admins override the UA and include the plugin name for clarity.
Apply within this hunk:
- resp = requests.get(site, verify=False, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'), headers={"User-Agent": "NetAlertX"}) + resp = requests.get(site, verify=False, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'), headers=DEFAULT_HEADERS)Then add near other globals (outside this hunk):
# Default and configurable User-Agent for Website Monitor USER_AGENT = (get_setting_value('WEBMON_USER_AGENT') or 'NetAlertX WebsiteMonitor').strip() DEFAULT_HEADERS = {'User-Agent': USER_AGENT}Want me to open a small follow-up PR to wire the WEBMON_USER_AGENT setting into the UI/docs?
53-61: Minor: reuse a Session and set headers onceFor multiple URLs, a shared requests.Session with default headers can reduce overhead and centralize UA.
Example (outside this hunk):
SESSION = requests.Session() SESSION.headers.update(DEFAULT_HEADERS) # then: resp = SESSION.get(site, verify=verify_tls, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'))
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
front/plugins/website_monitor/script.py(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
front/plugins/website_monitor/script.py (1)
server/helper.py (1)
get_setting_value(358-411)
🪛 Ruff (0.12.2)
front/plugins/website_monitor/script.py
60-60: Probable use of requests call with verify=False disabling SSL certificate checks
(S501)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: docker_dev
🔇 Additional comments (1)
front/plugins/website_monitor/script.py (1)
60-60: Custom User-Agent addition: LGTMUsing a non-default UA helps avoid blocks and improves log attribution. No behavior change otherwise.
| requests.packages.urllib3.disable_warnings(InsecureRequestWarning) | ||
|
|
||
| try: | ||
| resp = requests.get(site, verify=False, timeout=get_setting_value('WEBMON_RUN_TIMEOUT')) | ||
| resp = requests.get(site, verify=False, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'), headers={"User-Agent": "NetAlertX"}) |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid verify=False; make TLS verification configurable and default to True
Disabling cert verification (S501) is a security risk. Keep verification on by default and only suppress warnings when explicitly disabled.
Change the call in this hunk:
- resp = requests.get(site, verify=False, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'), headers={"User-Agent": "NetAlertX"})
+ resp = requests.get(site, verify=verify_tls, timeout=get_setting_value('WEBMON_RUN_TIMEOUT'), headers={"User-Agent": "NetAlertX"})Add just above the request (outside this hunk):
verify_tls = get_setting_value('WEBMON_VERIFY_TLS')
if verify_tls in ('', None):
verify_tls = True
if not verify_tls:
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)If adding a new setting isn’t desired, at minimum consider setting verify=True and allowing per-URL exceptions.
🧰 Tools
🪛 Ruff (0.12.2)
60-60: Probable use of requests call with verify=False disabling SSL certificate checks
(S501)
🤖 Prompt for AI Agents
In front/plugins/website_monitor/script.py around lines 57 to 60, the
requests.get call currently uses verify=False which disables TLS verification;
change this to make TLS verification configurable and default to True by reading
a setting (e.g. get_setting_value('WEBMON_VERIFY_TLS')), treat empty/None as
True, store the boolean in a verify_tls variable, call
requests.packages.urllib3.disable_warnings(InsecureRequestWarning) only if
verify_tls is False, and pass verify=verify_tls into requests.get so
verification is enabled by default and only suppressed when explicitly
configured.
|
Thansk @cvc90 🙏 |
2 participants
Hi @jokob-sk,
Pull Request
Description
Problem:
By default, the requests library identifies itself with a User-Agent header of "python-requests/version". Many modern security systems (such as web application firewalls, intrusion prevention systems, or rate limiters) are configured to block, rate-limit, or challenge traffic based on this default user agent, interpreting it as suspicious or malicious.Solution:
This change adds a custom User-Agent string ("NetAlertX") to the HTTP requests made by the script website_monitor/script.py.What this change means:
Requests sent by the Website Monitor plugin will now be less likely to be blocked by firewalls or intrusion prevention systems.
NetAlertX traffic will be easier to identify and distinguish in server logs, since it carries its own unique user agent instead of the generic "python-requests".
This provides a more reliable monitoring experience, especially in environments with stricter security policies.
Changes
Update Plugin Website Monitor (/front/plugins/website_monitor/script.py)
Updated the requests.get(...) call to include:
headers={"User-Agent": "NetAlertX"}Ensuring all outgoing HTTP requests carry the custom identifier.
Test
Run local tests and verified that the User-Agent header is set correctly in outbound HTTP requests.
Confirmed that monitoring continues to operate as expected, without interference from security filters that previously flagged the "python-requests" user-agent.
Summary by CodeRabbit