High CPU and RAM usage during scan #203
Description
Describe the issue
Every scan, Pi.Alert hogs a full core and memory usage climbs to several gigabytes before the process is killed by the OOM reaper.
Moreover, the pialert_pholus.log has grown to 1.6 GiB large...
Paste last few lines from pialert.log
Permissions check (All should be True)
------------------------------------------------
/config/pialert.conf | READ | True
/config/pialert.conf | WRITE | True
/db/pialert.db | READ | True
/db/pialert.db | WRITE | True
------------------------------------------------
[Setup] Attempting to fix permissions.
[Setup] Attempting to fix permissions.
[LOG_LEVEL=debug] 14:37:15 Opening DB
[upgradeDB] Re-creating Settings table
[upgradeDB] Re-creating Parameters table
[2023-04-01 14:37:15] Plugins: Number of dynamically loaded plugins: 5
[Plugins] ---------------------------------------------
[Plugins] display_name: DHCP Leases
[Plugins] description: This plugin is to import devices from dhcp.leases files.
[Plugins] ---------------------------------------------
[Plugins] display_name: Services (NMAP)
[Plugins] description: This plugin shows all services discovered by NMAP scans.
[Plugins] ---------------------------------------------
[Plugins] display_name: UniFi import
[Plugins] description: This plugin is used to import devices from an UNIFI controller.
[Plugins] ---------------------------------------------
[Plugins] display_name: Website monitor
[Plugins] description: This plugin is to monitor status changes of services or websites.
[Plugins] ---------------------------------------------
[Plugins] display_name: Rogue DHCP
[Plugins] description: This plugin is to use NMAP to monitor for rogue DHCP servers.
[API] Updating table_settings.json file in /front/api
[2023-04-01 14:37:16] Config: Imported new config
[API] Updating table_devices.json file in /front/api
[API] Updating table_nmap_scan.json file in /front/api
Paste your pialert.conf (remove personal info)
#-------------------------------------------------------------------------------
# Pi.Alert
# Open Source Network Guard / WIFI & LAN intrusion detector
#
# pialert.conf - Back module. Configuration file
#-------------------------------------------------------------------------------
# Puche 2021 [email protected] GNU GPLv3
#-------------------------------------------------------------------------------
# GENERAL settings
# ----------------------
PRINT_LOG = False
TIMEZONE = 'Europe/Amsterdam'
PIALERT_WEB_PROTECTION = False
PIALERT_WEB_PASSWORD = ''
INCLUDED_SECTIONS = ['internet', 'new_devices', 'down_devices', 'events'] # Specifies which events trigger notifications.
# Remove the event type(s) you don't want to get notified on
# Overrides device-specific settings in the UI.
SCAN_CYCLE_MINUTES = 5 # delay between scans
# EMAIL settings
# ----------------------
SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = 587
SMTP_USER = '[email protected]'
SMTP_PASS = 'password'
SMTP_SKIP_TLS = False
SMTP_SKIP_LOGIN = False
REPORT_MAIL = False
REPORT_FROM = 'Pi.Alert <' + SMTP_USER +'>'
REPORT_TO = '[email protected]'
REPORT_DEVICE_URL = 'http://pi.alert/deviceDetails.php?mac='
REPORT_DASHBOARD_URL = 'http://pi.alert/'
# Webhook settings
# ----------------------
REPORT_WEBHOOK = False
WEBHOOK_URL = 'http://n8n.local:5555/webhook-test/aaaaaaaa-aaaa-aaaa-aaaaa-aaaaaaaaaaaa'
WEBHOOK_PAYLOAD = 'json' # webhook payload data format for the "body > attachements > text" attribute
# in https://github.com/jokob-sk/Pi.Alert/blob/main/docs/webhook_json_sample.json
# supported values: 'json', 'html' or 'text'
# e.g.: for discord use 'html'
WEBHOOK_REQUEST_METHOD = 'GET' # POST, GET...
# Apprise settings
#-----------------------
REPORT_APPRISE = False
APPRISE_HOST = 'http://localhost:8000/notify'
APPRISE_URL = 'mailto://smtp-relay.sendinblue.com:[email protected]&name=apprise&[email protected]&pass=password&[email protected]'
# NTFY (https://ntfy.sh/) settings
# ----------------------
REPORT_NTFY = False
NTFY_HOST = 'https://ntfy.sh'
NTFY_TOPIC = 'replace_my_secure_topicname_91h889f28'
NTFY_USER = 'user' # set username...
NTFY_PASSWORD = 'passw0rd' # ...and password if authentication is used https://docs.ntfy.sh/config/#users-and-roles
# PUSHSAFER (https://www.pushsafer.com/) settings
# ----------------------
REPORT_PUSHSAFER = False
PUSHSAFER_TOKEN = 'ApiKey'
# MQTT settings
# ----------------------
REPORT_MQTT = False
MQTT_BROKER = '192.168.1.2'
MQTT_PORT = 1883
MQTT_USER = 'mqtt'
MQTT_PASSWORD = 'passw0rd'
MQTT_QOS = 0
MQTT_DELAY_SEC = 2 # delay in seconds, increase if not all devices are created in Home Assistant
# DynDNS
# ----------------------
# QUERY_MYIP_SERVER = 'https://diagnostic.opendns.com/myip'
QUERY_MYIP_SERVER = 'http://ipv4.icanhazip.com'
DDNS_ACTIVE = False
DDNS_DOMAIN = 'your_domain.freeddns.org'
DDNS_USER = 'dynu_user'
DDNS_PASSWORD = 'A0000000B0000000C0000000D0000000'
DDNS_UPDATE_URL = 'https://api.dynu.com/nic/update?'
# PIHOLE settings
# ----------------------
PIHOLE_ACTIVE = False # if enabled you need to map '/etc/pihole/pihole-FTL.db' in docker-compose.yml
PIHOLE_DB = '/etc/pihole/pihole-FTL.db'
DHCP_ACTIVE = False # if enabled you need to map '/etc/pihole/dhcp.leases' in docker-compose.yml
DHCP_LEASES = '/etc/pihole/dhcp.leases'
# arp-scan options & samples
# ----------------------
#
# Scan local network (default)
# SCAN_SUBNETS = '--localnet'
#
# Scan two subnets
# SCAN_SUBNETS = '192.168.11.0/24 192.168.144.0/24'
#
# Scan using interface eth0
# SCAN_SUBNETS = '--localnet --interface=eth0'
#
# Scan multiple interfaces (eth1 and eth0):
# SCAN_SUBNETS = [ '192.168.1.0/24 --interface=eth1', '192.168.1.0/24 --interface=eth0' ]
SCAN_SUBNETS = ['192.168.1.0/24 --interface=vmbr2', '192.168.3.0/24 --interface=vmbr3', '192.168.4.0/24 --interface=vmbr0']
# Maintenance Task Settings
# ----------------------
DAYS_TO_KEEP_EVENTS = 90
Paste your docker-compose.yml and .env (remove personal info)
docker-compose.yml
version: "3"
services:
pialert:
container_name: pialert
image: "jokobsk/pi.alert:latest"
network_mode: "host"
restart: unless-stopped
volumes:
- /root/pialert/config:/home/pi/pialert/config
- /root/pialert/db/pialert.db:/home/pi/pialert/db/pialert.db
- /root/pialert/logs:/home/pi/pialert/front/log
environment:
- TZ=Europe/Amsterdam
- HOST_USER_ID=1000
- HOST_USER_GID=1000
- PORT=20211
Screenshots
CPU and RAM over time of the Pi.Alert container:
