Slow route server config generation with v6.0.0. #732
Description
ISSUE TYPE
Bug Report
OS
Ubuntu 20.04
VERSION
define( 'APPLICATION_VERSION', '6.0.0' );
define( 'APPLICATION_VERDATE', '2021071400' );
ENVIRONMENT
ixpmanager@portal2:/srv/ixpmanager$ php -v
PHP 8.0.9 (cli) (built: Jul 30 2021 13:03:39) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.9, Copyright (c) Zend Technologies
with Zend OPcache v8.0.9, Copyright (c), by Zend Technologies
ixpmanager@portal2:/srv/ixpmanager$ dpkg -l | grep php
ii libapache2-mod-php8.0 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (Apache 2 module)
ii php-common 2:84+ubuntu20.04.1+deb.sury.org+1 all Common files for PHP packages
ii php-composer-ca-bundle 1.2.6-1 all utility library to find a path to the system CA bundle
ii php-composer-semver 1.5.1-1 all utilities, version constraint parsing and validation
ii php-composer-spdx-licenses 1.5.3-1 all SPDX licenses list and validation library
ii php-composer-xdebug-handler 1.4.0-1 all Restarts a process without Xdebug
ii php-ds 1.3.0-6+ubuntu20.04.1+deb.sury.org+1 amd64 PHP extension providing efficient data structures for PHP 7
ii php-json-schema 5.2.9-1 all implementation of JSON schema
ii php-memcache 8.0+4.0.5.2+3.0.9~20170802.e702b5f9+-1+ubuntu20.04.1+deb.sury.org+2 amd64 memcache extension module for PHP
ii php-memcached 3.1.5+2.2.0-9+ubuntu20.04.1+deb.sury.org+1 amd64 memcached extension module for PHP, uses libmemcached
ii php-psr-container 1.0.0-2 all Common Container Interface (PHP FIG PSR-11)
ii php-psr-log 1.1.2-1 all common interface for logging libraries
ii php-rrd 2.0.3+1.1.3-1+ubuntu20.04.1+deb.sury.org+1 amd64 PHP bindings to rrd tool system
ii php-symfony-console 4.3.8+dfsg-1ubuntu1 all run tasks from the command line
ii php-symfony-filesystem 4.3.8+dfsg-1ubuntu1 all basic filesystem utilities
ii php-symfony-finder 4.3.8+dfsg-1ubuntu1 all find files and directories
ii php-symfony-process 4.3.8+dfsg-1ubuntu1 all execute commands in sub-processes
ii php-symfony-service-contracts 1.1.8-1 all Generic abstractions related to writing services
ii php-yaml 2.2.1+2.1.0+2.0.4+1.3.2-6+ubuntu20.04.1+deb.sury.org+1 amd64 YAML-1.1 parser and emitter for PHP
ii php8.0 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 all server-side, HTML-embedded scripting language (metapackage)
ii php8.0-bcmath 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 Bcmath module for PHP
ii php8.0-cgi 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (CGI binary)
ii php8.0-cli 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 command-line interpreter for the PHP scripting language
ii php8.0-common 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 documentation, examples and common module for PHP
ii php8.0-curl 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 CURL module for PHP
ii php8.0-ds 1.3.0-6+ubuntu20.04.1+deb.sury.org+1 amd64 PHP extension providing efficient data structures for PHP 7
ii php8.0-fpm 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 server-side, HTML-embedded scripting language (FPM-CGI binary)
ii php8.0-gd 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 GD module for PHP
ii php8.0-igbinary 3.2.4+2.0.8-1+ubuntu20.04.1+deb.sury.org+1 amd64 igbinary PHP serializer
ii php8.0-intl 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 Internationalisation module for PHP
ii php8.0-mbstring 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 MBSTRING module for PHP
ii php8.0-memcache 8.0+4.0.5.2+3.0.9~20170802.e702b5f9+-1+ubuntu20.04.1+deb.sury.org+2 amd64 memcache extension module for PHP
ii php8.0-memcached 3.1.5+2.2.0-9+ubuntu20.04.1+deb.sury.org+1 amd64 memcached extension module for PHP, uses libmemcached
ii php8.0-msgpack 2.1.2+0.5.7-6+ubuntu20.04.1+deb.sury.org+1 amd64 PHP extension for interfacing with MessagePack
ii php8.0-mysql 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 MySQL module for PHP
ii php8.0-opcache 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 Zend OpCache module for PHP
ii php8.0-readline 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 readline module for PHP
ii php8.0-rrd 2.0.3+1.1.3-1+ubuntu20.04.1+deb.sury.org+1 amd64 PHP bindings to rrd tool system
ii php8.0-snmp 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 SNMP module for PHP
ii php8.0-xml 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 DOM, SimpleXML, XML, and XSL module for PHP
ii php8.0-yaml 2.2.1+2.1.0+2.0.4+1.3.2-6+ubuntu20.04.1+deb.sury.org+1 amd64 YAML-1.1 parser and emitter for PHP
ii php8.0-zip 8.0.9-1+ubuntu20.04.1+deb.sury.org+1 amd64 Zip module for PHP
CONFIGURATION
APP_DEBUG=false
APP_URL="https://portal.mydomain.net"
APP_TIMEZONE="Europe/London"
APP_LOG="single"
APP_LOG_LEVEL=debug
IXP_MULTIIXP_ENABLED=false
IXP_RESELLER_ENABLED=true
IXP_RESELLER_RESOLD_BILLING=false
IXP_AS112_UI_ACTIVE=false
IXP_FE_FRONTEND_DISABLED_LOGO=false
IXP_FE_CUSTOMER_BILLING_UPDATES_NOTIFY="REDACTED"
IXP_FE_FRONTEND_DISABLED_RS_PREFIXES=true
IXP_FE_FRONTEND_DISABLED_NETINFO=true
IXP_FE_FRONTEND_DISABLED_CUSTKIT=true
IXP_FE_FRONTEND_DISABLED_CONSOLE=true
IXP_FE_FRONTEND_DISABLED_LOGO=false
IXP_FE_FRONTEND_DISABLED_LOOKING_GLASS=false
IXP_FE_FRONTEND_DISABLED_FILTERED_PREFIXES=false
IXP_FE_BETA_CORE_BUNDLES=true
IXP_FE_VLANINTERFACES_HOSTNAME_REQUIRED=false
GRAPHER_BACKENDS="mrtg|sflow"
GRAPHER_CACHE_ENABLED=true
GRAPHER_BACKEND_MRTG_DBTYPE="log"
GRAPHER_BACKEND_MRTG_WORKDIR="/srv/mrtg"
GRAPHER_BACKEND_MRTG_LOGDIR="REDACTED"
GRAPHER_BACKEND_SFLOW_ENABLED=true
GRAPHER_BACKEND_SFLOW_ROOT="REDACTED"
GRAPHER_BACKEND_SMOKEPING_ENABLED=false
IXP_API_JSONEXPORTSCHEMA_PUBLIC=true
VIEW_SKIN="REDACTED"
VIEW_SMARTY_SKIN="REDACTED"
CACHE_DRIVER=memcached
DOCTRINE_PROXY_AUTOGENERATE=false
DOCTRINE_CACHE=array
DOCTRINE_CACHE_NAMESPACE=IXPMANAGERNAMESPACE
[email protected]
[email protected]
IXP_IRRDB_BGPQ3_PATH=/usr/bin/bgpq3
IXP_IRRDB_MIN_V4_SUBNET_SIZE=24
IXP_IRRDB_MIN_V6_SUBNET_SIZE=48
IXP_RPKI_RTR1_HOST=REDACTED
IXP_RPKI_RTR1_PORT=3323
IXP_RPKI_RTR2_HOST=REDACTED
IXP_RPKI_RTR2_PORT=3323
AUTH_PEERINGDB_ENABLED=true
PEERINGDB_OAUTH_CLIENT_ID="REDACTED"
PEERINGDB_OAUTH_CLIENT_SECRET="REDACTED"
PEERINGDB_OAUTH_REDIRECT="REDACTED"
AUTH_PEERINGDB_PRIVS=2
AUTH_ZF1_PWHASH="bcrypt"
SESSION_DRIVER=file
SUMMARY
Route server configuration is very slow compared to our old host on 5.8.0.
We have set up a new host with Ubuntu 20.04, IXP Manager v6.0.0, PHP 8.0 (FPM), Nginx.
Old host is the same spec VM, on the same VM cluster, but running Ubuntu 18.04, IXP Manager v5.8.0, PHP 7.3, Apache.
Same MySQL server, using a copy of the production database (with migrations ran for v6.0.0).
STEPS TO REPRODUCE
time curl -H "X-IXP-Manager-API-Key: mykey" https://portal.mydomain.net/api/v4/router/gen-config/rs3-ipv4
EXPECTED RESULTS
Similar results to the old host/version.
ACTUAL RESULTS
v5.8.0:
v4: 1m11.659s, 0m55.893s, 1m11.338s, 0m51.738s
v6: 0m33.376s, 0m32.611s, 0m33.523s, 0m34.540s
v6.0.0:
v4: 2m56.270s, 2m53.221s, 2m44.164s, 2m47.211s
v6: 0m55.845s, 0m54.098s, 0m58.139s, 0m59.906s
IMPORTANCE
RELEVANT LOGS
[2021-08-18 11:57:47] production.INFO: Generated router configuration for rs3-ipv4 and used 1335466608 bytes (1542991872 real) of memory.
[2021-08-18 12:00:41] production.INFO: Generated router configuration for rs3-ipv4 and used 1335466576 bytes (1542991872 real) of memory.
[2021-08-18 12:03:25] production.INFO: Generated router configuration for rs3-ipv4 and used 1341154136 bytes (1220030464 real) of memory.
[2021-08-18 12:06:13] production.INFO: Generated router configuration for rs3-ipv4 and used 1341154136 bytes (892874752 real) of memory.