Unfortunately, it doesn't work well when the connector module is built with PCRE2, nginx is built with PCRE2, and libmodsecurity is built with PCRE1:

start:      SUMMARY: AddressSanitizer: 59014 byte(s) leaked in 2698 allocation(s).
1st reload: SUMMARY: AddressSanitizer: 18741824 byte(s) leaked in 291932 allocation(s).
2nd reload: SUMMARY: AddressSanitizer: 37483648 byte(s) leaked in 583864 allocation(s).
3rd reload: SUMMARY: AddressSanitizer: 56225472 byte(s) leaked in 875796 allocation(s).
stop:       SUMMARY: AddressSanitizer: 74967296 byte(s) leaked in 1167728 allocation(s).

Full error log here: https://gist.github.com/defanator/de14eacd93eeb44a82c608d624702b85

Perhaps better option for now would be to continue building ModSecurity-nginx with PCRE1. It won't work in long-term however as PCRE1 is basically not supported anymore and eventually everything (including libmodsecurity) hopefully will migrate to PCRE2.

I'll leave this one open just in case if anyone would suggest other options.

Surprisingly, there are memory leaks when connector is built with PCRE1 and nginx is built with PCRE2:

start:      SUMMARY: AddressSanitizer: 59014 byte(s) leaked in 2698 allocation(s).
1st reload: SUMMARY: AddressSanitizer: 18014174 byte(s) leaked in 288592 allocation(s).
2nd reload: SUMMARY: AddressSanitizer: 36028348 byte(s) leaked in 577184 allocation(s).
3rd reload: SUMMARY: AddressSanitizer: 54042522 byte(s) leaked in 865776 allocation(s).
stop:       SUMMARY: AddressSanitizer: 72056696 byte(s) leaked in 1154368 allocation(s).

Full error log: https://gist.github.com/defanator/274356c4f0594331e9d128af898182ae

JFTR, here are the versions of all involved components:

ModSecurity-nginx: 2497e6a
ModSecurity: owasp-modsecurity/ModSecurity@52958fa
nginx: nginx/nginx@1f01183

The environment was built from https://github.com/defanator/modsecurity-performance (Ubuntu 20.04 "focal", vagrant box generic/ubuntu2004, version 3.6.2).

UPDATE: finally, leaks are still there with module built with PCRE1 and nginx built with PCRE1, so something bad is definitely happening in connector + libmodsec combo. Also, the above numbers were gathered without any external load between nginx reloads. If e.g. nikto scanning tool is running in a cycle (while [ :: ]; do nikto -host localhost -root /modsec-full/ ; done), worker's memory consumption wents crazy with every next nginx reload, especially in case when connector is using PCRE2.