Skip to content

amuleweb: O(N^2) -> O(N) array_push_back via cached scan hint (#666) - #689

Merged
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/amuleweb-php-push-back-quadratic
May 23, 2026
Merged

amuleweb: O(N^2) -> O(N) array_push_back via cached scan hint (#666)#689
mrjimenez merged 1 commit into
amule-project:masterfrom
got3nks:fix/amuleweb-php-push-back-quadratic

Conversation

@got3nks

@got3nks got3nks commented May 23, 2026

Copy link
Copy Markdown
Contributor

Related to #666 — secondary amuleweb-side wedge surfaced by @Stoatwblr's latest gdb on amuleweb after #687 unblocked the amuled side.

Root cause

array_push_back in the PHP engine does:

for (int i = 0; i < 0xffff; i++) {
    PHP_VAR_NODE *arr_var_node = array_get_by_int_key(array, i);
    if (arr_var_node->value.type == PHP_VAL_NONE) {
        return arr_var_node;
    }
}

On a freshly-pushed array of size N, the scan walks i = 0..N-1 (each iteration does an snprintf("%d") + std::map<string, *>::count()) before finding the empty slot at i = N. Building the array via N push_backs costs O(N²).

amule_load_sharedamule_obj_array_create<SharedFileInfo, SharedFile> calls array_push_back once per shared file. On @Stoatwblr's 43585-file install that's ~950M map lookups + string conversions per amuleweb-main-shared.php render, pegging the amuleweb process at 100% CPU for the entire page load.

Backtrace top:

#0 __memcmp_avx2_movbe_rtm
#5 std::_Rb_tree<...>::_M_key_compare ("2051" vs "20509")
#8 std::map<string, PHP_VAR_NODE*>::count
#9 array_get_by_str_key at php_syntree.cpp:676
#10 array_get_by_int_key at php_syntree.cpp:695
#11 array_push_back at php_syntree.cpp:786
#12 amule_obj_array_create<SharedFileInfo, SharedFile> at php_amule_lib.cpp:766
#13 amule_load_shared at php_amule_lib.cpp:787

Fix

Cache the next-push starting index on PHP_ARRAY_TYPE and resume the scan from there, advancing on success. amuleweb's PHP code (engine glue + the default *.php templates) only ever push_backs — no unset, array_pop, array_shift, or array_splice callers — so the hint is always accurate and each push_back collapses to O(1).

For correctness against any future caller that does mixed insert+delete, a fallback low-range scan preserves the original "fill the first hole" semantics if the hint missed.

The anonymous typedef'd struct becomes a tagged struct so the new default-member-init doesn't trip -Wnon-c-typedef-for-linkage.

Test

Local macOS build of amule amuled amulegui amuleweb — clean, no warnings. The bottleneck is purely amuleweb-side, so combined with #687 (amuled-side) the full amuleweb-main-shared.php render path should be free of N² scans end-to-end.

PHP's array_push_back() linearly scans i = 0..0xffff every call,
looking for the first integer key whose value is PHP_VAL_NONE. On
a fresh array built by N successive push_backs the scan finds the
hole at i = N-1 each time, doing N-1 map lookups for push N, so
building an N-element array costs O(N^2). On a 43k-shared-files
amuleweb-main-shared.php this is ~950M std::map<string, *> lookups
plus snprintf("%d") per lookup, wedging amuleweb at 100% CPU for
the entire page render (issue amule-project#666 secondary wedge).

Cache the next-push starting index on PHP_ARRAY_TYPE and resume
the scan from there, advancing on success. amuleweb's PHP code
only ever push_backs (no unset/array_pop/array_shift across the
default templates or the PHP engine glue), so the hint is always
accurate and each push_back collapses to O(1). For correctness
against future mixed insert+delete callers a fallback low-range
scan keeps the original semantics.

Switch the typedef'd anonymous struct to a tagged struct so the
new default-member-init doesn't trip -Wnon-c-typedef-for-linkage.
@mrjimenez
mrjimenez merged commit bd71de3 into amule-project:master May 23, 2026
12 checks passed
@got3nks
got3nks deleted the fix/amuleweb-php-push-back-quadratic branch May 23, 2026 20:06
@got3nks got3nks mentioned this pull request May 24, 2026
ngosang pushed a commit to ngosang/amule that referenced this pull request Jul 29, 2026
…project#689)

Closes amule-project#617.

Downloads: a "Total queue size: X" field, right-aligned in the File
sources row, summing the files currently visible (selected category +
text filter). Kept live the same way as the file count -- reset in bulk
by RebuildVisibleList(), adjusted by GetFileSize() as ShowFile() adds or
removes a row -- so there is no per-tick cost.

Shared: the combined size of the visible shared files, shown as "Total
size of Shared Files: X" (reusing the existing string) in the statistics
box, in place of the "Percent of total files" label -- which was
misleading, since the gauges beside it show a session/all-time ratio per
selected file, not a percentage of files. The session/all-time gauges
stay.

Both auto-scale units via CastItoXBytes and work in amuleGUI too: file
sizes are already EC-streamed, so no core/protocol change is needed. The
"Total queue size: %s" string is new; the shared label reuses an existing
one. Catalogs regenerated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants