Skip to content

Improve performance of various load order operations #563

@Infernio

Description

@Infernio
  • Startup: _fetch_load_order falls victim to quadratic complexity. Fixing that chops off ~126000 calls and ~0.3s from SSE startup -> 2f16e82
  • Activating/deactivating plugins: Takes way too long. Around 0.4s on my SSE setup, GUI interactions should be <0.1s. Profiling shows around 600000 calls being issued, which is absurd. -> 2f16e82, 2be2dfd, 91cd88d, b925fc7, 6aafdbd and 6412963
  • For some reason, deactivating the same plugin takes ~0.05s and ~20000 calls longer than activating it. -> a19c63b
  • Undo/redo load order can make use of the same redraw optimization I introduced for activating/deactivating -> 346d6df
  • And the same also applies for dragging and dropping/using Ctrl+Arrow keys -> 64d29da and b1b1123

Good read on response times and why 0.1s is important: https://www.nngroup.com/articles/response-times-3-important-limits/

Not entirely related, but Path.__hash__ seems to pick bad hashes. The following dict maps each calling function to the number of Path.__eq__ calls it issued in total during the course of one plugin getting disabled:

{'GetMasterStatus': 18,
 'SetFile': 1,
 '__contains__': 31,
 '__getitem__': 72,
 '_children': 1995,
 '_fix_load_order': 18,
 '_fixed_order_plugins': 24,
 '_refresh_mod_inis': 21,
 'activeIndex': 10,
 'cached_is_active': 30,
 'getItem': 7933,
 'lindex': 150,
 'real_index': 9}

So e.g. all ModInfo.real_index calls combined amount to 9 calls of Path.__eq___. The notable winner here is of course DataTable.getItem, with a whopping 7933 calls.
The line in question is

if row in data and column in data[row]:

Specifically the row in data part. It seems that the hashes returned by Path.__hash__ aren't good enough to place the keys (plugin names in this case) into enough buckets to avoid having to make a ton of comparisons - which of course takes dict performance down from O(1).

And I suspect that the cause is the usage of self._cs for the hash - ModInfo.real_index also hashes plugin names, but uses the proper case instead.
Would using self._s be safe? And would we need backwards compat for it? -> not safe :(

Ref #123, #205, #353

Metadata

Metadata

Assignees

Labels

A-load-orderArea: Load Order (load_order.py and _games_lo.py)A-perfArea: Performance (runtime performance and/or memory usage)C-enhancementCategory: Enhancement, a request to add or enhance a featureM-relnotesMisc: Issue should be listed in the version history for its milestone

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions