Skip to content

fix(gui): stop the wxDataViewCtrl lists losing the user's place - #860

Merged
got3nks merged 8 commits into
amule-org:masterfrom
got3nks:fix/dataview-scroll-and-nav
Aug 8, 2026
Merged

fix(gui): stop the wxDataViewCtrl lists losing the user's place#860
got3nks merged 8 commits into
amule-org:masterfrom
got3nks:fix/dataview-scroll-and-nav

Conversation

@got3nks

@got3nks got3nks commented Aug 8, 2026

Copy link
Copy Markdown

Three ways the wxDataViewCtrl lists lost the user's place, reported against amulegui on macOS, plus the fallout from fixing them.

1. A re-sort threw the list back to the top

CMuleVirtualDataViewCtrl::SortList() ended in wxDataViewIndexListModel::Reset(), which says the model was replaced. Every backend answers that by rebuilding its view, and a rebuilt view starts at row 0 — horizontally too. With live sorting on a busy list that fires continuously, so shared files and the clients list could not be read while they updated.

A sort reorders rows; it does not replace them, and the count is identical on both sides of the std::sort. Rows are addressed by index and values are pulled per cell as they are drawn, so a repaint is all a reorder needs.

The bulk path cannot use that, and this is the trap worth knowing: AppendItemData() adds rows without telling the control — that is what makes a bulk load cheap — so Reset() was doubling as the only notification that the rows existed. Repainting there draws a list the control still believes is its old length, i.e. empty on a fresh populate. So the sort splits: SortItems() orders and reindexes silently, SortList() adds the repaint, FinishBulkLoad() keeps the Reset() it genuinely needs.

Affects servers, shared files, clients, downloads, file details and friends.

2. macOS page keys scrolled without taking the cursor

The unshifted page and home/end keys were left to NSOutlineView, on the grounds that scroll-without-select is the macOS convention. In a list it reads as broken: the view jumps, the cursor stays behind, and the next arrow key snaps straight back — so the page key looks like it did nothing.

They now run through the same handler as the shifted ones, which exists because the native control does nothing for those either. Unshifted replaces the selection with the row landed on, as an arrow key would; shifted keeps extending. Both move the cursor, which is the half that makes the following arrow key continue from what is on screen. Matches GTK and MSW, where the backends do this themselves.

3. The search list reset on every burst of results

Each arriving result marked the model dirty and the idle flush answered with Cleared(). During a search that is continuous, so the list snapped to the top constantly.

Arrivals now batch per idle into ItemsAdded() / ItemsChanged()on macOS only. On GTK, feeding this model's arrivals in one notification at a time aborts the application inside GtkTreeView's own red-black tree:

gtkrbtree.c:471:_gtk_rbtree_insert_after: assertion failed: (_gtk_rbtree_is_nil (tree->root))

That was first seen for ItemAdded() under a newly formed group, and routing grouped results back through Cleared() did not avoid it — the same abort came back from the batch of top-level additions. #796 had already found that neither ItemChanged() nor a delete-and-re-add made GTK or MSW re-derive container-ness; two aborts from two different notifications say the constraint is broader than container-ness, so this stops looking for the subset GTK tolerates. MSW is grouped with GTK deliberately: it has tree bookkeeping of its own and was not the platform this was reported on.

Those platforms keep one Cleared() per idle, and instead put the view back afterwards — the idle branch already restored the selection and the expanded rows, so the top row joins them. That works here and not in the virtual lists because the items differ: a CSearchFile pointer still names the same result after a rebuild, while a wxDataViewIndexListModel item is a row number that names whatever has since moved into it.

Measurements

Each of these was settled by probe rather than by argument, against wx 3.3.3:

GTK macOS
Refresh() / SetSelections() no scroll no scroll
SetCurrentItem(), row on screen no scroll no scroll
SetCurrentItem(), row off screen clamps onto the cursor
Cleared() / Reset() top top
EnsureVisible() after Cleared() exact lands at the bottom
ItemsAdded() / ItemsChanged() aborts holds position

Two consequences. The cursor restore is gated on the row being on screen — with no row ever clicked GTK's cursor sits at row 0, so restoring it unconditionally clamped the view to the top, which is the very thing being fixed. And GetScrollPos() is unusable on this control: wx asserts "this window is not scrollable" and answers 0, so an earlier save-and-restore built on it was inert.

From review

  • The synthetic first child of a new group is now announced. AddChild() creates a copy of the parent as the first child — the result received first — and notified nobody, so a freshly formed group reported one child where the model had two. macOS hid it by re-reading the model as it draws, which is the same thing that masked the original feat(search): port CSearchListCtrl to wxDataViewCtrl #796 defect.
  • Both batches de-duplicate. SetDownloadStatus() notifies every child of the parent it updated, so one arrival into a 50-variant group queued 51 entries and a busy idle window multiplied that.
  • Pending pointers are dropped through MuleNotify::SearchFileBeingDestroyed, which CCommentDialogLst already consumes for this purpose. The previous guard checked membership in a parent's child list, which nothing ever unlinks from — it could not detect a freed child.
  • SortList() restores the cursor by identity alongside the selection; Reset() used to invalidate it, and a repaint does not.

Verification

Built clean on macOS ARM64, Ubuntu ARM64 and Windows ARM64 — no warnings from project sources. clang-format and both clang-tidy tiers clean over the diff.

Confirmed interactively on amulegui on all three platforms: shared files and clients hold position through auto-sort, the search list holds position while results stream in, group expansion is intact, and the page keys move the cursor with the view on macOS.

got3nks added 3 commits August 8, 2026 13:14
SortList() ended in wxDataViewIndexListModel::Reset(), which says the
model was replaced. Every backend answers that by rebuilding its view,
and a rebuilt view starts at the top -- so an auto-sort while the user
was reading further down threw them back to row 0, horizontally as well.
With live sorting on a busy list that fires repeatedly.

A sort reorders rows; it does not replace them, and the count is the
same on both sides of the std::sort. Rows are addressed by index and
their values are pulled per cell as they are drawn, so a repaint is all
a reorder needs. Measured against a standalone sorted wxDataViewCtrl:
an insert or a value change holds the top row, Cleared()/Reset() drops
it to the first.

The bulk path cannot use that, though: AppendItemData() adds rows
without telling the control -- which is what makes a bulk load cheap --
so SortList()'s Reset() was doubling as the only notification that the
rows existed at all. Repainting there would draw a list the control
still believes is the length it was before, i.e. empty on a fresh
populate. So the sort splits: SortItems() orders and reindexes and tells
the control nothing, SortList() adds the repaint, and FinishBulkLoad()
keeps the Reset() it actually needs.

Affects every list on this base -- servers, shared files, clients,
downloads, file details, friends.
The unshifted page and home/end keys were left to NSOutlineView, on the
grounds that scrolling without moving the selection is the macOS
convention. In a list it reads as broken: the view jumps to the end, the
cursor stays where it was, and the next arrow key snaps straight back to
it -- so the page key looks like it did nothing at all.

They now go through the same handler as the shifted ones, which already
existed because the native control does nothing for those. Unshifted
replaces the selection with the row landed on, as an arrow key would;
shifted keeps extending, as before. Either way the cursor moves too,
which is the half that makes the following arrow key continue from what
is on screen rather than from where the user last left it.

Matches GTK and MSW, where the backends do this themselves.

PageExtendSelection() becomes MoveByPage(motion, extend) -- it no longer
only extends.
Each arriving result marked the model dirty and the idle flush answered
with wxDataViewModel::Cleared() -- "everything changed". The control
responds by rebuilding, which loses the scroll position, so a running
search could not be read: the list snapped back to the top on every
burst, and with an eD2k search that is continuous.

Arrivals now batch per idle into ItemsAdded()/ItemsChanged(), grouped by
parent since wx takes one parent per batch. Measured against a
standalone sorted wxDataViewCtrl, an insert or a value change holds the
top row where Cleared() drops it to the first.

Cleared() is kept where it is still the only correct answer: a filter
change, and any arrival while a filter is active -- m_filterKnown makes
a row's visibility a live function of its download status, so a value
change can require a row to appear or disappear, which only
re-evaluating the tree catches.

Grouped results -- ones that joined an existing result -- also keep it,
on every platform except macOS. PR amule-project#796 had already found that GTK and
MSW would not re-derive container-ness from ItemChanged() or a
delete-and-re-add. ItemAdded() under the new parent, the notification
that actually means "this row now has a child", does worse than fail on
GTK: it aborts inside GtkTreeView's own red-black tree,

  gtkrbtree.c:471:_gtk_rbtree_insert_after:
    assertion failed: (_gtk_rbtree_is_nil (tree->root))

because the row is inserted into a child tree GTK never built for a
parent it still considers a leaf. Native NSOutlineView has no such
structure to corrupt and re-queries IsContainer() as it draws, which is
why it takes the incremental path happily. Only grouping is affected: a
plain new result and a value change stay incremental everywhere, and
those are the bulk of what arrives.

The batch holds CSearchFile pointers between the notification and the
flush, where the old code held only a bool. A dropped search frees its
results in that window, so the flush re-checks each one: top-level
results against the indexed list, children against their parent still
being live and still owning them (IndexResult() keeps only top-level
results, so children are not in that list).

The control skips its save-and-restore of selection and expansion on the
incremental path, having nothing to protect them from.
@got3nks
got3nks force-pushed the fix/dataview-scroll-and-nav branch from bb37d22 to 01dbcc2 Compare August 8, 2026 11:26
got3nks added 2 commits August 8, 2026 13:35
…ointers

Five things review turned up in the incremental-notification work.

Forming a group creates two children, not one. AddChild() synthesises a
copy of the parent as the first child -- the result received first -- and
notifies nobody, while AddToList() infers survival from the child count
and announces only the incoming result. So the batch said one child where
the model reports two. macOS hides it by re-reading the model as it
draws, which is the same thing that masked the original amule-project#796 defect; a
backend that builds its rows from the notifications shows a two-variant
group holding one row. Announced at the point of creation now, so a
freshly formed group is reported in full.

Sorting left the cursor addressed by row. SetSelectedItemData() restores
the selection by identity, but nothing round-tripped GetCurrentItem();
Reset() used to invalidate it along with the rest of the view, and a
repaint does not. On a live-sorted list it comes to rest on whatever file
moved into that row, so the next arrow key steps from the wrong place and
a shifted page key extends from the wrong anchor -- the same disagreement
between cursor and view that the macOS page keys were fixed for.

The pending batches held pointers vouched for by membership in a parent's
child list, which is not a liveness test: nothing in the tree ever
unlinks a child, so being listed survives being freed. CSearchFile
already broadcasts its own destruction, and CCommentDialogLst already
consumes it for this exact purpose -- the model now does too, and drops
the pointer when the result dies. The membership dance goes away with it.

Both batches now de-duplicate. SetDownloadStatus() notifies every child
of the parent it updated, so a single arrival into a 50-variant group
queued 51 entries and a busy idle window multiplied that; the control is
handed each row once.

And the comment above the idle flush still described one Cleared() per
idle as the policy, which is now only the fallback branch.
Reporting arrivals one at a time aborts amulegui on GTK:

  gtkrbtree.c:471:_gtk_rbtree_insert_after:
    assertion failed: (_gtk_rbtree_is_nil (tree->root))

First seen for ItemAdded() under a newly formed group, and routing
grouped results back through Cleared() did not avoid it -- the same abort
came back from the batch of top-level additions. wx's GTK backend keeps
its own mirror of the model's tree, and this model's arrivals corrupt
GtkTreeView's red-black tree whichever notification carries them.

PR amule-project#796 had already found that neither ItemChanged() nor a
delete-and-re-add made GTK or MSW re-derive container-ness, and settled
on Cleared() for that reason. Two aborts from two different notifications
say the constraint is broader than container-ness, so this stops looking
for the subset GTK tolerates and gives those platforms back exactly the
behaviour they had: one Cleared() per idle.

macOS keeps the incremental path, which is where the problem was reported
and where it is verified. Native NSOutlineView keeps no parallel tree --
it re-queries the model as it draws, which is what masked the original
amule-project#796 defect -- so it takes the notifications happily.

MSW is grouped with GTK deliberately: its generic implementation has tree
bookkeeping of its own, it has not been tested here, and keeping today's
behaviour costs it nothing but a repaint.

The rest of the review fixes stand on their own and apply everywhere: the
synthetic child is announced, the sort restores the cursor, the batches
are de-duplicated, and a destroyed result is dropped from them.
@got3nks
got3nks force-pushed the fix/dataview-scroll-and-nav branch from 6531788 to 91ac5a2 Compare August 8, 2026 11:52
The previous commit blamed the selection and the cursor for the lists
still jumping on GTK, and guarded against it by reading and writing back
GetScrollPos(). Both halves were wrong. Measured against wx 3.3.3 and
GTK 3.24, with a wxDataViewCtrl scrolled to row 199 of 400:

  Refresh()                        toprow=199   (does not scroll)
  SetSelections(205,206)           toprow=199   (does not scroll)
  SetCurrentItem(205, on screen)   toprow=199   (does not scroll)
  SetCurrentItem(0, off screen)    toprow=0     (clamps onto the cursor)
  Reset(400)                       toprow=0

GetScrollPos() does not work on this control at all -- wx asserts "this
window is not scrollable" and answers 0 -- so the guard read 0, wrote 0,
and in a debug build produced an assertion per sort. It is gone.

The cause is the cursor restore added a commit ago, which is this
branch's own regression rather than anything older: with no row ever
clicked GTK's cursor sits at row 0, so restoring it after a sort clamps
the view onto row 0 and the list snaps to the top -- the exact behaviour
being fixed. It only bites when the cursor is off screen, which is also
when nobody is about to arrow from it, so the restore now happens only
while the row it lands on is within [GetTopItem(), +GetCountPerPage()).
Both accessors are implemented on all three backends; GTK answers 12 for
a 300px list, so the window is real rather than a fallback.

A cursor the user cannot see is left stale, as it was before this branch.
A cursor they can see is the one that has to point at the file they
clicked, which is what finding it in the wrong place after a live re-sort
was about.
@got3nks
got3nks force-pushed the fix/dataview-scroll-and-nav branch from 91ac5a2 to 5cdb2e5 Compare August 8, 2026 11:58
got3nks added 2 commits August 8, 2026 14:38
On GTK and MSW the search list still resets on every burst of results:
incremental notifications abort the application there, so those backends
keep one Cleared() per idle, which drops the view to the top.

The rebuild itself is unavoidable; losing the reading position is not.
The idle branch already captures the selection and the expanded rows and
puts them back afterwards, so the row at the top joins them. Measured on
GTK with wx 3.3.3, a wxDataViewCtrl scrolled to row 199 of 400:

  after Cleared()                 top=0
  after EnsureVisible(saved)      top=199

Exactly back, not approximately: the minimal scroll GTK performs to bring
a row below the viewport into view puts it at the top. (A two-step
reach-past-and-return, which sounded more precise, lands on row 1 -- so
the simple call is also the correct one.)

This works here and not in the virtual lists because the items differ. A
CSearchFile pointer still names the same result after the rebuild; a
wxDataViewIndexListModel item is a row number, which names whatever has
since moved into it -- restoring one after Reset() does nothing, which
is why those lists avoid the reset instead.

Checked against the live tree before use, like the selection and
expansion beside it.
@got3nks
got3nks merged commit 8f2d8c7 into amule-org:master Aug 8, 2026
14 checks passed
@got3nks
got3nks deleted the fix/dataview-scroll-and-nav branch August 8, 2026 13:53
got3nks added a commit that referenced this pull request Aug 13, 2026
…ctory (#905)

Browsing a large share froze both ends for minutes, reported in #898. The instrumented build put numbers on it, and the two ends turned out to be stuck on different things that share a shape: O(directories x files).

Measured on the reporter's share of **39,450 files across 1,691 directories**.

## Sender: 51 s

A peer browses one directory at a time, and `GetSharedFilesByDirectory()` answered each request by walking the whole of `m_Files_map` calling `CPath::IsSameDir()` - which normalises *both* paths every time. That is 66.7 million comparisons, so 133 million normalisations, for one browse.

The files are now grouped by directory once per browse and each request answered from the grouping: **41,000 normalisations instead of 133 million**.

## The key

`CPath::GetDirKey()` is added here, and it is not a new comparison rule. `IsSameAs()` already worked by canonicalising both sides and comparing the results; this names that form so it can be computed once per path instead of twice per comparison. For paths containing a separator, which every shared directory has, grouping by it is equivalent to the walk it replaces, including the case folding that differs by platform, because `wxPATH_NORM_CASE` is part of the same reduction.

The equivalence is not unconditional, and the header says so: `IsSameDir()` compares two *bare* filenames with `PATHCMP` while `GetDirKey()` normalises them, so on Windows `wxPATH_NORM_LONG` would give `PROGRA~1` and `Program Files` one key where `PATHCMP` calls them different. Unreachable through this call path, since shared directories are always absolute, and the test table contains only separator paths so what is asserted is exactly what holds.

`IsSameAs()` keeps its structure exactly: bare filenames still compare through `PATHCMP`, everything else still normalises both sides. `NormalizedKey()` is a pure extraction of the second branch.

Invalidation comes from `m_listGeneration`, already bumped under `list_mut` wherever `m_Files_map` changes, which is the lock the grouping is built and read under. `RefreshPathIndex()` now bumps it too: a completed download is re-pathed from Temp to Incoming without entering or leaving the map, so `AddFile()`'s insert no-ops and nothing else marked the move. The pairwise walk re-read `GetFilePath()` every call and so never needed telling; a cache does.

## Receiver: 232 s

Every burst of arriving results took `MarkDirty()` -> `Cleared()`, rebuilding the whole accumulated tree: **384 rebuilds** over a set growing to 39,450 rows, 232 s in total, the last one 5.2 s on its own. The measured main-loop stalls came to 228 s, which is that same work seen from the other side.

While a browse is still streaming, the rebuild now waits for the row count to grow by half. The threshold resets when a browse starts, since a re-browse reuses the same tab and the previous browse's final count would otherwise be a bar the new one never clears. The rebuilds then form a geometric series whose total is a small multiple of the final one, and results still appear as the browse runs rather than only at the end. A finished or failed browse always rebuilds, so the last state is exact.

## Testing

The tests assert the property the grouping depends on rather than the key's contents, which are platform-dependent by design: for a table of paths, `a.IsSameDir(b)` exactly when their keys are equal. The table includes case variants, which are the same directory on Windows and different ones elsewhere, so one assertion covers both regimes without a platform `#ifdef` - and that is precisely where a hand-rolled key would diverge. Both branches kept by the `IsSameAs()` extraction are asserted too.

Built and run on **macOS**, **Linux** and **Windows**: 34/34 unit tests pass on Linux and Windows, `PathTest` green on macOS.

Not yet exercised against a real browse - that is the next step, with instrumentation rebased on top of this, and the reporter of #898 re-running it.

## Not addressed

The duplicate scan in `CSearchList::AddToList` is genuinely quadratic - 778 million steps in the same log - but it measured 6 s of 238 s, and it is left for its own change.




## Measured in the field

The reporter of #898 re-ran the same browse on this branch, 39,450 files across 1,691 directories, with an instrumented build.

**Sender, fixed outright.** No main-loop stall at all, against 53.5 s before. The directory-list reply takes 186 ms for 1,690 directories, and all 1,691 per-directory lookups together take 173 ms - of which 161 ms is the first call building the grouping, so the remaining 1,690 share about 12 ms. The whole browse costs the sender 1.0 s.

**Receiver, improved but not solved.** Rebuilds fell from 384 to 64 and their total from 232 s to 32.1 s, and recorded stalls from 119 totalling 228 s to 15 totalling 35.0 s. That is roughly a sevenfold improvement, and the reporter describes the machine as responsive throughout, but 35 s of blocking remains and the worst single stall is 6.3 s - essentially the final rebuild, 5.6 s at 39,450 rows, which `Cleared()` cannot avoid while that is the only notification wxGTK tolerates for this model (see #860).

Two things are therefore left for follow-up rather than claimed here: the rebuild count is higher than the growth schedule intends and is worth understanding before tuning, and the duplicate scan in `CSearchList::AddToList` still runs 778 million steps for 6.8 s, unchanged by this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant