Skip to content

Only hash keys once#259

Merged
xacrimon merged 1 commit into
xacrimon:masterfrom
arthurprs:small-optimizations
Aug 21, 2023
Merged

Only hash keys once#259
xacrimon merged 1 commit into
xacrimon:masterfrom
arthurprs:small-optimizations

Conversation

@arthurprs

Copy link
Copy Markdown
Contributor

Only hash keys once, by taking advantage of hashbrown raw entry api. This is roughly the same as #194.

@arthurprs
arthurprs force-pushed the small-optimizations branch from 3c0f7fa to ccb03b0 Compare March 25, 2023 21:11
@xacrimon

Copy link
Copy Markdown
Owner

Won't this break on resize AKA the reason the previous PR was reverted?

@arthurprs

Copy link
Copy Markdown
Contributor Author

Maybe I'm missing something, but I don't know how that would break. Can you point me to prior discussion?

AFAICT the number of shards and the Hasher are fixed for the lifetime of the DashMap. Thus, the routing result should remain correct for the lifetime of DashMap and so is the hash passed down to the hashbrown shard.

@xacrimon

Copy link
Copy Markdown
Owner

lgtm

@xacrimon
xacrimon merged commit 3448b6f into xacrimon:master Aug 21, 2023
Comment thread src/lib.rs
Comment on lines +1043 to +1044
let kptr: *const K = entry.key();
let vptr: *mut V = entry.get_mut().as_ptr();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this violates stacked borrows, the entry.get_mut() invalidates the *const to the key. Inverting the order of the operations should be enough though.

@Turbo87

Turbo87 commented Aug 29, 2023

Copy link
Copy Markdown

👋 I'm not sure how exactly, but it looks like this change broke the crate download counting code on crates.io (see rust-lang/crates.io#7039). Admittedly, we are us the dashmap API in some interesting ways, but I still didn't expect these changes to have such a big effect on us... 😅

@xacrimon

Copy link
Copy Markdown
Owner

hm, interesting. I'll yank and revert for now.

xacrimon added a commit that referenced this pull request Aug 29, 2023
@xacrimon

Copy link
Copy Markdown
Owner

@Turbo87 Thanks for reporting. v5.5.2 is now out with this PR reverted. Since the earlier release has some other changes, would you guys mind updating to 5.5.2 and seeing if the issue is still resolved?

@arthurprs

Copy link
Copy Markdown
Contributor Author

This is quite surprising. Seems like some bad interaction with dashmap raw-api.

@Turbo87

Turbo87 commented Aug 29, 2023

Copy link
Copy Markdown

would you guys mind updating to 5.5.2 and seeing if the issue is still resolved?

yep, we'll try that. can't promise an exact timeframe yet, but we'll probably test it within the next couple of days.

@arthurprs

arthurprs commented Aug 29, 2023

Copy link
Copy Markdown
Contributor Author

The problem seems to be that crates.io is replacing the Hashmaps shards yielded by the raw-api 😬 which in turn causes them to get a different hasher from Default::default.

Specifically the mem::take(..)
https://github.com/rust-lang/crates.io/blob/5b4b6aae45241c648379ed968e8ed02642db805f/src/downloads_counter.rs#L80
and
https://github.com/rust-lang/crates.io/blob/5b4b6aae45241c648379ed968e8ed02642db805f/src/downloads_counter.rs#L93C5-L93C63

I believe the issue would be prevented by replacing the hashmap with a new hashmap with the same hasher as before. This hasher can be acquired via DashMap::hasher().

@Turbo87

Turbo87 commented Aug 30, 2023

Copy link
Copy Markdown

Since the earlier release has some other changes, would you guys mind updating to 5.5.2 and seeing if the issue is still resolved?

just deployed with v5.5.3 and I can confirm that the issue is still resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants