Skip to content

rust data loader example#1452

Merged
mythmon merged 5 commits into
mainfrom
mythmon/240607/rust-example
Jun 13, 2024
Merged

rust data loader example#1452
mythmon merged 5 commits into
mainfrom
mythmon/240607/rust-example

Conversation

@mythmon
Copy link
Copy Markdown
Contributor

@mythmon mythmon commented Jun 11, 2024

For our recent data loaders, I had prepared two examples, but only had time to demo one of them in the presentation. This is the other one, a data loader example that uses Rust to simulate 10 million poker hands and reports the probability of getting each hand.

@mythmon mythmon requested review from allisonhorst and mbostock June 11, 2024 23:04
Comment thread examples/loader-rust-to-json/src/data/poker.json.rs Outdated
@@ -0,0 +1,163 @@
#!/usr/bin/env rust-script
//! Since Framework uses rust-script, we can define dependencies here.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is really cool

Comment thread examples/loader-rust-to-json/src/index.md Outdated
Comment on lines +221 to +222
// Displays both large and small percentages by adding decimal places
// until there is something besides zeroes, or until the limit is hit.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat! Another “trick” I like with percentages is to not round 0.99999 to 100% but display it as 99.9% (not applicable here, but I can see this function being reused in other contexts).

Comment thread examples/loader-rust-to-json/src/index.md Outdated
@Fil
Copy link
Copy Markdown
Contributor

Fil commented Jun 12, 2024

Just a few nitpicking comments :) It's a great example!

Comment thread examples/loader-rust-to-json/src/index.md Outdated
@mythmon
Copy link
Copy Markdown
Contributor Author

mythmon commented Jun 12, 2024

As well as the review feedback, I made a fairly mechanical conversion of the Rust data loader into TypeScript, to get a speed comparison. Unsurprisingly, the Rust example is much faster, about 65x faster on my system. 20x of that can be theoretically attributed to parallelism, leaving a ~3x language speed up. I'm not sure if this should be included in the example though. What do you all think?

@Fil
Copy link
Copy Markdown
Contributor

Fil commented Jun 12, 2024

I like the idea!

Comment thread examples/loader-rust-to-json/src/index.md Outdated
@mythmon mythmon merged commit 41f874c into main Jun 13, 2024
@mythmon mythmon deleted the mythmon/240607/rust-example branch June 13, 2024 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants