Skip to content

rb_ext_ractor_safe() to declare ractor-safe ext - #3824

Merged
ko1 merged 1 commit into
ruby:masterfrom
ko1:ractor_safe_ext
Dec 1, 2020
Merged

rb_ext_ractor_safe() to declare ractor-safe ext#3824
ko1 merged 1 commit into
ruby:masterfrom
ko1:ractor_safe_ext

Conversation

@ko1

@ko1 ko1 commented Nov 30, 2020

Copy link
Copy Markdown
Contributor

C extensions can violate the ractor-safety, so only ractor-safe
C extensions (C methods) can run on non-main ractors.
rb_ext_ractor_safe(true) declares that the successive
defined methods are ractor-safe. Otherwiwze, defined methods
checked they are invoked in main ractor and raise an error
if invoked at non-main ractors.

[Feature #17307]

C extensions can violate the ractor-safety, so only ractor-safe
C extensions (C methods) can run on non-main ractors.
rb_ext_ractor_safe(true) declares that the successive
defined methods are ractor-safe. Otherwiwze, defined methods
checked they are invoked in main ractor and raise an error
if invoked at non-main ractors.

[Feature ruby#17307]
@ko1
ko1 merged commit 182fb73 into ruby:master Dec 1, 2020
@ko1
ko1 deleted the ractor_safe_ext branch December 1, 2020 06:44
okeeblow added a commit to okeeblow/ox that referenced this pull request Jul 26, 2021
…ax` example.

For ohler55#277

I've been experimenting with `Ractor`s since upgrading to Ruby 3.0 for ohler55#275
but quickly ran into `Ractor::UnsafeError` when trying to call `Ox::sax_parse` in anything except the main `Ractor`.

Per Ruby's `Ractor` C extension documentation (link below):
"By default, all C extensions are recognized as `Ractor`-unsafe. If C extension becomes `Ractor`-safe,
the extension should call `rb_ext_ractor_safe(true)` at the `Init_` function and all defined method marked as `Ractor`-safe.
`Ractor`-unsafe C-methods only been called from main-ractor. If non-main ractor calls it, then `Ractor::UnsafeError` is raised."

I don't like to open seemingly-large feature requests like this without making some attempt at it myself first,
and luckily it seems like `Ox::sax_parse` Just Works™ since I marked it `Ractor`-safe, even with the `class_cache`.
Confirming this safety, making any remaining changes to `Ox::Sax`, and expanding this to the non-`Sax` parts of `Ox`
are all unfortunately out of my depth as a n00b C coder, so I would appreciate if you could take this over if it interests you.
I am happy with just `Sax` support since I have no current need for marshalling,
but I imagine other `Ox` users wouldn't be satisfied if stratified.

In this commit:
- Enable `rb_ext_ractor_safe` preprocessor macro via `have_func` in `extconf.rb`.
- Mark `Init_Ox` and `ox_sax_parse` as `Ractor` -safe.
- Add a new `Ractor`-based `Ox::Sax` example exercising both parallel and serial `Ox::Sax` handler `Ractor`s
  to parse data from `shared-mime-info` XML files many users likely already have on their systems.

Official `Ractor` info:

- "Ractor: a proposal for a new concurrent abstraction without thread-safety issues": https://bugs.ruby-lang.org/issues/17100 (ruby/ruby#3365)
- Ruby's official `Ractor` documentation: https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- "A way to mark C extensions as thread-safe, Ractor-safe, or unsafe": https://bugs.ruby-lang.org/issues/17307 (ruby/ruby#3824)
- Ruby's C Extension `Ractor` documention covering `rb_ext_ractor_safe`: https://docs.ruby-lang.org/en/master/doc/extension_rdoc.html#label-Appendix+F.+Ractor+support
- A `Ractor` C Extension from the creator of `Ractor` that might serve as a useful example: https://github.com/ko1/ractor-tvar

Blogs:

- "Ractors: Multi-Core Parallel Processing Comes to Ruby 3": https://www.ruby3.dev/ruby-3-fundamentals/2021/01/27/ractors-multi-core-parallel-processing-in-ruby-3/
- "Ruby Ractor Experiments: Safe async communication" :https://ivoanjo.me/blog/2021/02/14/ractor-experiments-safe-async/
- "Playing with Ruby Ractors": https://billy-ruffian.co.uk/playing-with-ruby-ractors/
- "How Fast are Ractors?": https://www.fastruby.io/blog/ruby/performance/how-fast-are-ractors.html (https://github.com/noahgibbs/ractor_basic_benchmarks/tree/main/benchmarks)

Before this change:

```
[okeeblow@emi#CHECKING-YOU-OUT] time ./bin/checking-you-out ~/224031-dot-jpg
Received /home/okeeblow/.local/share/mime/packages/user-extension-rsrc.xml
/home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `sax_parse': ractor unsafe method called from not main ractor (Ractor::UnsafeError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `block in open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:24:in `block (2 levels) in remember_me'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `loop'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `block in remember_me'
<internal:ractor>:583:in `send': The incoming-port is already closed (Ractor::ClosedError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:86:in `block in extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each_with_object'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `extend'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `<top (required)>'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `require_relative'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `<top (required)>'
        from ./bin/checking-you-out:3:in `require_relative'
        from ./bin/checking-you-out:3:in `<main>'
./bin/checking-you-out ~/224031-dot-jpg  0.12s user 0.05s system 100% cpu 0.168 total
```

My new `Ox::Sax` Ractor example script's usage:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb
Please provide the path to a `shared-mime-info` XML package and some media-type query arguments (e.g. 'image/jpeg')
```

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml
Please provide some media-type query arguments (e.g. 'image/jpeg')
```

Finding all-extant types:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml image/jpeg font/ttf application/xhtml+xml image/x-pict
Parallel Ractors
["Worker 0 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]",
 "Worker 1 gave us TrueType 字型 (font/ttf) [.ttf]",
 "Worker 2 gave us XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]",
 "Worker 3 gave us Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]"]

Serial Ractor
"ONLY ONE OX gave us [#<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>, #<CYO TrueType 字型 (font/ttf) [.ttf]>, #<CYO XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]>, #<CYO Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]>]"
```

…and not finding invalid ones:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml lol/rofl fart/butt image/jpeg
Parallel Ractors
["Worker 0 gave us nothing",
 "Worker 1 gave us nothing",
 "Worker 2 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]"]

Serial Ractor
"ONLY ONE OX gave us [nil, nil, #<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>]"
```

Unit tests pass.
ohler55 pushed a commit to ohler55/ox that referenced this pull request Jul 27, 2021
…ax` example. (#278)

For #277

I've been experimenting with `Ractor`s since upgrading to Ruby 3.0 for #275
but quickly ran into `Ractor::UnsafeError` when trying to call `Ox::sax_parse` in anything except the main `Ractor`.

Per Ruby's `Ractor` C extension documentation (link below):
"By default, all C extensions are recognized as `Ractor`-unsafe. If C extension becomes `Ractor`-safe,
the extension should call `rb_ext_ractor_safe(true)` at the `Init_` function and all defined method marked as `Ractor`-safe.
`Ractor`-unsafe C-methods only been called from main-ractor. If non-main ractor calls it, then `Ractor::UnsafeError` is raised."

I don't like to open seemingly-large feature requests like this without making some attempt at it myself first,
and luckily it seems like `Ox::sax_parse` Just Works™ since I marked it `Ractor`-safe, even with the `class_cache`.
Confirming this safety, making any remaining changes to `Ox::Sax`, and expanding this to the non-`Sax` parts of `Ox`
are all unfortunately out of my depth as a n00b C coder, so I would appreciate if you could take this over if it interests you.
I am happy with just `Sax` support since I have no current need for marshalling,
but I imagine other `Ox` users wouldn't be satisfied if stratified.

In this commit:
- Enable `rb_ext_ractor_safe` preprocessor macro via `have_func` in `extconf.rb`.
- Mark `Init_Ox` and `ox_sax_parse` as `Ractor` -safe.
- Add a new `Ractor`-based `Ox::Sax` example exercising both parallel and serial `Ox::Sax` handler `Ractor`s
  to parse data from `shared-mime-info` XML files many users likely already have on their systems.

Official `Ractor` info:

- "Ractor: a proposal for a new concurrent abstraction without thread-safety issues": https://bugs.ruby-lang.org/issues/17100 (ruby/ruby#3365)
- Ruby's official `Ractor` documentation: https://docs.ruby-lang.org/en/master/doc/ractor_md.html
- "A way to mark C extensions as thread-safe, Ractor-safe, or unsafe": https://bugs.ruby-lang.org/issues/17307 (ruby/ruby#3824)
- Ruby's C Extension `Ractor` documention covering `rb_ext_ractor_safe`: https://docs.ruby-lang.org/en/master/doc/extension_rdoc.html#label-Appendix+F.+Ractor+support
- A `Ractor` C Extension from the creator of `Ractor` that might serve as a useful example: https://github.com/ko1/ractor-tvar

Blogs:

- "Ractors: Multi-Core Parallel Processing Comes to Ruby 3": https://www.ruby3.dev/ruby-3-fundamentals/2021/01/27/ractors-multi-core-parallel-processing-in-ruby-3/
- "Ruby Ractor Experiments: Safe async communication" :https://ivoanjo.me/blog/2021/02/14/ractor-experiments-safe-async/
- "Playing with Ruby Ractors": https://billy-ruffian.co.uk/playing-with-ruby-ractors/
- "How Fast are Ractors?": https://www.fastruby.io/blog/ruby/performance/how-fast-are-ractors.html (https://github.com/noahgibbs/ractor_basic_benchmarks/tree/main/benchmarks)

Before this change:

```
[okeeblow@emi#CHECKING-YOU-OUT] time ./bin/checking-you-out ~/224031-dot-jpg
Received /home/okeeblow/.local/share/mime/packages/user-extension-rsrc.xml
/home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `sax_parse': ractor unsafe method called from not main ractor (Ractor::UnsafeError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:375:in `block in open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival/mr_mime.rb:331:in `open'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:24:in `block (2 levels) in remember_me'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `loop'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:19:in `block in remember_me'
<internal:ractor>:583:in `send': The incoming-port is already closed (Ractor::ClosedError)
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:86:in `block in extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `each_with_object'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/ghost_revival.rb:63:in `extended'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `extend'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out/inner_spirit.rb:216:in `<top (required)>'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `require_relative'
        from /home/okeeblow/Works/DistorteD/CHECKING-YOU-OUT/lib/checking-you-out.rb:10:in `<top (required)>'
        from ./bin/checking-you-out:3:in `require_relative'
        from ./bin/checking-you-out:3:in `<main>'
./bin/checking-you-out ~/224031-dot-jpg  0.12s user 0.05s system 100% cpu 0.168 total
```

My new `Ox::Sax` Ractor example script's usage:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb
Please provide the path to a `shared-mime-info` XML package and some media-type query arguments (e.g. 'image/jpeg')
```

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml
Please provide some media-type query arguments (e.g. 'image/jpeg')
```

Finding all-extant types:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml image/jpeg font/ttf application/xhtml+xml image/x-pict
Parallel Ractors
["Worker 0 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]",
 "Worker 1 gave us TrueType 字型 (font/ttf) [.ttf]",
 "Worker 2 gave us XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]",
 "Worker 3 gave us Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]"]

Serial Ractor
"ONLY ONE OX gave us [#<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>, #<CYO TrueType 字型 (font/ttf) [.ttf]>, #<CYO XHTML 網頁 (application/xhtml+xml) [.xhtml,.xht]>, #<CYO Macintosh Quickdraw/PICT 繪圖 (image/x-pict) [.pct,.pict,.pict1,.pict2]>]"
```

…and not finding invalid ones:

```
[okeeblow@emi#ox] ./examples/sax_ractor.rb /usr/share/mime/packages/freedesktop.org.xml lol/rofl fart/butt image/jpeg
Parallel Ractors
["Worker 0 gave us nothing",
 "Worker 1 gave us nothing",
 "Worker 2 gave us JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]"]

Serial Ractor
"ONLY ONE OX gave us [nil, nil, #<CYO JPEG 影像 (image/jpeg) [.jpeg,.jpg,.jpe]>]"
```

Unit tests pass.
c72578 pushed a commit to oetiker/rrdtool-1.x that referenced this pull request Apr 4, 2025
Ruby has a new method for concurrency called Ractors.

For C bindings to be used inside a Ractor they need to be marked
Ractor-safe [1]:
  If an extension desires to be marked as Ractor-safe the extension
  should call rb_ext_ractor_safe(true) at the Init_ function for the
  extension, and all defined methods will be marked as Ractor-safe.

By marking them Ractor-safe ruby programs can generate graphs concurrently.

See also ruby/ruby#3824 for more information concerning this method.

[1] https://docs.ruby-lang.org/en/master/extension_rdoc.html#label-Appendix+F.+Ractor+support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant