ruby codegen: support generation of rbs files by HoneyryderChuck · Pull Request #15633 · protocolbuffers/protobuf

HoneyryderChuck · 2024-01-29T17:37:28Z

this introduces support for a new protoc option, --rbs_out, which points to a directory where ruby type definition files, defined in the RBS format, are stored.

rbs is the type signature syntax blessed by the ruby core team, used by static analysis tools such as steep, which integrates with VS Code, the irb console (for features such as autocompletion). and typeprof.

It relies on type definitions written into .rbs files.

The protobuf library already exposes type definitions in gem_rbs_collection, which is used to source type definitions for libraries which do not want, or can't maintain type definitions themselves.

(protobuf could arguably import these into this repository, lmk if you're interested).

This should fix gaps such as better IDE integration, such as the ones described
here.

The plan is to do roughly the same type of integration as was done for .pyi annotations, which also write to separate files: add the cli option and the rbs generator.

protobuf classes in ruby rely on dynamic attribution of the base class, which makes what I'm trying to achieve a bit difficult. Ideally the type hierarchy could be specified statically in the ruby source code.

class Bar < AbstractMessage
class Bar <
Google::Protobuf::DescriptorPool.generated_pool.lookup("Bar").msgclass

google-cla · 2024-01-29T17:37:33Z

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

HoneyryderChuck · 2024-01-29T17:38:08Z

@acozzette can I ask for your input on directions here, given your thoughts described in #9495 ?

acozzette · 2024-02-06T21:49:17Z

@HoneyryderChuck This looks like a great direction to me, but I will let @haberman review this since he is much more knowledgeable about Ruby than I am.

HoneyryderChuck · 2024-03-27T12:54:20Z

@haberman any feedback? waiting for comments about the direction, before coming back to this.

haberman · 2024-05-06T22:32:51Z

Sorry for the delay. I think this is a good direction, and I appreciate the small and incremental PR. I think we should move forward with this.

It looks like the CLA check is failing.

HoneyryderChuck · 2024-05-09T08:49:51Z

thx for the feedback @haberman . Did you have a look at #15881 , which may be already the endgame?

haberman · 2024-05-09T20:43:12Z

#15881 has a much greater scope, and I think we'd need to individually evaluate its many parts.

I'm convinced that we want a .rbs code generator that has static types for all of the generated APIs. I'm less certain about:

Adding .rbs files for our public core APIs
Adding .rbs files for our internal core APIs
Adding .rbs files for our unit tests
Adding Steep type checking to our build

HoneyryderChuck · 2024-05-10T09:02:41Z

makes sense 👍 will deal with CLA and move on from there.

HoneyryderChuck · 2024-05-10T15:54:38Z

@haberman went with cherry-picking @qnighy 's commit, and removing the parts not related with sole generation of rbs files on target protos. There was a lot done there that I'd struggle to replicate with the same quality, as I'm still fighting with the build system to generate binaries I can test with locally.

FWIW RBS definitions to protobuf itself are already available here; while it's possible to have the definitions within this repo itself as per the other MR, it's a choice of the maintainers.

qnighy · 2024-05-13T11:15:32Z

Honestly, from the experience working on the PR, I agree that it is probably too early to introduce a typechecker to an existing large project like google-protobuf, which naturally have idioms incompatible with the current type system.

That said, I think it is better to bundle the public type definition here.

The reason: firstly, just similarly to other ecosystems like TypeScript, type definition files are not mere derivative of the runtime code; writing type definitions inherently involves additional design decision. For example, whether to give a type an alias or not, and its name, depends on the author of the type definitions.

Secondly, in google-protobuf, the runtime and the generated code are expected to work in cooperation, and this would also be true in type definitions as well. Therefore, design decision made in the runtime type definition naturally depends on the decisions in the generator.

So, to consistently maintain type definitions, I think it should bundle the one for the runtime as well.

haberman · 2024-05-13T13:51:52Z

Those arguments make sense to me, but we also have to balance the benefits against the maintenance cost. The primary motivation for introducing type signatures for generated code is to make generated interfaces more discoverable and to help IDEs, because the Ruby generated code is currently very opaque, especially since we merged #12462.

The public APIs from the core runtime will also have this issue to some extent, since many interfaces are implemented in C and are probably not discoverable to IDEs. We have Rubydoc embedded in comments, like this example, but that won't help IDEs:

protobuf/ruby/ext/google/protobuf_c/defs.c

Lines 146 to 152 in f2d8c2b

    
           /* 
        
            * call-seq: 
        
            *     DescriptorPool.lookup(name) => descriptor 
        
            * 
        
            * Finds a Descriptor, EnumDescriptor or FieldDescriptor by name and returns it, 
        
            * or nil if none exists with the given name. 
        
            */

But once we're talking about internal-only APIs (like everything in ffi/) and especially unit tests, I think we start to get diminishing returns. The cost is having to write everything twice (once in .rb and once in .rbs), using an unfamiliar language/tool that nobody on the team currently knows, and we don't get any benefits unless we perform type checking in our own project. I think the maintenance burden would outweigh the benefit.

Steep describes itself as "Gradual Typing", which I assume means that type signatures are not required to be complete. Anything that has type signatures is checked, but everything else can continue to work even if it's not typed.

I think the sweet spot would be if we could generate .rbs for generated code, and possibly for the public interfaces in the core runtime, and add just enough type checking in our CI tests to ensure that those type signatures are correct with respect to the actual code. But we wouldn't bother with typing the internal-only interfaces or unit tests.

How does that sound?

HoneyryderChuck · 2024-05-13T14:40:20Z

@haberman that sounds reasonable, but I don't think we need to do this as part of this PR. FWIW anyone forcibly generating rbs defs for their protobufs will very likely know about rbs-collection already, and get typedefs for core runtime.

There'd still be value in adding typedefs to core runtime; I could see steep integration in CI via the added rbs-based suite catching basic bugs, and could be gradually adopted by the core team (or not).

haberman · 2024-05-14T19:54:30Z

I totally agree. If anything, I'm arguing to limit the scope of each individual PR and make things as incremental as possible.

HoneyryderChuck · 2024-05-14T21:05:05Z

@haberman thx for the feedback. Would you mind adding the safe for tests label, so we can continue moving forward?

this introduces support for a new protoc option, `--rbs_out`, which points to a directory where ruby type definition files, defined in the RBS format, are stored. [rbs](https://github.com/ruby/rbs) is the type signature syntax blessed by the ruby core team, used by static analysis tools such as [steep](https://github.com/soutaro/steep), which integrates with VS Code, the `irb` console (for features such as autocompletion). and [typeprof](https://github.com/ruby/typeprof). It relies on type definitions written into `.rbs` files. The `protobuf` library already exposes type definitions in [gem_rbs_collection](https://github.com/ruby/gem_rbs_collection/tree/main/gems/google-protobuf/3.22), which is used to source type definitions for libraries which do not want, or can't maintain type definitions themselves. (`protobuf` could arguably import these into this repository, lmk if you're interested). This should fix gaps such as better IDE integration, such as the ones described [here](#9495). The plan is to do roughly the same type of integration as was done for `.pyi` annotations, which also write to separate files: add the cli option and the rbs generator. protobuf classes in ruby rely on dynamic attribution of the base class, which makes what I'm trying to achieve a bit difficult. Ideally the type hierarchy could be specified statically in the ruby source code. ```ruby class Bar < AbstractMessage class Bar < Google::Protobuf::DescriptorPool.generated_pool.lookup("Bar").msgclass ``` ``` Closes #15633 FUTURE_COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 569f032 PiperOrigin-RevId: 820790992

zhangskz · 2025-10-20T22:35:54Z

Note this change required some changes on our side to handle merge conflicts / transforms / linters when importing.

Workflow changes in test_ruby.yml currently fail with:

ERROR: Skipping 'ruby': no such target '//ruby:ruby': target 'ruby' not declared in package 'ruby' defined by /workspace/ruby/BUILD.bazel
ERROR: no such target '//ruby:ruby': target 'ruby' not declared in package 'ruby' defined by /workspace/ruby/BUILD.bazel

https://github.com/protocolbuffers/protobuf/actions/runs/18666531950/job/53218943443?pr=24064

You'll likely want to at least use a more up-to-date Bazel 7 or 8 image e.g. us-docker.pkg.dev/protobuf-build/containers/test/linux/ruby:ruby-8.0.1-3.4-b77fdae6d4771789dfc66a56bf8d806354e8011a since Bazel 6 is out of support which might explain the current error.

Also noticing that the typecheck.gemfile.lock being copied in this image doesn't seem to exist, and bundle isn't actually available in this image (neither is gem install bundler). We use bundle in a few other GHA tests in this file but using other images. I'm not as familiar with ruby type checking and this test, but looks like the setup for typecheck action needs some work to include the right dependencies.

github-actions · 2026-01-19T10:09:58Z

We triage inactive PRs and issues in order to make it easier to find active work. If this PR should remain active, please add a comment.

This PR is labeled inactive because the last activity was over 90 days ago. This PR will be closed and archived after 14 additional days without activity.

qnighy · 2026-01-19T10:35:31Z

I hope it lands some day. Thank you for your efforts.

HoneyryderChuck · 2026-01-19T17:16:01Z

@qnighy thx, I hope so too. Something for @JasonLunn to answer. Thx for the initial work.

this introduces support for a new protoc option, `--rbs_out`, which points to a directory where ruby type definition files, defined in the RBS format, are stored. [rbs](https://github.com/ruby/rbs) is the type signature syntax blessed by the ruby core team, used by static analysis tools such as [steep](https://github.com/soutaro/steep), which integrates with VS Code, the `irb` console (for features such as autocompletion). and [typeprof](https://github.com/ruby/typeprof). It relies on type definitions written into `.rbs` files. The `protobuf` library already exposes type definitions in [gem_rbs_collection](https://github.com/ruby/gem_rbs_collection/tree/main/gems/google-protobuf/3.22), which is used to source type definitions for libraries which do not want, or can't maintain type definitions themselves. (`protobuf` could arguably import these into this repository, lmk if you're interested). This should fix gaps such as better IDE integration, such as the ones described [here](#9495). The plan is to do roughly the same type of integration as was done for `.pyi` annotations, which also write to separate files: add the cli option and the rbs generator. protobuf classes in ruby rely on dynamic attribution of the base class, which makes what I'm trying to achieve a bit difficult. Ideally the type hierarchy could be specified statically in the ruby source code. ```ruby class Bar < AbstractMessage class Bar < Google::Protobuf::DescriptorPool.generated_pool.lookup("Bar").msgclass ``` ``` Closes #15633 COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 FUTURE_COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 PiperOrigin-RevId: 792682000

.github/workflows/test_ruby.yml

JasonLunn · 2026-01-19T18:06:57Z

I took a swing at getting this landed back in August, but got derailed and then had to focus on other projects that have since been completed.

There is an internal process behind-the-scenes that has to be performed for any change that touches protoc, and that's why this has been stuck in the interim.

The only challenge I see right now is the new type checking test, which appears to require files that don't appear in the PR.

this introduces support for a new protoc option, `--rbs_out`, which points to a directory where ruby type definition files, defined in the RBS format, are stored. [rbs](https://github.com/ruby/rbs) is the type signature syntax blessed by the ruby core team, used by static analysis tools such as [steep](https://github.com/soutaro/steep), which integrates with VS Code, the `irb` console (for features such as autocompletion). and [typeprof](https://github.com/ruby/typeprof). It relies on type definitions written into `.rbs` files. The `protobuf` library already exposes type definitions in [gem_rbs_collection](https://github.com/ruby/gem_rbs_collection/tree/main/gems/google-protobuf/3.22), which is used to source type definitions for libraries which do not want, or can't maintain type definitions themselves. (`protobuf` could arguably import these into this repository, lmk if you're interested). This should fix gaps such as better IDE integration, such as the ones described [here](#9495). The plan is to do roughly the same type of integration as was done for `.pyi` annotations, which also write to separate files: add the cli option and the rbs generator. protobuf classes in ruby rely on dynamic attribution of the base class, which makes what I'm trying to achieve a bit difficult. Ideally the type hierarchy could be specified statically in the ruby source code. ```ruby class Bar < AbstractMessage class Bar < Google::Protobuf::DescriptorPool.generated_pool.lookup("Bar").msgclass ``` ``` Closes #15633 COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 FUTURE_COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 PiperOrigin-RevId: 792682000

HoneyryderChuck · 2026-01-21T10:29:37Z

@JasonLunn appreciate all the effort that you have put in helping move forward with this. This would probably have not been so close to be merged without your help in the last few months.

From my part, it's been 2y since this PR has been created (and it was itself based on a prior PR which had been lingering for a while). A lot changed since then, and I no longer work at the company that would have benefited from this, and with it, I have lost access to the setup that allowed me to troubleshoot this locally. It was a multi-day time-consuming effort that I no longer have the will to go through again (I only ever touched bazel for this project, and beyond the learning curve that I never quite got over, it was quite dev-unfriendly in a Mac).

Moreover, feedback has also been sparse through the whole timeline, and waiting multiple days for someone to manually unlock CI after every change only added to the frustration. I'm sure there were justifiable reasons for this due to the internal processes google enforces, so not looking at blaming individuals here, just that, all of this made my contribution way harder than it could have been. Hope this feedback helps you create internal processes more friendly to outside contributions like this one.

this introduces support for a new protoc option, `--rbs_out`, which points to a directory where ruby type definition files, defined in the RBS format, are stored. [rbs](https://github.com/ruby/rbs) is the type signature syntax blessed by the ruby core team, used by static analysis tools such as [steep](https://github.com/soutaro/steep), which integrates with VS Code, the `irb` console (for features such as autocompletion). and [typeprof](https://github.com/ruby/typeprof). It relies on type definitions written into `.rbs` files. The `protobuf` library already exposes type definitions in [gem_rbs_collection](https://github.com/ruby/gem_rbs_collection/tree/main/gems/google-protobuf/3.22), which is used to source type definitions for libraries which do not want, or can't maintain type definitions themselves. (`protobuf` could arguably import these into this repository, lmk if you're interested). This should fix gaps such as better IDE integration, such as the ones described [here](#9495). The plan is to do roughly the same type of integration as was done for `.pyi` annotations, which also write to separate files: add the cli option and the rbs generator. protobuf classes in ruby rely on dynamic attribution of the base class, which makes what I'm trying to achieve a bit difficult. Ideally the type hierarchy could be specified statically in the ruby source code. ```ruby class Bar < AbstractMessage class Bar < Google::Protobuf::DescriptorPool.generated_pool.lookup("Bar").msgclass ``` ``` Closes #15633 COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 FUTURE_COPYBARA_INTEGRATE_REVIEW=#15633 from HoneyryderChuck:ruby-rbs-integration 2167aa9 PiperOrigin-RevId: 792682000

JasonLunn · 2026-01-21T19:58:26Z

@HoneyryderChuck this has been merged, and should be included in the next RC candidate. Thank you for your contributions, your patience, and your feedback. We will strive to improve!

protoc v34.0 gained --rbs_out, type signature file for Ruby. https://github.com/protocolbuffers/protobuf/releases/tag/v34.0 protocolbuffers/protobuf#15633

HoneyryderChuck requested review from a team as code owners January 29, 2024 17:37

HoneyryderChuck requested review from ericsalo and removed request for a team January 29, 2024 17:37

mkruskal-google requested a review from haberman January 29, 2024 17:38

zhangskz mentioned this pull request Feb 5, 2024

Improve IDE support for Ruby #9495

Closed

zhangskz requested review from acozzette and removed request for a team and ericsalo February 5, 2024 16:47

acozzette removed their request for review February 6, 2024 21:49

zhangskz mentioned this pull request Feb 21, 2024

Add support for RBS, Ruby type signature #15881

Closed

haberman added the 🅰️ safe for tests Mark a commit as safe to run presubmits over label May 6, 2024

github-actions bot removed the 🅰️ safe for tests Mark a commit as safe to run presubmits over label May 6, 2024

HoneyryderChuck force-pushed the ruby-rbs-integration branch from dac2251 to a69f3e9 Compare May 10, 2024 15:47

HoneyryderChuck requested a review from a team as a code owner May 10, 2024 15:47

HoneyryderChuck changed the title ~~draft: ruby codegen: support generation of rbs files~~ ruby codegen: support generation of rbs files May 10, 2024

github-actions bot added the inactive Denotes the issue/PR has not seen activity in the last 90 days. label Jan 19, 2026

copybara-service bot mentioned this pull request Jan 19, 2026

ruby codegen: support generation of rbs files (#15633) #25406

Closed

JasonLunn removed the inactive Denotes the issue/PR has not seen activity in the last 90 days. label Jan 19, 2026

JasonLunn reviewed Jan 19, 2026

View reviewed changes

.github/workflows/test_ruby.yml Show resolved Hide resolved

JasonLunn added the wait for user action label Jan 19, 2026

copybara-service bot closed this in 6ebdf85 Jan 21, 2026

protobuf-team-bot added a commit that referenced this pull request Jan 21, 2026

Auto-generate files after PR #15633

a4f8ecc

sorah mentioned this pull request Feb 27, 2026

protoc_builtin: Support --rbs_out bufbuild/buf#4361

Open

Conversation

HoneyryderChuck commented Jan 29, 2024

Uh oh!

google-cla bot commented Jan 29, 2024

Uh oh!

HoneyryderChuck commented Jan 29, 2024

Uh oh!

acozzette commented Feb 6, 2024

Uh oh!

HoneyryderChuck commented Mar 27, 2024

Uh oh!

haberman commented May 6, 2024

Uh oh!

HoneyryderChuck commented May 9, 2024

Uh oh!

haberman commented May 9, 2024

Uh oh!

HoneyryderChuck commented May 10, 2024

Uh oh!

HoneyryderChuck commented May 10, 2024

Uh oh!

qnighy commented May 13, 2024

Uh oh!

haberman commented May 13, 2024

Uh oh!

HoneyryderChuck commented May 13, 2024

Uh oh!

haberman commented May 14, 2024

Uh oh!

HoneyryderChuck commented May 14, 2024

Uh oh!

zhangskz commented Oct 20, 2025

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

qnighy commented Jan 19, 2026

Uh oh!

HoneyryderChuck commented Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

JasonLunn commented Jan 19, 2026

Uh oh!

HoneyryderChuck commented Jan 21, 2026

Uh oh!

JasonLunn commented Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

HoneyryderChuck commented Jan 19, 2026 •

edited

Loading