[PROF-15412] Import Ruby 4.0.6 headers and bump gem version to 3.5.3#28
Conversation
These are almost the same as the 4.0.0 headers, yet the change to add
`is_ready` in `rb_thread_sched_item` shifted the in-memory layouts of
things inside `rb_thread_struct` which broke the profiler.
```diff
$ diff -uw ruby-4.0.0-p0/ ruby-4.0.6-p0/
Common subdirectories: ruby-4.0.0-p0/ccan and ruby-4.0.6-p0/ccan
diff -uw ruby-4.0.0-p0/id_table.h ruby-4.0.6-p0/id_table.h
--- ruby-4.0.0-p0/id_table.h 2026-01-07 15:43:22.492338852 +0000
+++ ruby-4.0.6-p0/id_table.h 2026-07-15 09:07:09.200855897 +0100
@@ -47,6 +47,16 @@
extern const rb_data_type_t rb_managed_id_table_type;
+VALUE rb_marked_id_table_new(size_t capa);
+int rb_marked_id_table_insert(VALUE table, ID id, VALUE val);
+VALUE rb_marked_id_table_dup(VALUE table);
+
+// alisases
+#define rb_marked_id_table_size rb_managed_id_table_size
+#define rb_marked_id_table_lookup rb_managed_id_table_lookup
+#define rb_marked_id_table_foreach rb_managed_id_table_foreach
+#define rb_marked_id_table_foreach_values rb_managed_id_table_foreach_values
+
RUBY_SYMBOL_EXPORT_BEGIN
size_t rb_id_table_size(const struct rb_id_table *tbl);
RUBY_SYMBOL_EXPORT_END
Common subdirectories: ruby-4.0.0-p0/internal and ruby-4.0.6-p0/internal
Common subdirectories: ruby-4.0.0-p0/prism and ruby-4.0.6-p0/prism
diff -uw ruby-4.0.0-p0/thread_pthread.h ruby-4.0.6-p0/thread_pthread.h
--- ruby-4.0.0-p0/thread_pthread.h 2026-01-07 15:43:22.497873080 +0000
+++ ruby-4.0.6-p0/thread_pthread.h 2026-07-15 09:07:09.202007892 +0100
@@ -56,6 +56,9 @@
// connected to ractor->threads.sched.reqdyq
// locked by ractor->threads.sched.lock
struct ccan_list_node readyq;
+ // Indicates whether thread is on the readyq.
+ // There is no clear relationship between this and th->status.
+ bool is_ready;
// connected to vm->ractor.sched.timeslice_threads
// locked by vm->ractor.sched.lock
@@ -127,6 +130,11 @@
struct rb_thread_struct *lock_owner;
#endif
struct rb_thread_struct *running; // running thread or NULL
+ // Most recently running thread or NULL. If this thread wakes up before the newly running
+ // thread completes the transfer of control, it can interrupt and resume running.
+ // The new thread clears this field when it takes control.
+ struct rb_thread_struct *runnable_hot_th;
+ int runnable_hot_th_waiting;
bool is_running;
bool is_running_timeslice;
bool enable_mn_threads;
diff -uw ruby-4.0.0-p0/vm_core.h ruby-4.0.6-p0/vm_core.h
--- ruby-4.0.0-p0/vm_core.h 2026-01-07 15:43:22.497873080 +0000
+++ ruby-4.0.6-p0/vm_core.h 2026-07-15 09:07:09.202386196 +0100
@@ -763,6 +763,7 @@
const VALUE special_exceptions[ruby_special_error_count];
/* Ruby Box */
+ rb_box_t *master_box;
rb_box_t *root_box;
rb_box_t *main_box;
@@ -2233,6 +2234,7 @@
int rb_signal_exec(rb_thread_t *th, int sig);
void rb_threadptr_check_signal(rb_thread_t *mth);
void rb_threadptr_signal_raise(rb_thread_t *th, int sig);
+void rb_threadptr_interrupt_raise(rb_thread_t *th);
void rb_threadptr_signal_exit(rb_thread_t *th);
int rb_threadptr_execute_interrupts(rb_thread_t *, int);
void rb_threadptr_interrupt(rb_thread_t *th);
```
TonyCTHsu
left a comment
TonyCTHsu
left a comment
There was a problem hiding this comment.
btw, the doc is outdated. Perhaps use this pull request to update
You mean the "3.3+" part? I kinda read that as >= 3.3, e.g. anything after that Ruby, including Ruby 4. (I don't mind updating it if you still find it confusing, just clarifying what you're pointing out as wrong) |
|
So for curiosity it's because of ruby/ruby@21a2595 |
You are absolutely right! I am blind! |
Unfortunately this is the downside of depending on some internal APIs -- I guess this is our regular reminder that it's probably a good idea to ask for public APIs to access some of this information ;) Personally I think that backport is worth it, even if it's a bit outside of the usual norm :D
👀 👀 👀 Ack, I'll leave it as-is. |
|
Just to double-check:
|
Yes. The logic lives in https://github.com/DataDog/datadog-ruby_core_source/blob/master/lib/datadog/ruby_core_source.rb#L13 . You can see the logic working when you run with 4.0.5 with 4.0.6:
Yes, exactly. If you run those two steps, the result is the same output as this PR. |
|
Quick update:
|
**What does this PR do?** This PR bumps the minimum version of the datadog-ruby_core_source gem to the latest version (3.5.3), having the usual diff (e.g. DataDog#5278 was the previous update). **Motivation:** Doing this bump forces updates to the datadog gem to also pull in the latest version of the datadog-ruby_core_source gem. This ensures users have the best experience and don't e.g. run into issues because somehow they're running the latest version of the gem with an outdated version of the headers. Specifically version 3.5.3 adds support for Ruby 4.0.6, fixing DataDog#6045 . **Additional Notes:** N/A **How to test the change?** This will fix the "test-asan" check failing in master (e.g. https://github.com/DataDog/dd-trace-rb/actions/runs/29363868729 ) because test-asan is running on 4.0.6, so green CI is good. Also, I've locally tested with both Ruby 4.0.5 and 4.0.6 (see DataDog/datadog-ruby_core_source#28 ).
3 participants
What does this PR do?
This PR imports the Ruby 4.0.6 headers and bumps the gem version from 3.5.2 to 3.5.3 so we can make a new release.
Motivation:
The Ruby 4.0.6 headers are almost the same as the 4.0.0 headers (see below), yet the change to add
is_readyinrb_thread_sched_itemshifted the in-memory layouts of things insiderb_thread_structwhich broke the profiler, see DataDog/dd-trace-rb#6045 .Importing the new headers fixes the issue.
Additional Notes:
Here's the full diff between the 4.0.0 and 4.0.6 headers.
How to test the change?
I've locally tested this change by building the .gem, and then trying it with both Ruby 4.0.5 (picks up the 4.0.0 headers correctly) and Ruby 4.0.6 (picks up the new headers).
With this version, the profiler specs are back to green.