Skip to content

Implement Weak References in the GC#8113

Merged
peterzhu2118 merged 5 commits into
ruby:masterfrom
peterzhu2118:weak-ref-gc
Aug 25, 2023
Merged

Implement Weak References in the GC#8113
peterzhu2118 merged 5 commits into
ruby:masterfrom
peterzhu2118:weak-ref-gc

Conversation

@peterzhu2118

Copy link
Copy Markdown
Member

Comment thread gc.c
Comment thread method.h Outdated
#define METHOD_ENTRY_CACHED(me) ((me)->flags & IMEMO_FL_USER4)
#define METHOD_ENTRY_CACHED_SET(me) ((me)->flags |= IMEMO_FL_USER4)
#define METHOD_ENTRY_INVALIDATED(me) ((me)->flags & IMEMO_FL_USER5)
#define METHOD_ENTRY_INVALIDATED(me) (UNDEF_P((VALUE)(me)) || (me)->flags & IMEMO_FL_USER5)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @tenderlove since you were worried this may add overhead to checking entries, apparently benchmarks indicate they don't?

Another possibility we mentioned with @peterzhu2118 was to have two rb_gc_mark_weak, one that would set to Qundef on sweep, that the other that would set to 0.

That would however complexify the GC.

@peterzhu2118 peterzhu2118 Jul 25, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if setting it to 0 will change anything here sine instead UNDEF_P((VALUE)(me)) we have to check that (me) == 0.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hum, I was assuming whatever the default value is is already checked elsewhere. No?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CC is created with a non-null CME, so the code just assumes that the CME exists.

ruby/vm_eval.c

Line 400 in 99162de

cc_new(VALUE klass, ID mid, int argc, const rb_callable_method_entry_t *cme)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. Yeah I suppose != Qundef and != 0 should have similarish perf.

@peterzhu2118
peterzhu2118 force-pushed the weak-ref-gc branch 4 times, most recently from dfb2838 to 83e5dd0 Compare July 25, 2023 17:11
Comment thread gc.c Outdated
const struct rb_callcache *cc = (const struct rb_callcache *)obj;
// should not mark klass here
gc_mark(objspace, (VALUE)vm_cc_cme(cc));
rb_gc_mark_weak((VALUE *)&cc->cme_);

@wks wks Aug 8, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the klass field has weak reference semantics, too. Currently, the klass field is cleared by the obj_free function for classes (or whenever vm_cc_invalidate is called, including obj_free). However, the following line in gc_ref_update_imemo (introduced in 087a2de) may indicate that the klass filed may not always contain a valid object reference:

// excerpt fro gc_ref_update_imemo
if (moved_or_living_object_strictly_p(objspace, cc->klass) &&

It may be a result that some imemo:callcache instance are not invalidated when cc->klass dies.

We may add rb_gc_mark_weak((VALUE *)&cc->klass); and see if it ever contains a reference to a dead class.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. Right now we probably don't need it as the klass field is cleared when the callcache is invalidated. Since the callcache belongs to the klass, the klass should live at least as long as the callcache.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind the above comment. Ultimately I chose to mark the klass field as weak as well. I think that will fix the crashes on CI.

After 6dc15cc, during reference updating of compaction, we no longer directly set the klass of the CC as 0 when the CC has been invalidated. Therefore, we need mark it as weak so that it gets overwritten with a Qundef when the klass has been reclaimed.

@peterzhu2118
peterzhu2118 force-pushed the weak-ref-gc branch 5 times, most recently from b5c602b to 21d00dd Compare August 18, 2023 19:43
peterzhu2118 and others added 5 commits August 24, 2023 10:53
These functions manipulate darray without the possibility of triggering
GC, which is used for places that cannot trigger GC. These functions
crash when the allocation fails.
[Feature #19783]

This commit adds support for weak references in the GC through the
function `rb_gc_mark_weak`. Unlike strong references, weak references
does not mark the object, but rather lets the GC know that an object
refers to another one. If the child object is freed, the pointer from
the parent object is overwritten with `Qundef`.

Co-Authored-By: Jean Boussier <[email protected]>
[Feature #19783]

This commit adds stats about weak references to `GC.latest_gc_info`.
It adds the following two keys:

- `weak_references_count`: number of weak references registered during
  the last GC.
- `retained_weak_references_count`: number of weak references that
  survived the last GC.
@peterzhu2118
peterzhu2118 merged commit f5c8bda into ruby:master Aug 25, 2023
@peterzhu2118
peterzhu2118 deleted the weak-ref-gc branch August 25, 2023 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants