Alright, digging in here. If I am understanding the intent of the benchmark, this is a clearer way to write it:

Benchmark code

require 'set'
require 'benchmark/memory'

class Retainer
  def initialize(container)
    @container = container
  end

  def retain!
    @container << { x: 1 }
  end
end

ra = Retainer.new([])
rs = Retainer.new(Set.new)

Benchmark.memory do |b|
  calls = 10_001

  b.report('using Array') do
    calls.times { ra.retain! }
  end

  b.report('using Set') do
    calls.times { rs.retain! }
  end

  b.compare!
end

Running this gives the following output:

Benchmark output

Calculating -------------------------------------
         using Array     2.320M memsize (     2.320M retained)
                        10.001k objects (    10.001k retained)
                         0.000  strings (     0.000  retained)
           using Set     2.321M memsize (   928.000  retained)
                        10.004k objects (     4.000  retained)
                         0.000  strings (     0.000  retained)

Comparison:
         using Array:    2320232 allocated
           using Set:    2320928 allocated - 1.00x more

If I'm understanding the source of the question, you were finding this confusing because the report is about the magnitude of allocations rather than the magnitude of retentions. In that case, I don't think that calling GC.start would do what you expect because it's already done as part of the measurement.

I think what I would like to do to address this is twofold:

Output a comparison of retentions as well as allocations
Allow you to use allocations OR retentions to sort the comparison

(1) would make the output less confusing by default and (2) would allow you to tune the tool to be better suited for testing memory leaks.

What do you think? Would those two changes have made the situation less confusing?

Should GC.start be run before reporting results? #12

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions