Skip to content

Comments

wasm: fix C realloc and optimize it a bit#4898

Merged
deadprogram merged 1 commit intodevfrom
fix-wasm-realloc
Oct 20, 2025
Merged

wasm: fix C realloc and optimize it a bit#4898
deadprogram merged 1 commit intodevfrom
fix-wasm-realloc

Conversation

@aykevl
Copy link
Member

@aykevl aykevl commented May 21, 2025

  • Do not use make([]byte, ...) to allocate, instead allocate a slice of pointers. This makes sure the precise GC will scan the contents of the allocation, since C could very well put pointers in there.
  • Simplify the map to use the pointer as the key and the size as the value, instead of storing the slices directly in the map.

I didn't really test this apart from running ./testdata/cgo. I'm not entirely sure how to properly test this. But if we start using libc_realloc for cabi_realloc I think @ydnar has some test cases on hand?

@aykevl aykevl requested a review from ydnar May 21, 2025 10:37
@aykevl aykevl mentioned this pull request May 21, 2025
Draft
@github-actions
Copy link

github-actions bot commented May 21, 2025 •
edited
Loading

Size difference with the dev branch:

Binary size difference 
 flash                          ram
 before   after   diff          before   after   diff
  18480   18480      0   0.00%    6236    6236      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/adafruit4650
  61676   61676      0   0.00%    6180    6180      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adt7410/main.go
   8848    8848      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/adxl345/main.go
  13584   13584      0   0.00%    6796    6796      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/amg88xx
   9012    9012      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/main.go
  11896   11896      0   0.00%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/apds9960/proximity/main.go
   9272    9272      0   0.00%    4752    4752      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/apa102/itsybitsy-m0/main.go
   7588    7588      0   0.00%    2320    2320      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/at24cx/main.go
   8096    8096      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bh1750/main.go
   7456    7456      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/blinkm/main.go
  71564   71564      0   0.00%    3656    3656      0   0.00% tinygo build -size short -o ./build/test.hex -target=pinetime     ./examples/bma42x/main.go
  64388   64388      0   0.00%    6196    6196      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmi160/main.go
  27736   27736      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp180/main.go
  64216   64216      0   0.00%    6220    6220      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/bmp280/main.go
  11956   11956      0   0.00%    4812    4812      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bmp388/main.go
   7872    7872      0   0.00%    3352    3352      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/sram/main.go
  21788   21788      0   0.00%    3556    3556      0   0.00% tinygo build -size short -o ./build/test.hex -target=bluepill ./examples/ds1307/time/main.go
  69576   69576      0   0.00%    6368    6368      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/ds3231/main.go
   4640    4640      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/easystepper/main.go
  70240   70240      0   0.00%    6980    6980      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/flash/console/spi
  67032   67032      0   0.00%    9020    9020      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/flash/console/qspi
   7312    7312      0   0.00%    2284    2284      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/gc9a01/main.go
  67912   67912      0   0.00%    6360    6360      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/i2c/main.go
  68360   68360      0   0.00%    6504    6504      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/gps/uart/main.go
   8432    8432      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/hcsr04/main.go
   5768    5768      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/customchar/main.go
   5808    5808      0   0.00%    2280    2280      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hd44780/text/main.go
  10432   10432      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/hd44780i2c/main.go
  14784   14784      0   0.00%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/hts221/main.go
  16220   16220      0   0.00%    2364    2364      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/hub75/main.go
  10400   10400      0   0.00%    6916    6916      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/basic
  11196   11196      0   0.00%    4876    4876      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/basic
  29792   29792      0   0.00%   38076   38076      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/pyportal_boing
  10432   10432      0   0.00%    6916    6916      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/scroll
  11280   11280      0   0.00%    4876    4876      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/ili9341/scroll
 263900  263900      0   0.00%   46772   46772      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/ili9341/slideshow
  11080   11080      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis3dh/main.go
  14140   14140      0   0.00%    6580    6580      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-33-ble ./examples/lps22hb/main.go
  26464   26464      0   0.00%    2328    2328      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/lsm303agr/main.go
  28136   28136      0   0.00%    6828    6828      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/lsm303dlhc/main.go
  12400   12400      0   0.00%    4788    4788      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/lsm6ds3/main.go
  10104   10104      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mag3110/main.go
   9244    9244      0   0.00%    4772    4772      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017/main.go
   9648    9648      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp23017-multiple/main.go
   9496    9496      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp3008/main.go
  70444   70444      0   0.00%    6196    6196      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mcp2515/main.go
  27516   27516      0   0.00%    3832    3832      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/microbitmatrix/main.go
  27332   27332      0   0.00%    5812    5812      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit-v2 ./examples/microbitmatrix/main.go
   7548    7548      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mma8653/main.go
   7452    7452      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/mpu6050/main.go
  76232   76232      0   0.00%    7452    7452      0   0.00% tinygo build -size short -o ./build/test.hex -target=p1am-100 ./examples/p1am/main.go
  14076   14076      0   0.00%    5416    5416      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/pca9685/main.go
   6280    6280      0   0.00%    3292    3292      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setbuffer/main.go
   4660    4660      0   0.00%    2284    2284      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/pcd8544/setpixel/main.go
  12460   12460      0   0.00%    5392    5392      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/seesaw/soil-sensor
  13720   13720      0   0.00%    5400    5400      0   0.00% tinygo build -size short -o ./build/test.hex -target=qtpy-rp2040 ./examples/seesaw/rotary-encoder
   2841    2841      0   0.00%     558     558      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino ./examples/servo
  15780   15780      0   0.00%    5464    5464      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico     ./examples/sgp30
   8224    8224      0   0.00%    6788    6788      0   0.00% tinygo build -size short -o ./build/test.hex -target=pybadge ./examples/shifter/main.go
  57808   57808      0   0.00%    3688    3688      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht3x/main.go
  57800   57800      0   0.00%    3696    3696      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/sht4x/main.go
  57808   57808      0   0.00%    3688    3688      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/shtc3/main.go
  12788   12788      0   0.00%    8364    8364      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-ble ./examples/ssd1306/
  13128   13128      0   0.00%    5344    5344      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao-rp2040 ./examples/ssd1306/
  13172   13172      0   0.00%    5344    5344      0   0.00% tinygo build -size short -o ./build/test.hex -target=thumby ./examples/ssd1306/
   5912    5912      0   0.00%    2284    2284      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ssd1331/main.go
   6820    6820      0   0.00%    2284    2284      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7735/main.go
   6716    6716      0   0.00%    2284    2284      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/st7789/main.go
  16864   16864      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/thermistor/main.go
  10132   10132      0   0.00%    4532    4532      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-bluefruit ./examples/tone
  10192   10192      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/tm1637/main.go
  12624   12624      0   0.00%    5404    5404      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/touch/capacitive
   9164    9164      0   0.00%    6780    6780      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/fourwire/main.go
  12692   12692      0   0.00%    6976    6976      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal ./examples/touch/resistive/pyportal_touchpaint/main.go
  15764   15764      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl53l1x/main.go
  14396   14396      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=itsybitsy-m0 ./examples/vl6180x/main.go
  24796   24796      0   0.00%   13720   13720      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840-sense ./examples/waveshare-epd/epd1in54/main.go
   6544    6544      0   0.00%    2324    2324      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13/main.go
   6180    6180      0   0.00%    2316    2316      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd2in13x/main.go
   6464    6464      0   0.00%    2324    2324      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/waveshare-epd/epd4in2/main.go
  27956   27956      0   0.00%   18476   18476      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/waveshare-epd/epd2in66b/main.go
   6976    6976      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/ws2812
   5622    5622      0   0.00%    9538    9538      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
  63096   63096      0   0.00%    5948    5948      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/is31fl3731/main.go
   1581    1581      0   0.00%     598     598      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino   ./examples/ws2812
   1056    1056      0   0.00%     180     180      0   0.00% tinygo build -size short -o ./build/test.hex -target=digispark ./examples/ws2812
  32204   32204      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/bme280/main.go
  16440   16440      0   0.00%    4724    4724      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/microphone/main.go
  11620   11620      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/buzzer/main.go
  12332   12332      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=trinket-m0 ./examples/veml6070/main.go
   6868    6868      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/simple/main.go
   8788    8788      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l293x/speed/main.go
   6844    6844      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/simple/main.go
   9192    9192      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/l9110x/speed/main.go
   7428    7428      0   0.00%    3324    3324      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-f103rb ./examples/shiftregister/main.go
   6796    6796      0   0.00%    2272    2272      0   0.00% '-xesppie' is not a recognized feature for this target (ignoring feature)
  13364   13364      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=circuitplay-express ./examples/lis2mdl/main.go
   9896    9896      0   0.00%    4764    4764      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/max72xx/main.go
  76816   76816      0   0.00%    6336    6336      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/dht/main.go
  37720   37720      0   0.00%    6048    6048      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8523/
  71212   71212      0   0.00%    6344    6344      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/alarm/
   7264    7264      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/clkout/
  70644   70644      0   0.00%    6340    6340      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/time/
  71036   71036      0   0.00%    6344    6344      0   0.00% tinygo build -size short -o ./build/test.hex -target=xiao ./examples/pcf8563/timer/
  14100   14100      0   0.00%    5368    5368      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/qmi8658c/main.go
  12632   12632      0   0.00%    5352    5352      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-rp2040 ./examples/pcf8591/
   8768    8768      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina260/main.go
  12376   12376      0   0.00%    4780    4780      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m0 ./examples/ina219/main.go
   9420    9420      0   0.00%    5248    5248      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-l432kc ./examples/aht20/main.go
  74080   74080      0   0.00%   10756   10756      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/sdcard/console/
  62144   62144      0   0.00%    8228    8228      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-m4 ./examples/i2csoft/adt7410/
  10400   10400      0   0.00%    6788    6788      0   0.00% tinygo build -size short -o ./build/test.elf -target=wioterminal ./examples/axp192/m5stack-core2-blinky/
  10800   10800      0   0.00%    5340    5340      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/xpt2046/main.go
  13252   13252      0   0.00%    4936    4936      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/sx126x/lora_rxtx/
  33352   33352      0   0.00%    6796    6796      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/ssd1289/main.go
  13020   13020      0   0.00%    6308    6308      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/irremote/main.go
  13832   13832      0   0.00%    5384    5384      0   0.00% tinygo build -size short -o ./build/test.hex -target=badger2040 ./examples/uc8151/main.go
  12312   12312      0   0.00%    5412    5412      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/scd4x/main.go
   8036    8036      0   0.00%    4748    4748      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=circuitplay-express ./examples/makeybutton/main.go
   9584    9584      0   0.00%    4764    4764      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ds18b20/main.go
 122176  122176      0   0.00%    8068    8068      0   0.00% tinygo build -size short -o ./build/test.hex -target=nucleo-wl55jc ./examples/lora/lorawan/atcmd/
  17860   17860      0   0.00%    7012    7012      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/as560x/main.go
  11680   11680      0   0.00%    5360    5360      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu6886/main.go
   7852    7852      0   0.00%    4740    4740      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ttp229/main.go
  69072   69072      0   0.00%    6892    6892      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ndir/main_ndir.go
  62388   62388      0   0.00%    3784    3784      0   0.00% tinygo build -size short -o ./build/test.hex -target=microbit ./examples/ndir/main_ndir.go
  65424   65424      0   0.00%    6252    6252      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 ./examples/ndir/main_ndir.go
  11128   11128      0   0.00%    5352    5352      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mpu9150/main.go
  13408   13408      0   0.00%    5388    5388      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/sh1106/macropad_spi
  10156   10156      0   0.00%    5824    5824      0   0.00% tinygo build -size short -o ./build/test.hex -target=macropad-rp2040 ./examples/encoders/quadrature-interrupt
  68576   68576      0   0.00%    6860    6860      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/mcp9808/main.go
  44192   44192      0   0.00%    7144    7144      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=pico ./examples/tmc2209/main.go
  15092   15092      0   0.00%    5348    5348      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/tmc5160/main.go
  12472   12472      0   0.00%    4544    4544      0   0.00% tinygo build -size short -o ./build/test.uf2 -target=nicenano ./examples/sharpmem/main.go
  60640   60640      0   0.00%    5968    5968      0   0.00% tinygo build -size short -o ./build/test.hex -target=feather-nrf52840 ./examples/max6675/main.go
  14748   14748      0   0.00%    5408    5408      0   0.00% tinygo build -size short -o ./build/test.hex -target=pico ./examples/ens160/main.go
  88224   88224      0   0.00%    7260    7260      0   0.00% tinygo build -size short -o ./build/test.hex -target=challenger-rp2040 ./examples/net/ntpclient/
 347056  347056      0   0.00%   16764   16764      0   0.00% tinygo build -size short -o ./build/test.hex -target=pyportal -stack-size 8kb ./examples/net/http-get/
 119928  119928      0   0.00%    8020    8020      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-nano33 -stack-size 8kb ./examples/net/tcpclient/
 308248  308248      0   0.00%   15924   15924      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/websocket/dial/
 103568  103568      0   0.00%   10000   10000      0   0.00% tinygo build -size short -o ./build/test.hex -target=metro-m4-airlift -stack-size 8kb ./examples/net/socket/
 388420  388420      0   0.00%   18772   18772      0   0.00% tinygo build -size short -o ./build/test.hex -target=matrixportal-m4 -stack-size 8kb ./examples/net/webstatic/
 166364  166364      0   0.00%    9992    9992      0   0.00% tinygo build -size short -o ./build/test.hex -target=arduino-mkrwifi1010 -stack-size 8kb ./examples/net/tlsclient/
 155780  155780      0   0.00%    8812    8812      0   0.00% tinygo build -size short -o ./build/test.hex -target=nano-rp2040 -stack-size 8kb ./examples/net/mqttclient/natiu/
 116732  116732      0   0.00%   13296   13296      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webclient/
 337980  337980      0   0.00%   21816   21816      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/webserver/
 338128  338128      0   0.00%   21740   21740      0   0.00% tinygo build -size short -o ./build/test.hex -target=wioterminal -stack-size 8kb ./examples/net/mqttclient/paho/
 174588  174588      0   0.00%   13840   13840      0   0.00% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2040 -stack-size 8kb ./examples/net/tlsclient/
 120264  120264      0   0.00%   11864   11864      0   0.00% tinygo build -size short -o ./build/test.hex -target=elecrow-rp2350 -stack-size 8kb ./examples/net/ntpclient/
6254988 6254988      0   0.00%  963434  963434      0   0.00%

@ydnar
Copy link
Contributor

ydnar commented May 21, 2025

The Component Model never calls free on these allocations. We'd need some way to assert ownership of them so the GC can free them later.

@deadprogram
Copy link
Member

deadprogram commented May 21, 2025

Looks like real test failure here https://github.com/tinygo-org/tinygo/actions/runs/15159995695/job/42637182590#step:17:28

@ydnar
Copy link
Contributor

ydnar commented May 21, 2025

The Component Model never calls free on these allocations. We'd need some way to assert ownership of them so the GC can free them later.

It looks like we can have the generated code call runtime.libc_free on any allocated pointers it knows about, since libc_free doesn’t actually free the memory. I think I can wire this up into package cm with a small TinyGo-specific code path.

@aykevl would it help to have libc_free call some inner function release or something, that explicitly doesn’t free the memory, doing what it does now, allowing the GC to clean it up later?

@aykevl
Copy link
Member Author

aykevl commented May 21, 2025

Huh, this is weird. This PR increases binary size by a large amount for wasm. For example, testdata/alias.go went from 17kb to 49kb. I don't know why this PR causes such a large increase, but I am pretty sure that we wouldn't have this problem if malloc and free weren't exported by default on wasm.

@aykevl
Copy link
Member Author

aykevl commented May 21, 2025

The Component Model never calls free on these allocations. We'd need some way to assert ownership of them so the GC can free them later.

@ydnar so the host calls cabi_realloc (implicitly claiming ownership of the memory) and then at some point in the future transfers ownership to the guest/Go size? Is that correct? (Or at least the spec says something like "we won't use the memory after this point"?)

@ydnar
Copy link
Contributor

ydnar commented May 21, 2025

Transfer is implicit. There is not a specific call for each pointer that transfers ownership.

The component model does specify a post-call mechanism for exported functions to allow the host to free allocs.

This is where we can call libc_free.

@ydnar
Copy link
Contributor

ydnar commented May 22, 2025

Huh, this is weird. This PR increases binary size by a large amount for wasm. For example, testdata/alias.go went from 17kb to 49kb. I don't know why this PR causes such a large increase, but I am pretty sure that we wouldn't have this problem if malloc and free weren't exported by default on wasm.

Can this split the implementation from the //export?

@hb9cwp hb9cwp mentioned this pull request May 23, 2025
Merged
@ydnar
Copy link
Contributor

ydnar commented May 25, 2025

@aykevl for this to support the Component Model, I think we need a function like libc_free that:

  1. Removes the pointer from the allocs map.
  2. Does not panic if it’s an unknown pointer.

Can it be something like runtime.Release(ptr unsafe.Pointer) unsafe.Pointer that returns nil if the pointer was unknown?

@dgryski
Copy link
Member

dgryski commented May 27, 2025

I'll try to help out here.

@deadprogram
Copy link
Member

deadprogram commented Jun 9, 2025

It would be very good to get this fix, or some version of it, merged in time for the next release... 😸

@deadprogram
Copy link
Member

deadprogram commented Jun 11, 2025

I think this PR will pass CI with this change:

diff --git a/builder/sizes_test.go b/builder/sizes_test.go
index 931e9970..e4d5396a 100644
--- a/builder/sizes_test.go
+++ b/builder/sizes_test.go
@@ -81,8 +81,9 @@ func TestSizeFull(t *testing.T) {
                "wasip1",
        }
 
-       libMatch := regexp.MustCompile(`^C [a-z -]+$`) // example: "C interrupt vector"
-       pkgMatch := regexp.MustCompile(`^[a-z/]+$`)    // example: "internal/task"
+       libMatch := regexp.MustCompile(`^C [a-z -]+$`)      // example: "C interrupt vector"
+       goMatch := regexp.MustCompile(`^Go [a-z -]+$`)      // example: "Go interface method"
+       pkgMatch := regexp.MustCompile(`^[a-z][a-z/0-9]+$`) // example: "internal/task"
 
        for _, target := range tests {
                target := target
@@ -106,6 +107,9 @@ func TestSizeFull(t *testing.T) {
                                if libMatch.MatchString(pkg) {
                                        continue
                                }
+                               if goMatch.MatchString(pkg) {
+                                       continue
+                               }
                                if pkgMatch.MatchString(pkg) {
                                        continue
                                }

@aykevl
Copy link
Member Author

aykevl commented Jun 11, 2025

@deadprogram it would, but that's not the problem. The problem is that it increases binary size for wasm substantially. I'll need to investigate why that is.

@deadprogram
Copy link
Member

deadprogram commented Jun 11, 2025

The problem is that it increases binary size for wasm substantially. I'll need to investigate why that is.

Thanks for the clarification.

@aykevl aykevl force-pushed the fix-wasm-realloc branch from 8a49a78 to 63c39af Compare June 13, 2025 10:47
@aykevl
Copy link
Member Author

aykevl commented Jun 13, 2025

I think I fixed the issue. The problem was that unsafe.Pointer was not treated like all the other pointers and forced map lookups using reflection. This fix might also optimize other code, we'll see.

@aykevl
Copy link
Member Author

aykevl commented Jun 13, 2025

Also, I optimized the code slightly by calling alloc directly instead of doing the weird round-up-and-divide to allocate a slice of pointers. The code is simpler now.

@deadprogram
Copy link
Member

deadprogram commented Jun 13, 2025

@aykevl still error in this PR failing CI:

panic: cannot convert pointer value to byte

@deadprogram
Copy link
Member

deadprogram commented Aug 14, 2025

@aykevl any chance of getting back to this PR soon? I think it is kinda needed...

@aykevl
Copy link
Member Author

aykevl commented Oct 3, 2025

Rebased since it's been a while and I want to see why CI fails (logs were expired).

@aykevl
Copy link
Member Author

aykevl commented Oct 3, 2025

Removed the first commit that was causing problems.

@aykevl
Copy link
Member Author

aykevl commented Oct 3, 2025

Actually no I think it is needed.

@aykevl aykevl marked this pull request as draft October 3, 2025 09:00
@aykevl
wasm: fix C realloc and optimize it a bit
cdb0a71 
  - Do not use make([]byte, ...) to allocate, instead call the allocator
    directly with a nil (undefined) layout. This makes sure the precise
    GC will scan the contents of the allocation, since C could very well
    put pointers in there.
  - Simplify the map to use the pointer as the key and the size as the
    value, instead of storing the slices directly in the map.
@aykevl aykevl marked this pull request as ready for review October 3, 2025 09:21
@aykevl
Copy link
Member Author

aykevl commented Oct 3, 2025

Ok, this should do it for now. Not pretty, but fixing the interp issue is more work. We can do that later.

@deadprogram
Copy link
Member

deadprogram commented Oct 3, 2025

@dgryski @ydnar anyone else WASM oriented please take a look.

@deadprogram
Copy link
Member

deadprogram commented Oct 11, 2025

Last call for comments! 😸

@deadprogram
Copy link
Member

deadprogram commented Oct 20, 2025

OK, now merging. Thanks for working on this @aykevl

@deadprogram deadprogram merged commit 26ee3fb into dev Oct 20, 2025
31 of 32 checks passed
@deadprogram deadprogram deleted the fix-wasm-realloc branch October 20, 2025 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ydnar ydnar Awaiting requested review from ydnar

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@aykevl @ydnar @deadprogram @dgryski