…#6)

…oxcaml#6)

…#6)

…#6)

Coercions only apply to their own domains - that is, a coercion on
closures doesn't apply to pairs (which might contain, say, a closure
and an int). So we don't want to recurse into types in general when
applying coercions. Unfortunately, the situation with closures is
... complicated, as explained in comments.

5e7dfce331 Merge flambda-backend changes
9f7f2d24a7 Edit script for test merge
1ae71ce59d Review comments (oxcaml#6)
1845365bd0 Layouts version 1
aba6294 Immediacy rework (oxcaml#122)
cf4eeef Add no-stack-allocation variant of some tests that print lambda (oxcaml#133)
8f22438 Fully switch over Jane Street Merlin support to `.local-*` (oxcaml#136)
5482a8d Remove `Lev_module_definition` from lambda (oxcaml#135)
261e016 Change modular extensions to produce `AST_desc` types (oxcaml#132)
0760c82 Disable module patterns in comprehensions (oxcaml#131)
6acac80 Add Ctype global state debug printers (oxcaml#130)
bc32037 Enable support for Jane Street's internal Merlin configuration (oxcaml#64)
d1a8d03 Split out `Clflags.Extension` into a new file, `Language_extension` (oxcaml#125)
435de6d Fix bootstrap and add legacy CI (oxcaml#126)
7e5a626 Improve the API of language extensions to better support upstream compatibility (and also tooling) (oxcaml#13)
c4e17b0 Replace var with local for faster mode checking (oxcaml#53)
6d477d8 Merge pull request oxcaml#124 from riaqn/merge-backend
d737533 minor fixes after merge
f1710d6 Merge flambda-backend changes
cc18534 Just run make boostrap (oxcaml#123)

git-subtree-dir: ocaml
git-subtree-split: 5e7dfce331d0e39c695fab9b00e3d2cda7d9ebb4

The toplevel printer detects cycles by keeping a hashtable of values
that it has already traversed.

However, some OCaml runtime types (at least bigarrays) may be
partially uninitialized, and hashing them at arbitrary program points
may read uninitialized memory. In particular, the OCaml testsuite
fails when running with a memory-sanitizer enabled, as bigarray
printing results in reads to uninitialized memory:

```
==133712==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x4e6d11 in caml_ba_hash /var/home/edwin/git/ocaml/runtime/bigarray.c:486:45
    oxcaml#1 0x52474a in caml_hash /var/home/edwin/git/ocaml/runtime/hash.c:251:35
    oxcaml#2 0x599ebf in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1065:14
    oxcaml#3 0x5a909a in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
    oxcaml#4 0x540ccb in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
    oxcaml#5 0x7f0910abb087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    oxcaml#6 0x7f0910abb14a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    oxcaml#7 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)

  Uninitialized value was created by a heap allocation
    #0 0x47d306 in malloc (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x47d306) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)
    oxcaml#1 0x4e7960 in caml_ba_alloc /var/home/edwin/git/ocaml/runtime/bigarray.c:246:12
    oxcaml#2 0x4e801f in caml_ba_create /var/home/edwin/git/ocaml/runtime/bigarray.c:673:10
    oxcaml#3 0x59b8fc in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1058:14
    oxcaml#4 0x5a909a in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
    oxcaml#5 0x540ccb in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
    oxcaml#6 0x7f0910abb087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    oxcaml#7 0x7f0910abb14a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    oxcaml#8 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /var/home/edwin/git/ocaml/runtime/bigarray.c:486:45 in caml_ba_hash
```

The only use of hashing in genprintval is to avoid cycles, that is, it
is only useful for OCaml values that contain other OCaml values
(including possibly themselves). Bigarrays cannot introduce cycles,
and they are always printed as "<abstr>" anyway.

The present commit proposes to be more conservative in which values
are hashed by the cycle detector to avoid this issue: we skip hashing
any value with tag above No_scan_tag -- which may not contain any
OCaml values.

Suggested-by: Gabriel Scherer <[email protected]>

Signed-off-by: Edwin Török <[email protected]>
Co-authored-by: Edwin Török <[email protected]>