Synchronized Pervasives implementation between stdlib and threads#943
Conversation
stdlib/pervasives.ml
Outdated
| external out_channel_length : out_channel -> int = "caml_ml_channel_size" | ||
| external close_out_channel : out_channel -> unit = "caml_ml_close_channel" | ||
| let close_out oc = flush oc; close_out_channel oc | ||
| let close_out oc = (try flush oc with _ -> ()); close_out_channel oc |
There was a problem hiding this comment.
Buffered data could be silently lost if flush fails. That makes this particular change seem dangerous.
There was a problem hiding this comment.
@hcarty Note that this patch only makes stdlib and threads consistent. In this case I chose the semantics as already implemented in threads, which I considered saner. It will close the channel for sure, otherwise we would be leaking file descriptors. If a user absolutely wants to make sure that their data has been flushed, they need to explicitly call flush themselves before calling close_out, thus potentially raising Sys_error exceptions. flush_all also suppresses such exceptions, because the user could not possibly know anyway which channel was broken.
There was a problem hiding this comment.
@mmottl Scope of a surrounding try/with would affect whether there is a leak or not here.
Regardless, this change should be reflected in the changelog and documentation to call out the requirement for an explicit call to flush to guarantee flushing of buffered output.
There was a problem hiding this comment.
@hcarty Ok, I have looked at the documentation and it actually already specifies that close_out may raise even while flushing and before closing. Users who don't like that behavior should use close_out_noerr, which completely ignores errors. This means that the previous implementation in threads was not compliant with the manual while the stdlib was.
I have just submitted a patch to make the current stdlib behavior the default with both implementations. They must definitely behave the same for that function. It would be very confusing for users if that wasn't the case. If some thread-related library needed the previous closing behavior, that code would have to be changed to use close_out_noerr. I have looked, but haven't seen any such code in the distribution. The semantics as specified in the manual should definitely prevail.
|
I'm worried about changes that make |
|
@gasche In this case I actually believe that it's the other way round: the Concerning |
|
I've just updated the initial comment describing the pull request to reflect changes wrt. |
|
I tend to agree that the threads library is probably in fact better tested. |
|
This may be true, but for all discrepancies pointed out above, |
|
This patch touches otherlibs/threads, which is for vmthreads. I think none of the code in this directory is used when you are using system threads. I would think that no one uses vmthreads these days so the code in otherlibs/threads is likely to be the less tested one. |
stdlib/pervasives.ml
Outdated
| ('a, 'b, 'c, 'd, 'e, 'f) format6 = "%identity" | ||
|
|
||
| let ( ^^ ) (Format (fmt1, str1)) (Format (fmt2, str2)) = | ||
| let (^^) (Format (fmt1, str1)) (Format (fmt2, str2)) = |
There was a problem hiding this comment.
You should fix threads/pervasives to add the spaces rather than remove them here.
|
(I made a mistake in my previous comment above; I was thinking that the threads library being referred to was systhreads, but it is not.) Is there a reason to keep the vmthreads library around? Does anyone use it? |
|
@mshinwell At the time the I guess the question boils down to whether there are still any platforms left that don't have POSIX-threads and are worth supporting. My guess is probably not. |
|
@xavierleroy Would you be happy removing vmthreads or not? |
|
Does the implementation of vmthreads make sense in the context of compiling the ocaml bytecode interpreter + runtime system to WebAssembly, for instance? |
|
You need to ask the greater OCaml user community to see if vmthreads are still in use. The only use I know of is in JoCaml, as a more lightweight alternative to system threads. But JoCaml isn't particularly active these days... Personally, it wouldn't break my heart to remove vmthreads altogether. System threads are enough of a mess already :-) |
|
I can ask on the caml-list. At worse I suppose that this could easily be split into a separate library. The only builtin support in the compiler seems to be adding @alainfrisch it seems that browsers are getting support for pthreads in javascript. I guess if this happens WebAssembly will get pthreads as well. In any case, lwt/async are much more popular and provides similar features, so it doesn't seem too bad to me to let anyone interested maintain it as a separate library. BTW, we'll we need the consurtium approval to drop support for vmthreads? |
alainfrisch
left a comment
There was a problem hiding this comment.
LGTM.
@mmottl Can you add a Changelog entry, marked as potentially breaking (mentioning the change of behavior for close_out in the thread version)?
afe7af7 to
105880b
Compare
|
@alainfrisch Thanks for the approval, Alain, and to all the reviewers. I have added the Changelog entry and rebased + squashed this branch wrt. the current trunk. |
b11eea1 flambda-backend: Introduce Import_info (ocaml#1036) bc5b135 flambda-backend: Fix `ocamlobjinfo` on flambda2 .cmx files (ocaml#1029) c8babbd flambda-backend: Compilation_unit optimisations (ocaml#1035) e8d3e22 flambda-backend: Use 4.14.0 opam switch for building (includes upgrading ocamlformat to 0.24.1) (ocaml#1030) eb14a86 flambda-backend: Port PR81 from ocaml-jst (ocaml#1024) 131bc12 flambda-backend: Merge ocaml-jst 2022-12-13 (ocaml#1022) 06c189a flambda-backend: Make stack allocation the default (ocaml#1013) 98debd5 flambda-backend: Initial support for value slots not of value kind (ocaml#946) deb1714 flambda-backend: Add is_last flag to closinfo words (ocaml#938) d07fce1 flambda-backend: Disable poll insertion in Configure (ocaml#967) 0f1ce0e flambda-backend: Regenerate ocaml/configure autoconf 2.69 (instead of 2.71) (ocaml#1012) 27132d8 flambda-backend: Fix for spurious typing error related to expanding through functor arguments (ocaml#997) 724fb68 flambda-backend: Use `Compilation_unit.t` instead of `Ident.t` for globals (ocaml#871) 396d5b8 flambda-backend: Add a test for frametable setup in natdynlinked libraries (ocaml#983) b73ab12 flambda-backend: Fix invocation of `caml_shared_startup` in native dynlink (ocaml#980) 7c7d75a flambda-backend: Fix split_default_wrapper which did not trigger anymore with flambda2 (ocaml#970) 8fb75bd flambda-backend: Port ocaml#11727 and ocaml#11732 (ocaml#965) fdb7987 flambda-backend: Fix include functor issue after 4.14 merge. (ocaml#948) 9745cdb flambda-backend: Print -dprofile/-dtimings output to stdout like 4.12 (ocaml#943) 5f51f21 flambda-backend: Merge pull request ocaml#932 from mshinwell/4.14-upgrade 841687d flambda-backend: Run make alldepend in ocaml/ (ocaml#936) 72a7658 flambda-backend: Remove reformatting changes only in dynlink/dune (preserving PR889 and adjusting to minimise diff) 6d758cd flambda-backend: Revert whitespace changes in dune files, to match upstream c86bf6e flambda-backend: Remove duplicate tests for polling 971dbeb flambda-backend: Testsuite fixes 32f8356 flambda-backend: Topeval fix for symbols patch befea01 flambda-backend: Compilation fixes / rectify merge faults a84543f flambda-backend: Merge ocaml-jst 8e65056 flambda-backend: Merge ocaml-jst 4d70045 flambda-backend: Remove filename from system frametable (amd64) (ocaml#920) 5e57b7d flambda-backend: Bugfix for runtime frame_descr logic for C frames (ocaml#918) 6423d5e flambda-backend: Merge pull request ocaml#914 from mshinwell/merge-ocaml-jst-2022-10-24 ead605c flambda-backend: Add a missing Extract_exception (ocaml#916) c8f1481 flambda-backend: Resolve conflicts and add specialise/specialised attributes to Builtin_attributes cf4d0d3 flambda-backend: Merge fixes (ocaml#21) c2f742f flambda-backend: Re-enable some tests for Flambda2 (ocaml#881) 3d38d13 flambda-backend: Long frames in frametable (ocaml#797) 85aec7b flambda-backend: Add loop attribute to Builtin_attributes c0f16e3 flambda-backend: Compilation fixes 90dea23 flambda-backend: Merge flambda-backend/main 5acc6ea flambda-backend: Fixes after merge e501946 flambda-backend: Merge ocaml-jst 115083b flambda-backend: Merge ocaml-jst 9943b2e flambda-backend: Revert "Revert "Transform tail-recursive functions into recursive continuations (ocaml#893)"" (ocaml#909) ce339f1 flambda-backend: Fix alloc modes and call kinds for overapplications (ocaml#902) e6a317c flambda-backend: Revert "Transform tail-recursive functions into recursive continuations (ocaml#893)" 853c488 flambda-backend: Transform tail-recursive functions into recursive continuations (ocaml#893) 5a977e4 flambda-backend: Fix missing End_region primitives on switch arms (ocaml#898) 7fa7f9d flambda-backend: Add missing dependencies to Dune files (ocaml#889) 3cd36f0 flambda-backend: Have Lambda `Pgetglobal` and `Psetglobal` take `Compilation_unit.t` (ocaml#896) 7565915 flambda-backend: [@poll error] attribute (ocaml#745) 9eb9448 flambda-backend: Backport the main safepoints PRs (ocaml#740) 689bdda flambda-backend: Add strict mode for ocamldep (ocaml#892) git-subtree-dir: ocaml git-subtree-split: b11eea1
* Create ocamlorg.global library Directory: src/global Library Module Name: Ocamlorg Library Dependency Name: ocamlorg.global Submodules: - Import - Url The goal of this library is to group in a single place stuff used across several libraries and have a single place for extending Stdlib. --------- Co-authored-by: Cuihtlauac ALVARADO <[email protected]>
While trying to add some other features, I noticed that the
Pervasivesimplementations between the standard library and its threads implementation had diverged. This patch merges the two implementations as much as possible and should make it easier to compare or diff them.The only changes relevant to code execution are the following:
float_of_bitsinthreadsnow uses the same external declaration as instdlib. There does not seem to be anything thread-specific there.bytes_lengthused the wrong primitive instdlib. It shouldn't really make a difference with the current implementation, but the patch makes it more future-proof.flush_allin thestdlibnow only catchesSys_errorexceptions. I believe only assertion failures could be raised otherwise and might go unnoticed without the change. Alternatively, we could also suppress all exceptions inthreads, but maybe it's better to be more specific.close_outinthreadssuppressed exceptions escapingflush, which violates the manual and is inconsistent withstdlib.Please let me know if you would prefer to merge the two implementations differently.