Skip to content

Allow parallel build of native and bytecode stubs of the systhreads library.#811

Merged
alainfrisch merged 1 commit intoocaml:trunkfrom
shindere:improve-systhreads-buildsystem
Sep 14, 2016
Merged

Allow parallel build of native and bytecode stubs of the systhreads library.#811
alainfrisch merged 1 commit intoocaml:trunkfrom
shindere:improve-systhreads-buildsystem

Conversation

@shindere
Copy link
Contributor

This PR is a follow-up to #785 and to the discussion with @alainfrisch
that took place there.

@alainfrisch
Copy link
Contributor

Can we use the -o flag to choose different targets with the same source file? Alternatively, a "cp" before calling the compiler would be a bit lighter than the include hack, no?

@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 04:50 -0700):

Can we use the -o flag to choose different targets with the same
source file?

I don't know yet whether msvc supports it.

Alternatively, a "cp" before calling the compiler would be a bit
lighter than the include hack, no?

Did you read the comment in the commit log? What do you think about it?

@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 04:50 -0700):

Can we use the -o flag to choose different targets with the same
source file?

Okay so I now know that -o does not work with msvc.

As I said in the commit log, I chose the inclusion trick so that it is
easy, if necessary, to add backend-specific code. I realise it is also
possible by means of ifdefs. It's just that I do not like copying files
so much because it also means not forgetting to remove them when doing
the make clean.

So all in all, although I don't find the include trick especially
elegant, I have to say it is the solution I find the most satisfactory.
But I can change if another solution is preferred.

@shindere shindere force-pushed the improve-systhreads-buildsystem branch 2 times, most recently from 53abe7b to da90d0d Compare September 14, 2016 14:28
@alainfrisch
Copy link
Contributor

Okay so I now know that -o does not work with msvc.

But we are actually calling ocamlc to compile the C code, right? I think the best approach would be to ensure that ocamlc -c -o foo.obj foo.c works even with MSVC ports (and it is certainly possible to do so with /Fo). There has been a lot of back-and-forth to support -o in OCaml compilers (even for compiling C code) and I don't know what the current state is, but my preference would be to use the clean solution.

About allowing potentially different code in bytecode vs native: there are already many differences between the two cases in the existing code, currently managed by conditional compilation as in many other places of the runtime system. The differences are local enough to justify this approach. So I'm not sure this argument can justify the addition of the two source files.

@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 07:30 -0700):

Okay so I now know that -o does not work with msvc.

But we are actually calling ocamlc to compile the C code, right?

No. What is being used currently is $(BYTECC) and $(NATIVECC) which
refer to the C compiler directly and never go through ocamlc.

I think the best approach would be to ensure that ocamlc -c -o foo.obj foo.c works even with MSVC ports (and it is certainly
possible to do so with /Fo).

I just tried, it does not work here.

To be more precise, the command used by the Makefile (with this PR
applied) is:

cl -nologo -D_CRT_SECURE_NO_DEPRECATE -I../../byterun -O2 -Gy- -MD -Ox -c st_stubs_b.c

which works.

After having removed the object files, the command:

cl -nologo -D_CRT_SECURE_NO_DEPRECATE -I../../byterun -O2 -Gy- -MD -Ox /Fo st_stubs_b.obj -c st_stubs.c

produces the following message:

cl : Command line warning D9027 : source file 'st_stubs_b.obj' ignored

and creates a st_stubs.obj file.

There has been a lot of back-and-forth to support -o in OCaml
compilers (even for compiling C code) and I don't know what the
current state is, but my preference would be to use the clean
solution.

I quickly went through utils/ccomp.ml and couldn't see any option passed
to the C compiler to specify its output file. But perhaps I missed
something?

I agree the proposed solution is not "clean", but at the same time I'd
say it's still cleaner than what was done before so IMHO it is still an
improvement.

About allowing potentially different code in bytecode vs native: there
are already many differences between the two cases in the existing
code, currently managed by conditional compilation as in many other
places of the runtime system.

Yes, I agree. I mentionned it.

So to conclude, I don't think it is possible to specify the output file
when compiling a C source file through ocamlc, currently.
However, if somebody knows how to tell msvc the name of the object
file it should produce when compiling a C source file, then we
could add a Makefile CCOUTPUT variable which could be defined to "-o"
for gcc and to the right thing for msvc and that would suffice, I guess.

@shindere
Copy link
Contributor Author

Sébastien Hinderer (2016/09/14 17:09 +0200):

After having removed the object files, the command:

cl -nologo -D_CRT_SECURE_NO_DEPRECATE -I../../byterun -O2 -Gy- -MD -Ox /Fo st_stubs_b.obj -c st_stubs.c

produces the following message:

cl : Command line warning D9027 : source file 'st_stubs_b.obj' ignored

and creates a st_stubs.obj file.

There is actually no space between /Fo an the object file name.

So this command works:

cl -nologo -D_CRT_SECURE_NO_DEPRECATE -I../../byterun -O2 -Gy- -MD -Ox /Fost_stubs_b.obj -c st_stubs.c

@alainfrisch
Copy link
Contributor

I was about to suggest removing the space after /Fo. So, if this works, do you agree we should simply do this?

@alainfrisch
Copy link
Contributor

couldn't see any option passed to the C compiler to specify its output file.

This should have been implemented as part as http://caml.inria.fr/mantis/view.php?id=6475 .

Would it make sense to compile with "ocamlc" instead of directly calling the C compiler? (That would avoid having to deal with -o vs /Fo).

…ibrary.

Before this commit, it was not possible to build the object files for
st_stubs_b and st_stubs_n in parallel, because both builds were performed
by first building an st_stubs object file and then renaming it, resulting
in a filename clash in case of parallel build.

This commit fixes this situation by giving the C compiler the right options
so that object files are created with the right name from the beginning
and do not need to be renamed afterwards.
@shindere shindere force-pushed the improve-systhreads-buildsystem branch from da90d0d to 3bf7ad9 Compare September 14, 2016 16:10
@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 08:45 -0700):

couldn't see any option passed to the C compiler to specify its output file.

This should have been implemented as part as
http://caml.inria.fr/mantis/view.php?id=6475 .

I believe it does not work.

I created a toy hello.c and tried this from OCaml's source directory:

./boot/ocamlrun ./ocamlc -verbose -o hello2.o -c hello.c

  • gcc -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC -c -I'/usr/local/lib/ocaml' 'hello.c'

And with the -c and -o option reversed:

./boot/ocamlrun ./ocamlc -verbose -c hello.c -o hello2.o

  • gcc -std=gnu99 -O2 -fno-strict-aliasing -fwrapv -Wall -D_FILE_OFFSET_BITS=64 -D_REENTRANT -fPIC -c -I'/usr/local/lib/ocaml' 'hello.c'

Would it make sense to compile with "ocamlc" instead of directly
calling the C compiler? (That would avoid having to deal with -o vs
/Fo).

Even if it would work, I am not sure. One would have to make sure all
the options passed by ocamlc are okay and also one would have to add
-ccopt in front of the options, I know there is a make function to do
that but not sure it would result in a better solution.

The updated version of the PR seems to work.

@alainfrisch
Copy link
Contributor

But aren't the same options passed automatically by "ocamlc -c foo.c" (at least for the bytecode case)?

@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 09:31 -0700):

But aren't the same options passed automatically by "ocamlc -c foo.c"
(at least for the bytecode case)?

ocamlc -c passes more options I'd say.

Well, given that it currently does not work, I think it's not an option.

Moreover, AFAIK, in the OCaml libraries that are included in the
distribution, the current way things are done is to call the C compiler
directly so it's perhaps not the role of this PR to change this for one
library, even if it would have simplified things. But again, it does not
seem to work... :)

@shindere
Copy link
Contributor Author

Alain Frisch (2016/09/14 08:40 -0700):

I was about to suggest removing the space after /Fo. So, if this
works, do you agree we should simply do this?

Yes, definitely. It's what is currently implemented.

@alainfrisch alainfrisch merged commit 4f19220 into ocaml:trunk Sep 14, 2016
@shindere shindere deleted the improve-systhreads-buildsystem branch September 15, 2016 08:04
camlspotter pushed a commit to camlspotter/ocaml that referenced this pull request Oct 17, 2017
…ibrary. (ocaml#811)

Before this commit, it was not possible to build the object files for
st_stubs_b and st_stubs_n in parallel, because both builds were performed
by first building an st_stubs object file and then renaming it, resulting
in a filename clash in case of parallel build.

This commit fixes this situation by giving the C compiler the right options
so that object files are created with the right name from the beginning
and do not need to be renamed afterwards.
stedolan pushed a commit to stedolan/ocaml that referenced this pull request Sep 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants