There are a couple things that worry me about using Travis CI.

Subjective worries:

• this is one more proprietary service to rely on
• it pains me a bit to think that around twenty minutes of CPU time are spent compiling OCaml again each time a commit is pushed or a pull request is sent. I know that the OCaml community does not pay for this waste energy, but I still find it a bit problematic. On the other hand, compiling OCaml from scratch is a reasonably sensible use of CPU time (other travis setups that first install opam and a load of dependencies that should obviously be cached in a better-designed CI setup pains me more in this regard).

These two objections should be understood in the following context (which is clear to Anil but maybe not to everyone else): there already is a CI service running on the OCaml SVN repository. It is kindly administered by INRIA, is a bit more painful to get information from (you have to explicitly subscribe; this is open to everyone but there is an entrance fee in terms of effort), relies on Jenkins, runs only on trunk, but thanks to heroic work by Damien has both Win and OSX slaves that are very useful (they catch most of the minor troubles, and in the case of OSX some important stack-alignment one).

Objective worries:

• for now the Travis build does stricly less than the INRIA CI, so it's utility is not obvious. I understand that the long-term goal is to make it cover more areas (and maybe rely entirely on it, to lift a burden off Damien's shoulders), but so far the returns of one more CI should be about nil, with one exception: it would run the testsuite on pull requests.
• the actual configuration does not only build the ocaml repository, but also Camlp4, Opam and Utop. This is both a good and a bad thing: Camlp4 is excellent at catching bugs/breakage in the compiler and I lament its loss (from the core distribution) in term of compiler code coverage. However, as of last it has been consistently broken for trunk, and my understanding of its development model is that it will try to get up-to-date relatively shortly before the release. What will the value of CI be if it is always red during 70% of a version's timeframe?

I think you're overthinking a relatively minor addition to the repository (worrying about spending CPU time building every changeset? seriously?). To address some specific concerns:

• The Travis files also work in forks, which the INRIA CI doesn't.
• The CI results here are triageable by anyone as part of the GH workflow, whereas the INRIA one is harder to get to. More tests are always better, but more triage is even better.
• As we're entering a release cycle, the goal is to keep all the moving parts green, not red. The whole point of the CI is to find and fix camlp4 mismatches quickly!
• It's easier to hook Travis results into OPAM (e.g. to have a known-good compiler switch for trunk and camlp4).

-anil

On 1 May 2014, at 10:51, gasche [email protected] wrote:

There are a couple things that worry me about using Travis CI.

Subjective worries:

this is one more proprietary service to rely on
it pains me a bit to think that around twenty minutes of CPU time are spent compiling OCaml again each time a commit is pushed or a pull request is sent. I know that the OCaml community does not pay for this waste energy, but I still find it a bit problematic. On the other hand, compiling OCaml from scratch is a reasonably sensible use of CPU time (other travis setups that first install opam and a load of dependencies that should obviously be cached in a better-designed CI setup pains me more in this regard).
These two objections should be understood in the following context (which is clear to Anil but maybe not to everyone else): there already is a CI service running on the OCaml SVN repository. It is kindly administered by INRIA, is a bit more painful to get information from (you have to explicitly subscribe; this is open to everyone but there is an entrance fee in terms of effort), relies on Jenkins, runs only on trunk, but thanks to heroic work by Damien has both Win and OSX slaves that are very useful (they catch most of the minor troubles, and in the case of OSX some important stack-alignment one).

Objective worries:

for now the Travis build does stricly less than the INRIA CI, so it's utility is not obvious. I understand that the long-term goal is to make it cover more areas (and maybe rely entirely on it, to lift a burden off Damien's shoulders), but so far the returns of one more CI should be about nil, with one exception: it would run the testsuite on pull requests.
the actual configuration does not only build the ocaml repository, but also Camlp4, Opam and Utop. This is both a good and a bad thing: Camlp4 is excellent at catching bugs/breakage in the compiler and I lament its loss (from the core distribution) in term of compiler code coverage. However, as of last it has been consistently broken for trunk, and my understanding of its development model is that it will try to get up-to-date relatively shortly before the release. What will the value of CI be if it is always red during 70% of a version's timeframe?
—
Reply to this email directly or view it on GitHub.

The goal of Travis CI is primarly to get a quick and easy feedback (portability information and a vague idea of the possible breakage cone) to the one who created the patch.

Having an internal infrastructure to test trunk only is good for other purposes but does not help at all here.

More testing is always good. I expect that at some future time the CI done
at OCaml Labs will subsume the CI done at Inria, at which time the latter
can be retired. For the time being the two CI's are well worth having.

2014-05-01 12:04 GMT+02:00 Thomas Gazagnaire [email protected]:

The goal of Travis CI is primarly to get a quick and easy feedback
(portability information and a vague idea of the possible breakage cone) to
the one who created the patch.

Having an internal infrastructure to test trunk only is good for
other purposes but does not help at all here.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/47#issuecomment-41896919
.

Magister dixit. I have committed them.
Anil, please close this.

This will require some minor tweaks for the latest bits to pass, mainly due to OPAM 1.1 not building against 4.02.0dev. I'll make another pull request as soon as ocaml/opam#1367 (turn off warn-error) is applied.