MNT: Make sure AppVeyor fails if tests fail - #9773

So my guess at the problem seems to be wrong. Next up: is matplotlib.test somehow wrong? Find out on our next exciting episode...

I've just checked, and in tests.py the right retcode (1) is being passed to sys.exit: https://ci.appveyor.com/project/dstansby/matplotlib/build/1.0.3/job/adnrjcr682b5wc8h#L1930

Adding in the if ($LastExitCode -ne 0) { $host.SetShouldExit($LastExitCode) } appveyor script seems to work: https://ci.appveyor.com/project/dstansby/matplotlib/build/1.0.4/job/t0d6sntdir5hwb8w#L1931

Not sure why that's necessary for us here when I've never needed it before on other projects...

Then again we're the only one I've used that has a magic script rather than running pytest. 😈

No idea either... Appveyor has always seemed a bit mysterious to me. Now all we need to do is fix the actual errors 😛

I have a guess, maybe somehow set errorlevel= breaks the thing? Try to replace it with cmd /c "exit /b 0"

@Kojoley I just finally understood your comment. Yeah, I think that's what's going on. Let's see if the next one fails now that I took that out.

Ok, that finally took care of it. Now to fix the stupid repr problems on Python 2.7...

So it looks like the lesson to be learned here is: Don't set errorlevel. Ever.

Or unset.

Looks like that's because we do a git clean...

Actually idk why we are making a conda build here. It takes about 30% of time and our conda recipe demands some strange versions of packages (like freetype 3.8, which is available only from conda-forge) and requires to pin anaconda-client to 1.6.3.

For anaconda-client, we actually 8 lines later say:

conda install --yes conda-build jinja2 anaconda-client

So the pinning doesn't do any good. 🙄

So what we're doing is validating an internal conda recipe for building the package--which is a packaging issue, not a matplotlib code one. We're in control of conda-forge/matplotlib-feedstock, so I don't see why we should be maintaining a separate copy here. IMO, we should probably just defer to conda-forge on that and nuke it from our repo.

Other things I now see to clean up, which I'll do once we figure out what we're doing with the conda recipe:

• We configure conda with always_yes set to true, yet we pass --yes everywhere
• We don't use the CONDA_PY variable
• We don't use the CONDA_NPY variable--was the idea to test against different numpy versions? Because all we're doing is building conda packages against different versions (and we don't run unit tests on those)
• TEST_ALL is set to "no" for all builds.
• TARGET_ARCH is always "x64", and duplicates what's in the platform setting
• There's a command to fix vcvars64.bat for Python 3.4...which we no longer test.

So the pinning doesn't do any good.

#9319 (comment)

So I also remove installing obvious-ci--it doesn't seem to be needed to get us to build on AppVeyor. This simplified the conda setup significantly, and eliminated the need to worry about install order and other things. I expect this to work now.

This shaves about 20 minutes off the AppVeyor runs, so that's good.

I went ahead and remove the outdated comments.

Looks like it uploading to codecov now, merging.

There seem to be a conflict, please backport manually