https://freenode.logbot.info/saltstack-formulas/20190516#c2180979:

-	-	-
09:59	myii	@​gtmanfred A problem is beginning to manifest: as we convert more formulas to semantic-release (and generally get inspec testing working on formulas), our Travis queuing is slowing things down. It only appears to handle 3 jobs at a time; the most I've ever seen is 5, but that may have been on my own forks. So it's not uncommon to see 3/50 active jobs and when things get bad, it can take up to an hour to get results.I'm usually manually cancelling and rerunning (my own) jobs to minimise loads but contributors won't be able to do that. This load will only increase in time, as more formulas get converted. We could start limiting the number of jobs per formula but is there anything else that can be done?
10:06	slack1872	except paying for more concurrent builds, I'm not sure...
10:06	myii	I reckon we're going to have to start limiting testing per formula.
10:07	myii	Unless a formula has specific tests that need to be run on specific combinations of platform/salt-version.
10:08	slack1872	Same problem always pops: you had tests (which is great) and then your tests take too much time...We have the same problem @ work, with 8 kitchen VM created and highstate taking near 15 min. But on Jenkins at least we can run VM in parallel
10:09	myii	What do you say to this idea:
10:09	myii	5 instances max, one from each os.
10:10	myii	2019.2 x2, 2018.3 x2 and 2017.7 x1.
10:12	myii	We've got a nice set to choose from but now this is about how to select which ones to use per formula.
10:12	myii	Is 5 sufficient for testing purposes?

https://freenode.logbot.info/saltstack-formulas/20190516#c2181226:

-	-	-
12:39	gtmanfred	no, limit concurrent jobs
12:39	gtmanfred	in travis
12:39	gtmanfred	you can have as many runs as you want, but it will only run one per repo at a time

https://freenode.logbot.info/saltstack-formulas/20190516#c2181237:

-	-	-
12:41	gtmanfred	if that doesn’t work, we can look into moving to azure pipelines
12:41	gtmanfred	i have been playing with that recently and it is pretty good
12:41	myii	Excellent.
12:41	gtmanfred	not as easy to build a matrix automatically for, but still pretty easy
12:42	myii	Look forward to it.
12:43	gtmanfred	Here is a version that builds the matrix https://github.com/saltstack/pepper/blob/develop/azure-pipelines.yml
12:43	myii	Looks decent.
12:43	gtmanfred	and here is a version where you just define it https://github.com/gtmanfred/pepper/blob/f15ce586a02b377ef1f8bdb48e378c5505179074/azure-pipelines.yml
12:44	myii	The loop is nicer.
12:44	gtmanfred	agreed
12:44	gtmanfred	one thing that it does not have is allowed_failures
12:44	gtmanfred	and the yaml can be gross
12:45	myii	I don't think we're really using that.
12:45	myii	So it shouldn't be a problem.
12:45	gtmanfred	There is also this limitation with the yaml https://twitter.com/resticbackup/status/1117327446456655873
12:45	gtmanfred	it does not parse whole elements in dot net, it parses one line at a time
12:46	myii	OK, good to be aware of that.
12:47	myii	If this comes to fruition, we'll put these notes in a little doc.