Christmas comes early to DB2 for z/OS development team

With holiday season upon us, DB2 developers have one more reason for the cheer 😉 – we have transformed our development infrastructure to Git.

This was a monumental effort (both size and significance-wise) and took us close to 11 months to complete, including a month-long “live” switch between old and new environments (or as I call it, “changing tires on the car while driving”). Last similar effort took more than 2 years, so I think we must have set some kind of record for transforming a legacy build infrastructure 😉

Here are some highlights I think might be interesting:

  • Prior to git (we use GitHub) our source code was in several legacy SCMs: SPA, CMVC, and RTC. We moved all of it to GitHub (our older releases v10 and v11, and our current release v12).
  • Moving the code over was the simplest part. The biggest complexity was in the way we deliver fixes to customers. It’s difficult to describe in few sentences for someone not familiar with what a PTF is, but essentially, we ship each fix (commit) as individually applicable (via SMP/E) object, and customers can pick and choose which fix to apply. This means our build infrastructure has to track and manage all changes (commits), because if several commits changed the same file, they would need to be built, packaged, and later applied, in certain order.
  • Because Db2z is mission critical to all our customers, we had to minimize impact to the service stream. We had to engineer a way to handle in-flight fixes that were already built and were in the test pipeline (and had to be shipped from the old environment) while also allowing new fixes to be developed and tested in the new environment. Unfortunately, because developers knew the migration schedule, they naturally rushed in all their fixes in the last day of the old environment. We ended up with about 270 different fixes (APARs) in-flight to handle during the switch. That’s close to 6 times our normal in-flight numbers.
  • The switch was further complicated by the usual things common to many software development projects, such as: fixes failed test and had to be re-worked and re-submitted, major new function delivery perfectly timed to coincide with the migration, build infrastructure bugs, Murphy law (most of the things that could go wrong did in fact go wrong), etc. Nevertheless, through some heroic effort from everyone on the infrastructure team, the migration has been completed with a small window if “inactivity” (where no new fixes were shipped out).
  • We use GitHub, Jenkins builds that invoke z/OS build, and set of (micro)-services that we use as building blocks for automation and development/build tooling.

The reason I believe this is a big deal and is a good reason to celebrate is that git is such a great tool for developers. Some teams out there might take it for granted, but we all use SCM every day (and most of the day). So finally switching to a decent tool like git that doesn’t get in a way of your dev flow (and has abundant set of capabilities to address all sorts of corner use cases when needed), is a good enough reason, IMO, to jump with joy and even make a few cartwheels in the office hallway 😉
From the infrastructure development perspective, git is really easy to work with (and also GitHub APIs). Have you tried using RTC APIs? (what an abomination).
From performance perspective, git is at least an order of magnitude faster than RTC (RTC is what we used before). Developing Db2z will be a lot more fun with this new git-based infrastructure.

Christmas comes early to DB2 for z/OS development team