{"@attributes":{"version":"2.0"},"channel":{"title":"DEV Community: Johanan Idicula","description":"The latest articles on DEV Community by Johanan Idicula (@jidicula).","link":"https:\/\/dev.to\/jidicula","image":{"url":"https:\/\/media2.dev.to\/dynamic\/image\/width=90,height=90,fit=cover,gravity=auto,format=auto\/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F480624%2F8a81d50e-6d69-41f5-95b3-b37545da28ce.png","title":"DEV Community: Johanan Idicula","link":"https:\/\/dev.to\/jidicula"},"language":"en","item":[{"title":"Triggering Cloudflare Cache Purging with Netlify's Post-Deploy Hooks and a Google Cloud Function in Go","pubDate":"Wed, 27 Jul 2022 01:23:00 +0000","link":"https:\/\/dev.to\/jidicula\/triggering-cloudflare-cache-purging-with-netlifys-post-deploy-hooks-and-a-google-cloud-function-in-go-26ke","guid":"https:\/\/dev.to\/jidicula\/triggering-cloudflare-cache-purging-with-netlifys-post-deploy-hooks-and-a-google-cloud-function-in-go-26ke","description":"<p>I had been putting off some gnarly dependency upgrades for over a year and finally got around to it last week, after a <a href=\"https:\/\/github.com\/jidicula\/forcepush\/pull\/174\" rel=\"noopener noreferrer\">few<\/a> <a href=\"https:\/\/github.com\/jidicula\/forcepush\/pull\/242\" rel=\"noopener noreferrer\">false<\/a> <a href=\"https:\/\/github.com\/jidicula\/forcepush\/pull\/337\" rel=\"noopener noreferrer\">starts<\/a> of the <code>npm<\/code> variety. I eventually gave up and rebuilt from scratch using the <code>gatsby new<\/code> tool, then ported my <a href=\"https:\/\/forcepush.tech\/the-great-gatsby-migration\" rel=\"noopener noreferrer\">customizations<\/a>. After I finished my polishes to some assets, I noticed that the changes didn't seem to have applied on <a href=\"http:\/\/forcepush.tech\" rel=\"noopener noreferrer\">forcepush.tech<\/a>. This wasn't very mysterious, as I've since put a Cloudflare cache in front of my site.<\/p>\n\n<p>As a quick refresher, my website stack is:<\/p>\n\n<ul>\n<li>Gatsby site in a GitHub repo...<\/li>\n<li>deployed to Netlify on push to <code>main<\/code>...<\/li>\n<li>cached on Cloudflare edge with a roughly 2-hour TTL (which doesn't seem to be configurable at the free tier)<\/li>\n<\/ul>\n\n<p><a href=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvh7gmer2ap4gxa64w8oh.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fvh7gmer2ap4gxa64w8oh.png\" title=\"Cloudflare Cache purge panel\" alt=\"Cloudflare\"><\/a><\/p>\n\n<p>I found a button on my Cloudflare dashboard to purge the site cache manually, so I used it and my changes became visible. I noticed the API help tip too, which would allow a way to automate this each time my site builds on Netlify. The API endpoint is pretty straightforward: just<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>POST https:\/\/api.cloudflare.com\/client\/v4\/zones\/:identifier\/purge_cache\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>with headers for <code>\"Authorization: Bearer TOKEN\"<\/code>, <code>\"Content-Type\"<\/code>, and no data required in the request. The only problem that remained was how to automatically trigger it.<\/p>\n\n<h1>\n  \n  \n  Design Goals\n<\/h1>\n\n<ul>\n<li>Cache the latest deploy as soon as it completes<\/li>\n<li>Purge and rebuild the cache automatically<\/li>\n<li>Don't purge needlessly, because I assume that rebuilding and propagating an unchanged cache to Cloudflare's edge is compute-expensive<\/li>\n<li>Don't pay for anything<\/li>\n<\/ul>\n\n<h1>\n  \n  \n  Options\n<\/h1>\n\n<p>In my Netlify deploy config, I had a couple of options for triggering the Cloudflare endpoint: I could either append a <code>curl<\/code> to my build command or I could use a <a href=\"https:\/\/docs.netlify.com\/site-deploys\/notifications\/#outgoing-webhooks\" rel=\"noopener noreferrer\">post-deploy webhook<\/a>, which sends a POST to an arbitrary URL.<\/p>\n\n<p>The <code>curl<\/code> approach seemed a bit kludge-y since it has to be <code>&amp;&amp;<\/code>-chained with an already lengthy <code>rm -rf public\/jidicula-resume &amp;&amp; npm run build<\/code> command, and I didn't like the idea of using a single-line text field for a multi-command script. This also wouldn't strictly be a post-<em>deploy<\/em> cache purge, because this is the build command config that runs at the <em>beginning<\/em> of a deploy run, not the end.<\/p>\n\n<p>Hitting the purge endpoint before the deploy begins opens up 2 failure modes that obviate the benefits of automating a purge:<\/p>\n\n<ul>\n<li>the cache gets purged before deploy and the deploy subsequently fails: the cache is rebuilt using the last successful deploy -&gt; <strong>no-change cache rebuild<\/strong>\n<\/li>\n<li>the cache gets purged before deploy and the deploy is successful, but slow: the cache is rebuilt before the deploy completes, so it still contains the last deploy's stale content -&gt; <strong>no-change cache rebuild<\/strong>\n<\/li>\n<\/ul>\n\n<p>These reasons left me with the post-deploy POST hook approach... unfortunately Netlify doesn't allow custom headers with its POST hook and can only authenticate via JWS, so it can't meet Cloudflare's <a href=\"https:\/\/api.cloudflare.com\/#zone-purge-all-files\" rel=\"noopener noreferrer\"><code>purge_cache<\/code> API specification<\/a>.<\/p>\n\n<p>To solve this POST mismatch, I searched and found <a href=\"https:\/\/brianli.com\/how-to-automatically-clear-cloudflare-cache-after-deploying-a-netlify-site\/\" rel=\"noopener noreferrer\">Brian Li's blogpost<\/a> about using a serverless cloud function as middleware: upon receiving a POST to its trigger endpoint, send a POST request to Cloudflare to the cache-purge endpoint. Of course, I opted to do it in Go instead of Python: it would have an even tinier memory footprint, better performance without any tuning, and the speedy compilation would yield a faster function build.<\/p>\n\n<h1>\n  \n  \n  Implementation\n<\/h1>\n\n<p>Here's my Go implementation:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">purger<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"fmt \"<\/span>\n    <span class=\"s\">\"io\"<\/span>\n    <span class=\"s\">\"log\"<\/span>\n    <span class=\"s\">\"net\/http\"<\/span>\n    <span class=\"s\">\"strings\"<\/span>\n\n    <span class=\"s\">\"github.com\/GoogleCloudPlatform\/functions-framework-go\/functions\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">init<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">functions<\/span><span class=\"o\">.<\/span><span class=\"n\">HTTP<\/span><span class=\"p\">(<\/span><span class=\"s\">\"PurgeCache\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">purgeCache<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"c\">\/\/ httpError logs the error and returns an HTTP error message and code.<\/span>\n<span class=\"k\">func<\/span> <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"kt\">error<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span> <span class=\"kt\">string<\/span><span class=\"p\">,<\/span> <span class=\"n\">errorCode<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">errorMsg<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Sprintf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%s: %v\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">log<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%s\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">errorMsg<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Error<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">errorMsg<\/span><span class=\"p\">,<\/span> <span class=\"n\">errorCode<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">purgeCache<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"n\">log<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Received %s from %v\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Method<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">RemoteAddr<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Method<\/span> <span class=\"o\">==<\/span> <span class=\"s\">\"POST\"<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">body<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">io<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"error reading POST body\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">return<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"n\">log<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Request body: %s\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">body<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"c\">\/\/ Send POST request to Cloudflare<\/span>\n    <span class=\"n\">client<\/span> <span class=\"o\">:=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Client<\/span><span class=\"p\">{}<\/span>\n\n    <span class=\"n\">data<\/span> <span class=\"o\">:=<\/span> <span class=\"s\">`{\"purge_everything\":true}`<\/span>\n    <span class=\"n\">req<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">NewRequest<\/span><span class=\"p\">(<\/span><span class=\"s\">\"POST\"<\/span><span class=\"p\">,<\/span>\n                                <span class=\"s\">\"https:\/\/api.cloudflare.com\/client\/v4\/zones\/ZONE_ID\/purge_cache\"<\/span><span class=\"p\">,<\/span> \n                                <span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">data<\/span><span class=\"p\">))<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"error creating new Request\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"n\">req<\/span><span class=\"o\">.<\/span><span class=\"n\">Header<\/span><span class=\"o\">.<\/span><span class=\"n\">Add<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Authorization\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Bearer CLOUDFLARE-API-TOKEN\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">req<\/span><span class=\"o\">.<\/span><span class=\"n\">Header<\/span><span class=\"o\">.<\/span><span class=\"n\">Add<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Content-Type\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"application\/json\"<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"n\">cloudflareResp<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">client<\/span><span class=\"o\">.<\/span><span class=\"n\">Do<\/span><span class=\"p\">(<\/span><span class=\"n\">req<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"error sending POST request\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">defer<\/span> <span class=\"n\">cloudflareResp<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n\n    <span class=\"c\">\/\/ Pass cloudflare response to caller<\/span>\n\n    <span class=\"n\">cloudflareRespBody<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">io<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">cloudflareResp<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"error reading Cloudflare response\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">if<\/span> <span class=\"n\">cloudflareResp<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusCode<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusOK<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">msg<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Sprintf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"error non-200 status: %s\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">cloudflareRespBody<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"n\">log<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Cloudflare response: %s\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">cloudflareRespBody<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">cloudflareRespBody<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">httpError<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"error sending response to client\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusInternalServerError<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<ul>\n<li>There's some Google Cloud Functions boilerplate required: the <code>init()<\/code> and its <code>functions.HTTP<\/code> call that registers the function that's invoked.<\/li>\n<li>The invoked function seems to require receiving <code>http.ResponseWriter<\/code> and <code>*http.Request<\/code> in its parameters (I messed around to see if they could be omitted, as the documentation for the 2nd-gen Cloud Functions isn't exactly complete).\n\n<ul>\n<li>As usual in Go, I use <code>*http.Client<\/code> and <code>http.NewRequest()<\/code> for adding custom headers to a HTTP request - the steps are to create <code>Request<\/code> with <code>NewRequest<\/code> and pass it to <code>client<\/code> to send.<\/li>\n<\/ul>\n\n\n<\/li>\n\n<li>I use all the <a href=\"https:\/\/pkg.go.dev\/net\/http#pkg-constants\" rel=\"noopener noreferrer\">builtin error codes<\/a> that I can for handling various failure modes and informing the caller that something went wrong. Also for convenience, I factored out the usual <code>log.Printf()<\/code> &amp; <code>http.Error()<\/code> calls into a <code>httpError()<\/code> function.<\/li>\n\n<li>Regardless of Cloudflare's response, the function forwards it back to the caller.<\/li>\n\n<li>And as a final polish, I log where the request came from and its body if it's a POST (which should only come from Netlify). Google Cloud Functions can be <a href=\"https:\/\/cloud.google.com\/functions\/docs\/calling\/http\" rel=\"noopener noreferrer\">HTTP-triggered<\/a> with any of <code>POST<\/code>, <code>PUT<\/code>,\u00a0<code>GET<\/code>,\u00a0<code>DELETE<\/code>, or\u00a0<code>OPTIONS<\/code> requests.<\/li>\n\n<\/ul>\n\n<p>For the Google Cloud Function config, I used the 2nd-gen Cloud Functions since it uses the Artifact Registry for storing the function image, and Artifact Registry has a free tier (1st-gen Cloud Functions use the Container Registry, which costs money). Additional configs:<\/p>\n\n<ul>\n<li>Memory allocated: 128 MiB (the lowest option possible)<\/li>\n<li>Timeout: 60 seconds (default)<\/li>\n<li>Autoscaling: 0 instance minimum to 1 instance maximum (only need 1 run instance for a cache purge)<\/li>\n<li>Region: us-east4 (this is in North Virginia, the same place where AWS's major us-east region is - Netlify is hosted on AWS so hopefully this reduces some latency)<\/li>\n<\/ul>\n\n<h2>\n  \n  \n  Downside\n<\/h2>\n\n<p>The main downside of this approach is that it relies on security through obscurity - the trigger endpoint has to be publicly exposed for Netlify to be able to POST to it. In the worst case, the endpoint could get slammed with malicious requests, but end result is <em>probably<\/em> ok - I limited the function to 1 invocation at a time, so this could act as a throttle. If malicious requests become a problem, I'll add an early-return check to the function preamble for the request origin or content, or I might even work up some check that Netlify's hook can authenticate against using <a href=\"https:\/\/docs.netlify.com\/site-deploys\/notifications\/#payload-signature\" rel=\"noopener noreferrer\">JWS<\/a>.<\/p>\n\n<h1>\n  \n  \n  Summary\n<\/h1>\n\n<p>Overall, I'm pretty satisfied with this cache-purging solution - it meets all my design goals, and it's <em>fast<\/em>:<\/p>\n\n<ul>\n<li>\u2705 Cache the latest deploy as soon as it completes<\/li>\n<li>\u2705 Purge and rebuild the cache automatically<\/li>\n<li>\u2705 Don't purge needlessly, because I assume that rebuilding and propagating an unchanged cache to Cloudflare's edge is compute-expensive<\/li>\n<li>\u2705 Don't pay for anything<\/li>\n<\/ul>\n\n<p>If you have any questions or comments, email me at <a href=\"mailto:johanan+blog@forcepush.tech\">johanan+blog@forcepush.tech<\/a>, find me on Twitter <a href=\"http:\/\/twitter.com\/jidiculous\" rel=\"noopener noreferrer\">@jidiculous<\/a>, or post a comment below.<\/p>\n\n<p>Did you find this post useful? Buy me a beverage or sponsor me <a href=\"https:\/\/github.com\/sponsors\/jidicula\" rel=\"noopener noreferrer\">here<\/a>!<\/p>\n\n","category":["go","netlify","googlecloud","cloudflare"]},{"title":"Mount Docker container in Azure App Service as read-only filesystem","pubDate":"Mon, 07 Jun 2021 20:32:42 +0000","link":"https:\/\/dev.to\/jidicula\/mount-docker-container-in-azure-app-service-as-read-only-filesystem-4dnl","guid":"https:\/\/dev.to\/jidicula\/mount-docker-container-in-azure-app-service-as-read-only-filesystem-4dnl","description":"<div class=\"ltag__stackexchange--container\">\n  <div class=\"ltag__stackexchange--title-container\">\n    \n      <div class=\"ltag__stackexchange--title\">\n        <div class=\"ltag__stackexchange--header\">\n          <img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--7Gn-iPj_--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/dev.to\/assets\/stackoverflow-logo-b42691ae545e4810b105ee957979a853a696085e67e43ee14c5699cf3e890fb4.svg\" alt=\"\">\n          <a href=\"https:\/\/stackoverflow.com\/questions\/67878177\/mount-docker-container-in-azure-app-service-as-read-only-filesystem\" rel=\"noopener noreferrer\">\n            Mount Docker container in Azure App Service as read-only filesystem\n          <\/a>\n        <\/div>\n        <div class=\"ltag__stackexchange--post-metadata\">\n          <span>Jun  7 '21<\/span>\n            <span>Comments: 1<\/span>\n            <span>Answers: 2<\/span>\n        <\/div>\n      <\/div>\n      <a class=\"ltag__stackexchange--score-container\" href=\"https:\/\/stackoverflow.com\/questions\/67878177\/mount-docker-container-in-azure-app-service-as-read-only-filesystem\" rel=\"noopener noreferrer\">\n        <img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--Y9mJpuJP--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/dev.to\/assets\/stackexchange-arrow-up-eff2e2849e67d156181d258e38802c0b57fa011f74164a7f97675ca3b6ab756b.svg\" alt=\"\">\n        <div class=\"ltag__stackexchange--score-number\">\n          0\n        <\/div>\n        <img src=\"https:\/\/res.cloudinary.com\/practicaldev\/image\/fetch\/s--wif5Zq3z--\/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880\/https:\/\/dev.to\/assets\/stackexchange-arrow-down-4349fac0dd932d284fab7e4dd9846f19a3710558efde0d2dfd05897f3eeb9aba.svg\" alt=\"\">\n      <\/a>\n    \n  <\/div>\n  <div class=\"ltag__stackexchange--body\">\n    \n<p>Docker has a <code>run<\/code> command that accepts a <code>--read-only<\/code> argument for mounting a container with a read-only filesystem. Is there a way to set up an Azure App Service slot to run a container from an Azure Container Registry with a read-only filesystem? I haven't been able to find any\u2026<\/p>\n    \n  <\/div>\n  <div class=\"ltag__stackexchange--btn--container\">\n    <a href=\"https:\/\/stackoverflow.com\/questions\/67878177\/mount-docker-container-in-azure-app-service-as-read-only-filesystem\" class=\"ltag__stackexchange--btn\" rel=\"noopener noreferrer\">Open Full Question<\/a>\n  <\/div>\n<\/div>\n\n\n","category":["azure","docker"]},{"title":"Go Package CI\/CD with GitHub Actions","pubDate":"Sat, 15 May 2021 18:12:30 +0000","link":"https:\/\/dev.to\/jidicula\/go-package-ci-cd-with-github-actions-350o","guid":"https:\/\/dev.to\/jidicula\/go-package-ci-cd-with-github-actions-350o","description":"<p>In a <a href=\"https:\/\/forcepush.tech\/python-package-ci-cd-with-git-hub-actions\">previous post<\/a>, I wrote about how I implemented CI\/CD checks and autoreleases for the Python implementation of my random-standup program. I also developed some similar workflows for the Go implementation, so I thought I'd also write a Go-flavoured post about packaging CI\/CD using GitHub Actions. This post may seem very familiar if you read that previous post - as I described in my <a href=\"https:\/\/forcepush.tech\/writing-a-simple-cli-program-python-vs-go\">comparison between the Go and Python implementations of this program<\/a>, my CI\/CD goals are the same: PR checks and autoreleases.<\/p>\n\n<p>As I said before, I wanted to ensure that:<\/p>\n\n<ol>\n<li>Each change I make to my program won't break existing functionality (Continuous Integration), and<\/li>\n<li>Publishing a new release to <a href=\"https:\/\/pkg.go.dev\">pkg.go.dev<\/a> is automatic (Continuous Delivery\/Deployment).<\/li>\n<\/ol>\n\n<p>GitHub provides a workflow automation feature called <a href=\"https:\/\/docs.github.com\/en\/actions\">GitHub Actions<\/a>. Essentially, you write your workflow configurations in a YAML file in <code>your-repo\/.github\/workflows\/<\/code>, and they'll be executed on certain repository events.<\/p>\n\n<h1>\n  \n  \n  Continuous Integration\n<\/h1>\n\n<p>This automation is relatively straightforward. I want to run the following workflows on each commit into the repository trunk and on each pull request into trunk:<\/p>\n\n<ol>\n<li>Test syntax by running a linting check with <a href=\"https:\/\/golangci-lint.run\"><code>golangci-lint<\/code><\/a> - it's the best linter (actually, I suppose it's a meta-linter since it invokes several separate linters) available for Go and slaps your wrist if you slip into some well-known antipatterns.<\/li>\n<li>Test functionality by running automated unit tests on the entire program. This is an extremely simple program, so I definitely overengineered its factoring into functions to make it easier to unit test.<\/li>\n<li>Test build stability by attempting to build the program (but discarding the build artifact) across as many OS and arch combinations supported by Go. Of course, I don't expect that anyone would run my standup randomizer using Plan 9 on an ARM chip, but this was more of an exercise to learn about Go's cross-compilation capabilities.<\/li>\n<\/ol>\n\n<p>Here's the <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/build.yml\">full workflow<\/a>.<\/p>\n\n<h2>\n  \n  \n  Each commit to trunk\n<\/h2>\n\n<p>The trigger for this is declared at the top of the workflow file:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span><span class=\"nv\">main<\/span><span class=\"pi\">]<\/span>\n  <span class=\"na\">pull_request<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span><span class=\"nv\">main<\/span><span class=\"pi\">]<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Test syntax by checking formatting\n<\/h2>\n\n<p>First, we have to checkout the repository in GitHub Actions using <a href=\"https:\/\/github.com\/actions\/checkout\">GitHub's own <code>checkout<\/code> action<\/a>. Then, we have to set up the Go version using <a href=\"https:\/\/github.com\/actions\/setup-go\">GitHub's <code>setup-go<\/code> action<\/a>. GitHub Actions has 3 different OSes available for their runners, each with various <a href=\"https:\/\/github.com\/actions\/virtual-environments\/blob\/main\/images\/linux\/Ubuntu2004-README.md#go\">Go versions<\/a>, but it's safest to explicitly specify which Go version will be used.<\/p>\n\n<p>Finally, we can use <a href=\"https:\/\/github.com\/golangci\/golangci-lint-action\/blob\/master\/action.yml\">golangci-lint's provided GitHub Action<\/a> for linting - it runs <code>golangci-lint<\/code> on the workflow runner's clone of the repo and outputs an error code if any Go file in the repo fails rules of any linters in <code>golangci-lint<\/code>. Note that <code>golangci-lint<\/code> fails if the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Abstract_syntax_tree\">AST<\/a> cannot be parsed (i.e. if there are any syntax errors), so it can also be used for checking syntax correctness, which itself is a good proxy for checking for merge conflict strings. We can fail-fast with any checks this way - there's no need to spin up a compilation and a <code>go test<\/code> invocation if there are syntax errors.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">lint<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Lint files<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2.3.4<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/setup-go@v2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">go-version<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">1.16.4'<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">golangci-lint<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">golangci\/golangci-lint-action@v2.5.2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">version<\/span><span class=\"pi\">:<\/span> <span class=\"s\">latest<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Test Functionality\n<\/h2>\n\n<p>Again, we need to checkout the repo for this job and set up the Go version:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>    <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Run tests<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n    <span class=\"na\">needs<\/span><span class=\"pi\">:<\/span> <span class=\"s\">lint<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2.3.4<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/setup-go@v2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">go-version<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">1.16.4'<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">go test -v -cover<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Note that unlike Python, no setup is needed to install dependencies (<code>go test<\/code> automatically grabs dependencies defined in <code>go.mod<\/code>) or set up a virtual environment, so there's a lot less boilerplate in CI\/CD.<\/p>\n\n<h2>\n  \n  \n  Test build stability for different OSes and architectures\n<\/h2>\n\n<p>Go provides <a href=\"https:\/\/www.digitalocean.com\/community\/tutorials\/building-go-applications-for-different-operating-systems-and-architectures\">cross-compilation tooling<\/a> for a wide variety of operating systems and architectures. Essentially, you can run a command like<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ GOOS<\/span><span class=\"o\">=<\/span>plan9 <span class=\"nv\">GOARCH<\/span><span class=\"o\">=<\/span>arm go build\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>and the Go compiler will build a binary that will run on the OS specified in <code>GOOS<\/code> and the arch in <code>GOARCH<\/code>. To see the full list of GOOS and GOARCH options, run <code>go tool dist list<\/code>.<\/p>\n\n<p>We want to verify build stability across this set, so we can set up a matrix build for different GOOS and GOARCH options using GitHub Actions:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>  <span class=\"na\">build<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n    <span class=\"na\">needs<\/span><span class=\"pi\">:<\/span> <span class=\"s\">test<\/span>\n    <span class=\"na\">strategy<\/span><span class=\"pi\">:<\/span>\n      <span class=\"na\">matrix<\/span><span class=\"pi\">:<\/span>\n        <span class=\"na\">goosarch<\/span><span class=\"pi\">:<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">aix\/ppc64'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">android\/amd64'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">android\/arm64'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">darwin\/amd64'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">darwin\/arm64'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">dragonfly\/amd64'<\/span>\n          <span class=\"c1\"># ...<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This is defined in the <a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix\"><code>jobs.&lt;job_id&gt;.strategy.matrix<\/code> directive<\/a>. I've added just 1 variable for every GOOS and GOARCH pairing (truncated for this blogpost - there are <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/291b9a3cccdad0fece3c061029ecadb7c0676bc5\/.github\/workflows\/build.yml#L42\">39 pairs defined in my workflow file<\/a>).<\/p>\n\n<p>Internally, the steps are somewhat like:<\/p>\n\n<ol>\n<li>GitHub Actions parses the directives for the job and sees there's a matrix strategy.<\/li>\n<li>It spins up a separate runner for each matrix combination and defines the variables <code>matrix.goosarch<\/code> as the values for that combination.<\/li>\n<li>It runs the job steps in each runner it spun up in Step 2.<\/li>\n<\/ol>\n\n<p>You can see an example of how this matrix run looks like in the GitHub Actions console <a href=\"https:\/\/github.com\/jidicula\/random-standup\/actions\/runs\/835336772\">here<\/a> (see all the <code>goosarch<\/code> values in the left sidebar). These matrix options are run in parallel by default, so the runtime of the job determined by the slowest matrix option. Note that if your repository is private, you will be charged Actions minutes for each separate build matrix option, with some <a href=\"https:\/\/docs.github.com\/en\/github\/setting-up-and-managing-billing-and-payments-on-github\/about-billing-for-github-actions#about-billing-for-github-actions\">hefty multipliers for macOS and Windows runners<\/a> (1 macOS minute is 10 minutes of Actions credit, 1 Windows minute is 2 minutes of Actions credit as of May 2021).<\/p>\n\n<p>We do our usual checkout and Go version setup, then some basic Bash string-splitting on the <code>\/<\/code> character so we can set the <code>GOOS<\/code> and <code>GOARCH<\/code> environment variables separately from a single matrix option:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Get OS and arch info<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">GOOSARCH=${{matrix.goosarch}}<\/span>\n          <span class=\"s\">GOOS=${GOOSARCH%\/*}<\/span>\n          <span class=\"s\">GOARCH=${GOOSARCH#*\/}<\/span>\n          <span class=\"s\">BINARY_NAME=${{github.repository}}-$GOOS-$GOARCH<\/span>\n          <span class=\"s\">echo \"BINARY_NAME=$BINARY_NAME\" &gt;&gt; $GITHUB_ENV<\/span>\n          <span class=\"s\">echo \"GOOS=$GOOS\" &gt;&gt; $GITHUB_ENV<\/span>\n          <span class=\"s\">echo \"GOARCH=$GOARCH\" &gt;&gt; $GITHUB_ENV<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, we simply run Go's <code>go build<\/code> subcommand, which creates the binary:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Build<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">go build -o \"$BINARY_NAME\" -v<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Auto-merge\n<\/h2>\n\n<p>GitHub also allows pull requests to be merged automatically if branch protection rules are configured and if the pull request passes all required reviews and status checks. In the repo Settings &gt; Branches &gt; Branch Protection rules, I have a rule defined for <code>main<\/code> requiring all jobs in the <code>build.yml<\/code> workflow to pass before a branch can be merged into <code>main<\/code>.<\/p>\n\n<h1>\n  \n  \n  Release automation\n<\/h1>\n\n<p>There are 2 parts to GitHub release automation:<\/p>\n\n<ol>\n<li>Create the GitHub release using Git tags and add the build artifacts to it (<a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/release-draft.yml\">workflow<\/a>).<\/li>\n<li>Publish the package to pkg.go.dev (<a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/publish.yml\">workflow<\/a>).<\/li>\n<\/ol>\n\n<h2>\n  \n  \n  Create GitHub Release\n<\/h2>\n\n<p>We set up the workflow to trigger on push to a tag beginning with <code>v<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"c1\"># Sequence of patterns matched against refs\/tags<\/span>\n    <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">v*'<\/span> <span class=\"c1\"># Push events to matching v*, i.e. v1.0, v20.15.10<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, we define our <code>release<\/code> job, running on Ubuntu (cheapest and fastest GitHub Actions runner environment):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Create Release<\/span>\n\n<span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">autorelease<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Create Release<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I also set up the same <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/291b9a3cccdad0fece3c061029ecadb7c0676bc5\/.github\/workflows\/release-draft.yml#L42\">GOOS and GOARCH build matrix<\/a> as in <code>build.yml<\/code> - when we create the GitHub release, we'll build and upload the binaries as release assets.<\/p>\n\n<p>Our first 2 steps are almost the same as our Build workflow for pushes and PRs to <code>main<\/code>: we checkout the repo and set up Go. Our checkout step is slightly different, though: we provide <code>0<\/code> to the <code>fetch-depth<\/code> input so we make a deep clone with all commits, not a shallow clone with just the most recent commit.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Checkout code<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">fetch-depth<\/span><span class=\"pi\">:<\/span> <span class=\"m\">0<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Go specifies module versions using version control tagging, so we don't need to parse any manifest files like we did with Python. So, we can do the same Bash string splitting as before and build the binary:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Get OS and arch info<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">GOOSARCH=${{matrix.goosarch}}<\/span>\n          <span class=\"s\">GOOS=${GOOSARCH%\/*}<\/span>\n          <span class=\"s\">GOARCH=${GOOSARCH#*\/}<\/span>\n          <span class=\"s\">BINARY_NAME=${{github.repository}}-$GOOS-$GOARCH<\/span>\n          <span class=\"s\">echo \"BINARY_NAME=$BINARY_NAME\" &gt;&gt; $GITHUB_ENV<\/span>\n          <span class=\"s\">echo \"GOOS=$GOOS\" &gt;&gt; $GITHUB_ENV<\/span>\n          <span class=\"s\">echo \"GOARCH=$GOARCH\" &gt;&gt; $GITHUB_ENV<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Build<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">go build -o \"$BINARY_NAME\" -v<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The next step is to create some release notes. I keep a release template in the <code>.github<\/code> folder and append some gitlog output to it:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code> <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Release Notes<\/span>\n        <span class=\"s\">run<\/span><span class=\"err\">:<\/span> <span class=\"s\">git log $(git describe HEAD~ --tags --abbrev=0)..HEAD --pretty='format:* %h %s%n  * %an &lt;%ae&gt;' --no-merges &gt;&gt; \".github\/RELEASE-TEMPLATE.md\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>That gnarly gitlog command is checking all commits since the last tag to HEAD. For each commit, it appends the commit hash, the commit message subject, the author name, and the author email to the release template.<\/p>\n\n<p>Finally, we use a <a href=\"https:\/\/github.com\/softprops\/action-gh-release\">3rd-party release creation Action<\/a> for creating a release draft with the release notes and artifacts we just created:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Release with Notes<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">softprops\/action-gh-release@v1<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">body_path<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">.github\/RELEASE-TEMPLATE.md\"<\/span>\n          <span class=\"na\">draft<\/span><span class=\"pi\">:<\/span> <span class=\"no\">true<\/span>\n          <span class=\"na\">files<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{env.BINARY_NAME}}<\/span>\n        <span class=\"na\">env<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">GITHUB_TOKEN<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ secrets.GITHUB_TOKEN }}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This creates a draft visible at <a href=\"https:\/\/github.com\/jidicula\/random-standup\/releases\">https:\/\/github.com\/jidicula\/random-standup\/releases<\/a>. I modify the release announcements as needed, and publish the release.<\/p>\n\n<h2>\n  \n  \n  Publishing to pkg.go.dev\n<\/h2>\n\n<p>The final step of the release process is to notify pkg.go.dev that there's a new version available for the module. Here's the full <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/publish.yml\">workflow<\/a>.<\/p>\n\n<p>This time, we trigger the workflow to run on a release being published (the last step of the previous workflow is manually publishing a release draft):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">release<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">types<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"s\">published<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>We do the same checkout as before. Then, we simply run <code>curl<\/code> to the URL where the module is fetched from by <code>go get<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">bump-index<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s\">ubuntu-latest<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Checkout repo<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2.3.4<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Ping endpoint<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">curl \"https:\/\/proxy.golang.org\/github.com\/jidicula\/random-standup\/@v\/$(git describe HEAD --tags --abbrev=0).info\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>pkg.go.dev recommends this as <a href=\"https:\/\/go.dev\/about#adding-a-package\">one of the ways of adding a new module (or module version)<\/a> to its index.<\/p>\n\n<h1>\n  \n  \n  Putting it all together\n<\/h1>\n\n<p>So overall, working on this project would involve:<\/p>\n\n<ol>\n<li>Make a PR for my changes.<\/li>\n<li>Confirm auto-merge.<\/li>\n<li>Repeeat Steps 1 and 2 until I'm ready to release.<\/li>\n<li>Create a tag on <code>main<\/code> pointing to the version bump commit.<\/li>\n<li>Push the tag to GitHub.<\/li>\n<li>Wait for the <a href=\"https:\/\/github.com\/jidicula\/random-standup\/actions\/workflows\/release.yml\">Create Release<\/a> run to finish.<\/li>\n<li>Go to <a href=\"https:\/\/github.com\/jidicula\/random-standup\/releases\">https:\/\/github.com\/jidicula\/random-standup\/releases<\/a> and modify the Announcements for the just-created release draft.<\/li>\n<li>Publish the release.<\/li>\n<li>Wait for the <a href=\"https:\/\/github.com\/jidicula\/random-standup\/actions\/workflows\/publish.yml\">Publish<\/a> run to finish.<\/li>\n<li>Check <a href=\"https:\/\/pkg.go.dev\/github.com\/jidicula\/random-standup\">pkg.go.dev<\/a> for the updated package version.<\/li>\n<\/ol>\n\n<p>If you have any questions or comments, email me at <a href=\"mailto:johanan+blog@forcepush.tech\">johanan+blog@forcepush.tech<\/a>, find me on Twitter <a href=\"http:\/\/twitter.com\/jidiculous\">@jidiculous<\/a>, or post a comment below.<\/p>\n\n<p>Did you find this post useful? Buy me a beverage or sponsor me <a href=\"https:\/\/github.com\/sponsors\/jidicula\">here<\/a>!<\/p>\n\n","category":["go","github","devops"]},{"title":"Python Package CI\/CD with GitHub Actions and Poetry","pubDate":"Mon, 10 May 2021 01:12:31 +0000","link":"https:\/\/dev.to\/jidicula\/python-ci-cd-with-github-actions-2e26","guid":"https:\/\/dev.to\/jidicula\/python-ci-cd-with-github-actions-2e26","description":"<p>You can also read this on my blog <a href=\"https:\/\/forcepush.tech\/python-package-ci-cd-with-git-hub-actions\">here<\/a>.<\/p>\n\n<p>In a <a href=\"https:\/\/forcepush.tech\/writing-a-simple-cli-program-python-vs-go\">previous post<\/a>, I alluded to having pure CI\/CD checks and autoreleases for my random-standup program. I wanted to ensure that:<\/p>\n\n<ol>\n<li>Each change I make to my program won't break existing functionality (Continuous Integration), and<\/li>\n<li>Publishing a new release to PyPI is automatic (Continuous Delivery\/Deployment).<\/li>\n<\/ol>\n\n<p>GitHub provides a workflow automation feature called <a href=\"https:\/\/docs.github.com\/en\/actions\">GitHub Actions<\/a>. Essentially, you write your workflow configurations in a YAML file in <code>your-repo\/.github\/workflows\/<\/code>, and they'll be executed on certain repository events.<\/p>\n\n<h1>\n  \n  \n  Continuous Integration\n<\/h1>\n\n<p>This automation is relatively straightforward. I want to run the following workflows on each commit into the repository trunk and on each pull request into trunk:<\/p>\n\n<ol>\n<li>Test syntax by running a linting check for formatting (n.b. syntax correctness is a subset of formatting correctness).<\/li>\n<li>Test functionality <strong>across a variety of operating systems and Python versions<\/strong> by running automated tests on the entire program. For this program, I only included a single basic black-box test that's more demonstrative than useful (it checks for a regex match with program output). A suite of unit tests would be more appropriate for a more complex program.<\/li>\n<li>Test build stability by attempting to build the program (but discarding the build artifact) across the same combinations of operating systems and Python versions from Step 2.<\/li>\n<\/ol>\n\n<p>Here's the <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/blob\/main\/.github\/workflows\/release.yml\">full workflow<\/a>.<\/p>\n\n<h2>\n  \n  \n  Each commit to trunk\n<\/h2>\n\n<p>The trigger for this is declared at the top of the workflow file:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span><span class=\"nv\">main<\/span><span class=\"pi\">]<\/span>\n  <span class=\"na\">pull_request<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span><span class=\"nv\">main<\/span><span class=\"pi\">]<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Test syntax by checking formatting\n<\/h2>\n\n<p>First, we have to checkout the repository in GitHub Actions using <a href=\"https:\/\/github.com\/actions\/checkout\">GitHub's own <code>checkout<\/code> action<\/a>. Then, we have to set up the Python version using <a href=\"https:\/\/github.com\/actions\/setup-python\">GitHub's <code>setup-python<\/code> action<\/a>. Finally, we can use <a href=\"https:\/\/github.com\/psf\/black\/blob\/main\/action.yml\">Black's provided GitHub Action<\/a> for checking formatting - it runs <code>black --check --diff<\/code> on the workflow runner's clone of the repo and outputs an error code if any Python file in the repo fails Black's formatting rules. Note that Black fails if the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Abstract_syntax_tree\">AST<\/a> cannot be parsed (i.e. if there are any syntax errors), so it can also be used for checking syntax correctness, which itself is a good proxy for checking for merge conflict strings.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">black-formatting-check<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check formatting<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/setup-python@v2<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">psf\/black@stable<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Running a job across different build environments\n<\/h2>\n\n<p>GitHub Actions provides matrix build functionality where you provide the option set for each variable and it runs the dependent steps with the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Cartesian_product#n-ary_Cartesian_product\">n-ary Cartesian product<\/a> of these <em>n<\/em> variable option sets:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>  <span class=\"na\">build<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ matrix.os }}<\/span>\n    <span class=\"na\">needs<\/span><span class=\"pi\">:<\/span> <span class=\"s\">black-formatting-check<\/span>\n    <span class=\"na\">strategy<\/span><span class=\"pi\">:<\/span>\n      <span class=\"na\">matrix<\/span><span class=\"pi\">:<\/span>\n        <span class=\"na\">os<\/span><span class=\"pi\">:<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">macos-latest'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">windows-latest'<\/span>\n        <span class=\"na\">python-version<\/span><span class=\"pi\">:<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">3.7'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">3.8'<\/span>\n          <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">3.9'<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This is defined in the <a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/workflow-syntax-for-github-actions#jobsjob_idstrategymatrix\"><code>jobs.&lt;job_id&gt;.strategy.matrix<\/code> directive<\/a>. I've added 2 variables: one for OS (with Ubuntu, macOS, and Windows as options) and one for Python version (with 3.7, 3.8, and 3.9 as options). This means that everything in the <code>build<\/code> job will run on every combination of OS and Python version options:<\/p>\n\n<ul>\n<li>Ubuntu, Python 3.7<\/li>\n<li>Ubuntu, Python 3.8<\/li>\n<li>Ubuntu, Python 3.9<\/li>\n<li>macOS, Python 3.7<\/li>\n<li>macOS, Python 3.8<\/li>\n<li>etc<\/li>\n<\/ul>\n\n<p>Note that the <code>runs-on<\/code> directive is defined as <code>${{ matrix.os }}<\/code> which points to the value of the <code>os<\/code> variable in the current runner. Internally, the steps are somewhat like:<\/p>\n\n<ol>\n<li>GitHub Actions parses the directives for the job and sees there's a matrix strategy.<\/li>\n<li>It spins up a separate runner for each matrix combination and defines the variables <code>matrix.os<\/code> and <code>matrix.python-version<\/code> as the values for that combination. For example, in the Ubuntu\/Python 3.7 runner, <code>matrix.os = 'ubuntu-latest'<\/code> and <code>matrix.python-version = '3.7'<\/code>.<\/li>\n<li>It runs the job steps in each runner it spun up in Step 2.<\/li>\n<\/ol>\n\n<p>You can see an example of how this matrix run looks like in the GitHub Actions console <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/actions\/runs\/806535255\">here<\/a> (see all the OS\/Python combinations in the left sidebar). These matrix options are run in parallel by default, so the runtime of the job determined by the slowest matrix option. Note that if your repository is private, you will be charged Actions minutes for each separate build combination, with some <a href=\"https:\/\/docs.github.com\/en\/github\/setting-up-and-managing-billing-and-payments-on-github\/about-billing-for-github-actions#about-billing-for-github-actions\">hefty multipliers for macOS and Windows<\/a> (1 macOS minute is 10 minutes of Actions credit, 1 Windows minute is 2 minutes of Actions credit as of May 2021).<\/p>\n\n<h2>\n  \n  \n  Test Functionality\n<\/h2>\n\n<p>Again, we need to checkout the repo for this job and set up the Python version. The key difference with the Python version setup here compared to the Black formatting job is that the Python version is specified and points to the matrix option for <code>python-version<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Checkout code<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Setup Python<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/setup-python@v2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">python-version<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{matrix.python-version}}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, we need to set up the dependencies for the program to ensure it can run. I used Poetry for dependency and virtual environment management, and it's not included with any of the runner <a href=\"https:\/\/github.com\/actions\/virtual-environments\">environments<\/a>, so we have to install it in a workflow step. Installing it takes some time, though, so to speed up my workflow runtime, I \"permanently\" cache Poetry using <a href=\"https:\/\/github.com\/actions\/cache\">GitHub's provided <code>cache<\/code> action<\/a>. I only run the installation step if the cache is missed, which won't happen since the key is constant for each OS\/Python version combination.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"c1\"># Perma-cache Poetry since we only need it for checking pyproject version<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Cache Poetry<\/span>\n        <span class=\"na\">id<\/span><span class=\"pi\">:<\/span> <span class=\"s\">cache-poetry<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/cache@v2.1.5<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">path<\/span><span class=\"pi\">:<\/span> <span class=\"s\">~\/.poetry<\/span>\n          <span class=\"na\">key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ matrix.os }}-poetry<\/span>\n      <span class=\"c1\"># Only runs when key from caching step changes<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Install latest version of Poetry<\/span>\n        <span class=\"na\">if<\/span><span class=\"pi\">:<\/span> <span class=\"s\">steps.cache-poetry.outputs.cache-hit != 'true'<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">curl -sSL https:\/\/raw.githubusercontent.com\/python-poetry\/poetry\/master\/get-poetry.py | python -<\/span>\n      <span class=\"c1\"># Poetry still needs to be re-prepended to the PATH on each run, since<\/span>\n      <span class=\"c1\"># PATH does not persist between runs.<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Add Poetry to $PATH<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">echo \"$HOME\/.poetry\/bin\" &gt;&gt; $GITHUB_PATH<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Get Poetry version<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">poetry --version<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, I do another caching step for dependencies and install them if <code>poetry.lock<\/code> has changed:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check pyproject.toml validity<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">poetry check --no-interaction<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Cache dependencies<\/span>\n        <span class=\"na\">id<\/span><span class=\"pi\">:<\/span> <span class=\"s\">cache-deps<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/cache@v2.1.5<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">path<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{github.workspace}}\/.venv<\/span>\n          <span class=\"na\">key<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ matrix.os }}-${{ hashFiles('**\/poetry.lock') }}<\/span>\n          <span class=\"na\">restore-keys<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ matrix.os }}-<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Install deps<\/span>\n        <span class=\"na\">if<\/span><span class=\"pi\">:<\/span> <span class=\"s\">steps.cache-deps.cache-hit != 'true'<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">poetry config virtualenvs.in-project true<\/span>\n          <span class=\"s\">poetry install --no-interaction<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Finally, once dependency and virtual environment setup is done, I run pytest:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Run tests<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">poetry run pytest -v<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Test build stability\n<\/h2>\n\n<p>For testing build stability, we simply run Poetry's <code>build<\/code> subcommand, which creates the build artifacts:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Build artifacts<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">poetry build<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Auto-merge\n<\/h2>\n\n<p>GitHub also allows pull requests to be merged automatically if branch protection rules are configured and if the pull request passes all required reviews and status checks. In the repo Settings &gt; Branches &gt; Branch Protection rules, I have a rule defined for <code>main<\/code> requiring all jobs in the <code>build.yml<\/code> workflow to pass before a branch can be merged into <code>main<\/code>.<\/p>\n\n<h1>\n  \n  \n  Release automation\n<\/h1>\n\n<p>There are 2 parts to GitHub release automation:<\/p>\n\n<ol>\n<li>Create the GitHub release using Git tags and add the build artifacts to it (<a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/blob\/main\/.github\/workflows\/release.yml\">workflow<\/a>).<\/li>\n<li>Publish the package to PyPI (<a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/blob\/main\/.github\/workflows\/publish.yml\">workflow<\/a>).<\/li>\n<\/ol>\n\n<h2>\n  \n  \n  Create GitHub Release\n<\/h2>\n\n<p>We set up the workflow to trigger on push to a tag beginning with <code>v<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"c1\"># Sequence of patterns matched against refs\/tags<\/span>\n    <span class=\"na\">tags<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"s1\">'<\/span><span class=\"s\">v*'<\/span> <span class=\"c1\"># Push events to matching v*, i.e. v1.0, v20.15.10<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, we define our <code>autorelease<\/code> job, running on Ubuntu (cheapest and fastest GitHub Actions runner environment):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Create Release<\/span>\n\n<span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">autorelease<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Create Release<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s1\">'<\/span><span class=\"s\">ubuntu-latest'<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Our first 2 steps are almost the same as our Build workflow for pushes and PRs to <code>main<\/code>: we checkout the repo and set up Poetry. Our checkout step is slightly different, though: we provide <code>0<\/code> to the <code>fetch-depth<\/code> input so we make a deep clone with all commits, not a shallow clone with just the most recent commit.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Checkout code<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">fetch-depth<\/span><span class=\"pi\">:<\/span> <span class=\"m\">0<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The Poetry setup steps are identical, so I won't include them here.<\/p>\n\n<p>Then, we use Poetry to get the project version from <code>pyproject.toml<\/code>, store it in an environment variable, then check if the tag version matches the project version:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Add version to environment vars<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">PROJECT_VERSION=$(poetry version --short)<\/span>\n          <span class=\"s\">echo \"PROJECT_VERSION=$PROJECT_VERSION\" &gt;&gt; $GITHUB_ENV<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check if tag version matches project version<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">TAG=$(git describe HEAD --tags --abbrev=0)<\/span>\n          <span class=\"s\">echo $TAG<\/span>\n          <span class=\"s\">echo $PROJECT_VERSION<\/span>\n          <span class=\"s\">if [[ \"$TAG\" != \"v$PROJECT_VERSION\" ]]; then exit 1; fi<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This is a bit of a guardrail because of how I trigger the autorelease. I update the <code>pyproject.toml<\/code> version on my local clone using <code>poetry version &lt;version&gt;<\/code>, commit it to <code>main<\/code>, then tag it with the same <code>&lt;version&gt;<\/code> and push the commit and the tag, which then starts this workflow. We need to ensure that the version tag and the <code>pyproject.toml<\/code> versions match (in case we forget to bump versions properly).<\/p>\n\n<p>Then, we do the same dependency and virtualenv setup as in my Build workflow using Poetry, then run pytest and <code>poetry build<\/code>. The build artifacts will be used when we create the release in the final step of this workflow.<\/p>\n\n<p>The next step is to create some release notes. I keep a release template in the <code>.github<\/code> folder and append some gitlog output to it:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code> <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Release Notes<\/span>\n        <span class=\"s\">run<\/span><span class=\"err\">:<\/span> <span class=\"s\">git log $(git describe HEAD~ --tags --abbrev=0)..HEAD --pretty='format:* %h %s%n  * %an &lt;%ae&gt;' --no-merges &gt;&gt; \".github\/RELEASE-TEMPLATE.md\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>That gnarly gitlog command is checking all commits since the last tag to HEAD. For each commit, it appends the commit hash, the commit message subject, the author name, and the author email to the release template.<\/p>\n\n<p>Finally, we use a <a href=\"https:\/\/github.com\/softprops\/action-gh-release\">3rd-party release creation Action<\/a> for creating a release draft with the release notes and artifacts we just created:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Create Release Draft<\/span>\n        <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">softprops\/action-gh-release@v1<\/span>\n        <span class=\"na\">with<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">body_path<\/span><span class=\"pi\">:<\/span> <span class=\"s2\">\"<\/span><span class=\"s\">.github\/RELEASE-TEMPLATE.md\"<\/span>\n          <span class=\"na\">draft<\/span><span class=\"pi\">:<\/span> <span class=\"no\">true<\/span>\n          <span class=\"na\">files<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n            <span class=\"s\">dist\/random_standup-${{env.PROJECT_VERSION}}-py3-none-any.whl<\/span>\n            <span class=\"s\">dist\/random-standup-${{env.PROJECT_VERSION}}.tar.gz<\/span>\n        <span class=\"na\">env<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">GITHUB_TOKEN<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ secrets.GITHUB_TOKEN }}<\/span>\n          <span class=\"na\">GITHUB_REPOSITORY<\/span><span class=\"pi\">:<\/span> <span class=\"s\">jidicula\/random-standup<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This creates a draft visible at <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/releases\">https:\/\/github.com\/jidicula\/random-standup-py\/releases<\/a>. I modify the release announcements as needed, and publish the release.<\/p>\n\n<h2>\n  \n  \n  Publishing to PyPI\n<\/h2>\n\n<p>The final step of the release process is to publish the package release to the Python Package Index along with the release assets. Here's the full <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/blob\/main\/.github\/workflows\/publish.yml\">workflow<\/a>.<\/p>\n\n<p>This time, we trigger the workflow to run on a release being published (the last step of the previous workflow is manually publishing a release draft):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">release<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">types<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"s\">published<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>We do the same checkout and Poetry setup as before. Then, we simply run <code>poetry publish --build<\/code> using a PyPI token as a GitHub Secrets environment variable for authentication:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Publish to PyPI<\/span>\n        <span class=\"na\">env<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">PYPI_TOKEN<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ secrets.PYPI_TOKEN }}<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">poetry config pypi-token.pypi $PYPI_TOKEN<\/span>\n          <span class=\"s\">poetry publish --build<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h1>\n  \n  \n  Putting it all together\n<\/h1>\n\n<p>So overall, working on this project would involve:<\/p>\n\n<ol>\n<li>Make a PR for my changes.<\/li>\n<li>Confirm auto-merge.<\/li>\n<li>Repeeat Steps 1 and 2 until I'm ready to release.<\/li>\n<li>Bump the <code>pyproject.toml<\/code> version on my local clone using <code>poetry version &lt;new_version&gt;<\/code>. Commit the changes.<\/li>\n<li>Create a tag on <code>main<\/code> pointing to the version bump commit.<\/li>\n<li>Push both the tag and the version bump commit to GitHub.<\/li>\n<li>Wait for the <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/actions\/workflows\/release.yml\">Create Release<\/a> run to finish.<\/li>\n<li>Go to <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/releases\">https:\/\/github.com\/jidicula\/random-standup-py\/releases<\/a> and modify the Announcements for the just-created release draft.<\/li>\n<li>Publish the release.<\/li>\n<li>Wait for the <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/actions\/workflows\/publish.yml\">PyPI Publish<\/a> run to finish.<\/li>\n<li>Check <a href=\"https:\/\/pypi.org\/project\/random-standup\/\">PyPI<\/a> for the updated package version.<\/li>\n<\/ol>\n\n<p>If you have any questions or comments, email me at <a href=\"mailto:johanan+blog@forcepush.tech\">johanan+blog@forcepush.tech<\/a> or post a comment below.<\/p>\n\n<p>Did you find this post useful? Buy me a beverage or sponsor me <a href=\"https:\/\/github.com\/sponsors\/jidicula\">here<\/a>!<\/p>\n\n","category":["python","github","poetry","devops"]},{"title":"Writing a Simple CLI Program: Python vs Go","pubDate":"Fri, 07 May 2021 22:20:22 +0000","link":"https:\/\/dev.to\/jidicula\/writing-a-simple-cli-program-python-vs-go-59kf","guid":"https:\/\/dev.to\/jidicula\/writing-a-simple-cli-program-python-vs-go-59kf","description":"<p>As I mentioned in a <a href=\"https:\/\/forcepush.tech\/how-the-dnd-digital-hr-app-dev-team-does-agile-rituals\" rel=\"noopener noreferrer\">previous post<\/a>, I'm currently the Scrum Master of the DND Digital HR AppDev team. One of my duties is running the daily standup meetings, where each team member gives their update on what they're up to and if they have any impediments. Since we're a distributed team, we have our standups via an audio call. When I first started running standups, I found that when we did popcorn updates, there was always an awkward pause between updates because no one wanted to go next. I quickly decided on having a defined order in standups, but to also randomize the order to keep things varied. This was pretty simple to implement in a hardcoded Python script that uses <code>random.shuffle()<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"kn\">import<\/span> <span class=\"n\">random<\/span>\n<span class=\"kn\">from<\/span> <span class=\"n\">datetime<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">date<\/span>\n\n<span class=\"n\">members<\/span> <span class=\"o\">=<\/span> <span class=\"p\">[<\/span><span class=\"sh\">\"<\/span><span class=\"s\">Alice<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">Bob<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">Carol<\/span><span class=\"sh\">\"<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">David<\/span><span class=\"sh\">\"<\/span><span class=\"p\">]<\/span>\n<span class=\"n\">random<\/span><span class=\"p\">.<\/span><span class=\"nf\">shuffle<\/span><span class=\"p\">(<\/span><span class=\"n\">members<\/span><span class=\"p\">)<\/span>\n<span class=\"nf\">print<\/span><span class=\"p\">(<\/span><span class=\"sa\">f<\/span><span class=\"sh\">\"<\/span><span class=\"s\"># <\/span><span class=\"si\">{<\/span><span class=\"n\">date<\/span><span class=\"p\">.<\/span><span class=\"nf\">today<\/span><span class=\"p\">()<\/span><span class=\"si\">}<\/span><span class=\"s\">:<\/span><span class=\"se\">\\n<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">[<\/span><span class=\"nf\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">)<\/span> <span class=\"k\">for<\/span> <span class=\"n\">name<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">members<\/span><span class=\"p\">]<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>In the terminal, its invocation looked like:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ python standup-script-py\n# 2021-04-27\n\nDavid\nBob\nAlice\nCarol\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>and I could easily copy and paste this output into our meeting chat a few minutes before we started so everyone would know the order ahead of time. Not a well-engineered program by any means, but it did the job.<\/p>\n\n<h1>\n  \n  \n  The Rewrite\n<\/h1>\n\n<p>A few weeks ago, I began learning Go. I like it, <em>a lot<\/em>. Go seems very C-like, without the manual memory management footguns and a small syntax simpler than even C's already-sparse lexicon. As part of my journey, I thought it would be an interesting exercise to rewrite my little standup randomizer program in Go, with the following additional requirements:<\/p>\n\n<ul>\n<li>\n<a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/50789159f04985ee063f7fa92fc02cf7a83bb23f\/random-standup.go#L83\" rel=\"noopener noreferrer\">generalized<\/a>: no hardcoding of team members, preferably reading in a <a href=\"https:\/\/forcepush.tech\/why-toml\" rel=\"noopener noreferrer\">TOML<\/a> file defining the team roster<\/li>\n<li>covered by <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/50789159f04985ee063f7fa92fc02cf7a83bb23f\/random-standup_test.go\" rel=\"noopener noreferrer\">tests<\/a>\n<\/li>\n<li>publishable to <a href=\"https:\/\/pkg.go.dev\/github.com\/jidicula\/random-standup\" rel=\"noopener noreferrer\">pkg.go.dev<\/a>\n<\/li>\n<li>\n<a href=\"https:\/\/github.com\/jidicula\/random-standup\/tree\/main#usage\" rel=\"noopener noreferrer\">installable to PATH<\/a> with <a href=\"https:\/\/golang.org\/ref\/mod#go-get\" rel=\"noopener noreferrer\"><code>go get<\/code><\/a> (I discovered <a href=\"https:\/\/golang.org\/ref\/mod#go-install\" rel=\"noopener noreferrer\"><code>go install<\/code><\/a> later)<\/li>\n<li>pure CI\/CD <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/build.yml\" rel=\"noopener noreferrer\">PR checks<\/a> and <a href=\"https:\/\/github.com\/jidicula\/random-standup\/blob\/main\/.github\/workflows\/release-draft.yml\" rel=\"noopener noreferrer\">autoreleases<\/a>\n<\/li>\n<\/ul>\n\n<p>This was the final result: <a href=\"https:\/\/github.com\/jidicula\/random-standup\" rel=\"noopener noreferrer\">https:\/\/github.com\/jidicula\/random-standup<\/a>.<\/p>\n\n<p>It uses a team roster TOML that looks like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight toml\"><code><span class=\"nn\">[Subteam-1]<\/span>\n<span class=\"py\">members<\/span> <span class=\"p\">=<\/span> <span class=\"p\">[<\/span>\n        <span class=\"s\">\"Alice\"<\/span><span class=\"p\">,<\/span>                <span class=\"c\"># TOML spec allows whitespace to break arrays<\/span>\n        <span class=\"s\">\"Bob\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"Carol\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"David\"<\/span>\n        <span class=\"p\">]<\/span>\n\n<span class=\"nn\">[\"Subteam 2\"]<\/span>                   <span class=\"c\"># Keys can have whitespace in quoted strings<\/span>\n<span class=\"py\">members<\/span> <span class=\"p\">=<\/span> <span class=\"p\">[<\/span><span class=\"s\">\"Erin\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Frank\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Grace\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Heidi\"<\/span><span class=\"p\">]<\/span>\n\n<span class=\"nn\">[\"Empty Subteam\"]<\/span>               <span class=\"c\"># Subteam with 0 members won't be printed<\/span>\n\n<span class=\"nn\">[\"Subteam 3\"]<\/span>\n<span class=\"py\">members<\/span> <span class=\"p\">=<\/span> <span class=\"p\">[<\/span>\n        <span class=\"s\">\"Ivan\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"Judy\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"Mallory\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"Niaj\"<\/span>\n<span class=\"p\">]<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>When invoked, the program outputs:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ random-standup example-roster.toml\n# 2021-03-27\n## Subteam-1\nAlice\nDavid\nBob\nCarol\n\n## Subteam 2\nGrace\nHeidi\nFrank\nErin\n\n## Subteam 3\nJudy\nNiaj\nIvan\nMallory\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h1>\n  \n  \n  The Re-rewrite: Python Edition\n<\/h1>\n\n<p>I thought it would be an even more interesting exercise to try writing the same tool in Python too, just to compare the process of writing a CLI tool (and so I'd have a reason to write a blogpost). The Python implementation of this tool can be seen <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\" rel=\"noopener noreferrer\">here<\/a>. It accepts the same TOML file that the Go implementation accepts, and is invoked in the same way.<\/p>\n\n<h1>\n  \n  \n  The Differences\n<\/h1>\n\n<h2>\n  \n  \n  Project Structure\n<\/h2>\n\n<p>The Go implementation is very simple in this regard. Indeed, Go notably doesn't have any requirements about where files and folders should be located, despite some groups that <a href=\"https:\/\/github.com\/golang-standards\/project-layout\/issues\/117#issuecomment-828503689\" rel=\"noopener noreferrer\">claim otherwise<\/a>. The only real Go code in my repo are 2 <code>.go<\/code> files (1 for the program and 1 for its tests) and the <code>go.mod<\/code> and <code>go.sum<\/code> manifest files. There was only one sticky point I ran into, where I initially incorrectly defined the module name in <code>go.mod<\/code> - this has to match the repository name (<code>github.com\/jidicula\/random-standup<\/code>).<\/p>\n\n<p>For me, figuring this out with Python wasn't easy, especially when using <a href=\"https:\/\/www.python.org\/dev\/peps\/pep-0631\/\" rel=\"noopener noreferrer\"><code>pyproject.toml<\/code> for dependency specification<\/a>. I opted to use <a href=\"https:\/\/python-poetry.org\" rel=\"noopener noreferrer\">Poetry<\/a> to organize my dependencies and project settings (more on this later), and Poetry has a builtin command (<code>poetry new &lt;packagename&gt;<\/code>) for creating a \"recommended\" project structure that looks somewhat like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>foo-bar\n\u251c\u2500\u2500 README.rst\n\u251c\u2500\u2500 foo_bar\n\u2502   \u2514\u2500\u2500 __init__.py\n\u251c\u2500\u2500 pyproject.toml\n\u2514\u2500\u2500 tests\n    \u251c\u2500\u2500 __init__.py\n    \u2514\u2500\u2500 test_foo_bar.py\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>(This pretty filetree output is courtesy of <a href=\"http:\/\/mama.indstate.edu\/users\/ice\/tree\/\" rel=\"noopener noreferrer\"><code>tree<\/code><\/a>, which is also available via Homebrew.)<\/p>\n\n<p>This seemed to be a format more geared towards a Python package intended to be a library imported by other projects - probably overkill for a CLI tool. After some digging, I instead followed the <a href=\"https:\/\/packaging.python.org\/tutorials\/packaging-projects\/#creating-the-package-files\" rel=\"noopener noreferrer\">structure recommended by the Python Packaging Authority<\/a>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>packaging_tutorial\/\n\u251c\u2500\u2500 LICENSE\n\u251c\u2500\u2500 pyproject.toml\n\u251c\u2500\u2500 README.md\n\u251c\u2500\u2500 setup.cfg\n\u251c\u2500\u2500 setup.py  # optional, needed to make editable pip installs work\n\u251c\u2500\u2500 src\/\n\u2502   \u2514\u2500\u2500 example_pkg\/\n\u2502       \u2514\u2500\u2500 __init__.py\n\u2514\u2500\u2500 tests\/\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The key here is that the package's source code is in <code>project_name\/src\/package_name\/some_name.py<\/code> and its tests are in <code>project_name\/tests\/test_some_name.py<\/code>, with the cursory <code>__init__.py<\/code> in directories containing <code>.py<\/code> files. While retracing my steps for this blogpost, I also came across <a href=\"https:\/\/docs.python-guide.org\/writing\/structure\/\" rel=\"noopener noreferrer\">this recommended structure<\/a> from The Hitchhiker's Guide to Python:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>foo\n\u251c\u2500\u2500 LICENSE\n\u251c\u2500\u2500 README.rst\n\u251c\u2500\u2500 docs\n\u2502   \u251c\u2500\u2500 conf.py\n\u2502   \u2514\u2500\u2500 index.rst\n\u251c\u2500\u2500 requirements.txt\n\u251c\u2500\u2500 sample\n\u2502   \u251c\u2500\u2500 __init__.py\n\u2502   \u251c\u2500\u2500 core.py\n\u2502   \u2514\u2500\u2500 helpers.py\n\u251c\u2500\u2500 setup.py\n\u2514\u2500\u2500 tests\n    \u251c\u2500\u2500 test_advanced.py\n    \u2514\u2500\u2500 test_basic.py\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Overall, quite similar to what I went with, minus the <code>src\/<\/code> directory that doesn't seem to do much, and replacing <code>pyproject.toml<\/code> and <code>poetry.lock<\/code> with <code>setup.py<\/code> and <code>requirements.txt<\/code>. I didn't really try exploring different options at the time, as I wasn't sure if Poetry would be able to build the wheels with different project structures.<\/p>\n\n<p>All of this to say, packaging and file structure wasn't as immediately obvious with Python as it was with Go.<\/p>\n\n<h2>\n  \n  \n  Packaging and Publishing\n<\/h2>\n\n<p>Go was ridiculously simple in this area too. All that's needed for listing on <a href=\"https:\/\/pkg.go.dev\/github.com\/jidicula\/random-standup\" rel=\"noopener noreferrer\">pkg.go.dev<\/a> is a valid <code>go.mod<\/code> file. The site also has some other <a href=\"https:\/\/go.dev\/about#best-practices\" rel=\"noopener noreferrer\">recommendations<\/a>, like a stable tagged version and a LICENSE file. Go's package registry requires no additional authentication - when you navigate to <code>pkg.go.dev\/github.com\/username\/repo-name<\/code>, it prompts you to trigger autopopulation of the package entry:<\/p>\n\n<p><a href=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2Fa46765733ab5dd5df185941ec563788c%2F7abe2%2Fpkg-prompt.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2Fa46765733ab5dd5df185941ec563788c%2F7abe2%2Fpkg-prompt.png\" alt=\"pkg-prompt\"><\/a><br>\n(You can add a version tag to the end of the URL, like <code>@v1.0.0<\/code>, to autopopulate a newly released version of your package.)<\/p>\n\n<p>There are also some other programmatic ways to trigger addition of a package to the registry, listed <a href=\"https:\/\/go.dev\/about#adding-a-package\" rel=\"noopener noreferrer\">here<\/a>.<\/p>\n\n<p>Once again, Python was not as simple as Go. My choice to use <a href=\"https:\/\/python-poetry.org\" rel=\"noopener noreferrer\">Poetry<\/a> certainly simplified the process - I just had to run <code>poetry publish --build<\/code> and follow the prompts for PyPI username and password authentication. It's even simpler in my CI configs, as Poetry and PyPI allow token-based authentication - my <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/blob\/545bf6a98e04450b256ab594514efadffe2755e1\/.github\/workflows\/publish.yml#L47-L52\" rel=\"noopener noreferrer\">GitHub Actions workflow publishing step<\/a> looks like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code>      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Publish to PyPI<\/span>\n        <span class=\"na\">env<\/span><span class=\"pi\">:<\/span>\n          <span class=\"na\">PYPI_TOKEN<\/span><span class=\"pi\">:<\/span> <span class=\"s\">${{ secrets.PYPI_TOKEN }}<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">|<\/span>\n          <span class=\"s\">poetry config pypi-token.pypi $PYPI_TOKEN<\/span>\n          <span class=\"s\">poetry publish --build<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If I was doing this in Python before I had learned about Go, this process would seem pretty straightforward. The main issue I have with even this simplified process is needing a PyPI account. This ostensibly helps prevent supply-chain attacks like <a href=\"https:\/\/arstechnica.com\/information-technology\/2021\/02\/supply-chain-attack-that-fooled-apple-and-microsoft-is-attracting-copycats\/\" rel=\"noopener noreferrer\">dependency confusion<\/a>, where an account shadows the name of an internal package, or typo-squats a popular package in the hopes that a fat-fingered developer types a bit too hastily. However, I'm not sure if PyPI actually does any vetting for malicious packages - my package wasn't checked, to the best of my knowledge. On the other hand, Go took the rather sensible route of deferring any security concerns to the forge (i.e. GitHub, GitLab, Bitbucket, etc) hosting a module's source code and having no authentication step of its own. Since Go modules are named by their location (forge and username) as well as the package name itself, it's a bit trickier to shadow a company-internal package name with one published to <code>pkg.go.dev<\/code>. Additionally, typo-squatting is certainly still possible.<\/p>\n\n<p>Poetry also handles the extremely messy Python landscape of handling dependencies and virtual environments. It has simple functionality for setting up a virtual environment and separating development from main dependencies by adhering to the <a href=\"https:\/\/www.python.org\/dev\/peps\/pep-0631\/\" rel=\"noopener noreferrer\">PEP 631<\/a> specification for <code>pyproject.toml<\/code>. Before Poetry came along, most projects would opt for using <code>requirements.txt<\/code> and <code>pip<\/code>-installing from there, or using <code>setup.py<\/code>, or using other Python implementations like Anaconda. None of these can handle all 3 of: version-locking of dependencies, version resolution of multiple packages having the same dependent, and virtual environment setup. Setting up this project using Poetry just involves:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ poetry shell      # creates virtual environment\n$ poetry install    # installs main and dev dependencies\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If I wasn't using Poetry, I'd likely have to use <code>setuptools<\/code> for package configuration - I don't really know much about this process, but it seems more complex just by virtue of having more steps (which usually means a wider surface for mistakes). Python's packaging tutorial first recommends using <code>pip<\/code> itself for <a href=\"https:\/\/packaging.python.org\/tutorials\/packaging-projects\/#generating-distribution-archives\" rel=\"noopener noreferrer\">building the distribution archives<\/a> from a project based on <code>setup.cfg<\/code> (preferred) or <code>setup.py<\/code> (recommended against), then using Twine for <a href=\"https:\/\/packaging.python.org\/tutorials\/packaging-projects\/#uploading-the-distribution-archives\" rel=\"noopener noreferrer\">uploading the build artifacts<\/a>. I'm not familiar with this tooling, but the fact that there's no single PyPI-blessed approach is a cause for confusion in itself.<\/p>\n\n<h2>\n  \n  \n  Distribution\n<\/h2>\n\n<p>Go compiles to a single native binary - it includes everything it needs for its runtime without needing to link to any system libraries (unless you take the longer route and explicitly link to them). The clearest strength of this batteries-included approach is that the Go compiler allows you to cross-compile to 44 OS\/architecture pairs using the <code>GOOS<\/code> and <code>GOARCH<\/code> environment variables! This means that you can just run the native binary on any of these 44 OS and architecture combinations without any additional dependencies. You can see all the OS\/arch pairs by running <code>go tool dist list<\/code>. This batteries-included approach has a clear downside, though. A simple Hello, World program would contain:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"s\">\"fmt\"<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Println<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Hello, World!\"<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>When it's compiled with Go 1.16.3 on macOS 10.15.7 on an Intel processor, the Hello, World binary famously weighs in at a whopping 1.9 MB. The first release of the Go implementation of <code>random-standup<\/code> <a href=\"https:\/\/github.com\/jidicula\/random-standup\/releases\/tag\/v1.0.0\" rel=\"noopener noreferrer\">is around 3 MB<\/a>.<\/p>\n\n<p>The Python implementation isn't as simple to distribute across such a wide variety of hardware. Because Python is an interpreted language, executing Python source code requires a Python runtime (with the correct version!) to be already installed on the host machine. Installing and configuring that can be tedious or sometimes impossible for embedded systems - even on personal computers, managing multiple Python versions is a <a href=\"https:\/\/xkcd.com\/1987\/\" rel=\"noopener noreferrer\">headache<\/a>. On the size front, a Stack Overflow question suggests that the Python interpreter is <a href=\"https:\/\/stackoverflow.com\/q\/12631577\/6310633\" rel=\"noopener noreferrer\">roughly 1 MB in size<\/a>. Coupled with the <a href=\"https:\/\/github.com\/jidicula\/random-standup-py\/releases\/tag\/v1.0.0\" rel=\"noopener noreferrer\">4 KB wheel size<\/a> of <code>random-standup-py<\/code> v1.0.0, we're taking up less than half the space of the Go implementation's binary, with 99.6% of that space going towards a runtime that can be reused for other programs.<\/p>\n\n<h2>\n  \n  \n  Testing\n<\/h2>\n\n<p>Go has fantastic testing support built in. A common Go pattern is to use table-driven tests, where you create a map or list of structs containing the inputs for the function you're testing, and the desired output. In this table, I'm creating a table for testing a function I wrote to accept a slice of a subteam's members' names and the subteam's name and return a stringified shuffled list of the members. The test cases are stored in a map, with the test name as the key, and the struct representing the test table as the value.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"n\">tests<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">map<\/span><span class=\"p\">[<\/span><span class=\"kt\">string<\/span><span class=\"p\">]<\/span><span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">teamMembers<\/span> <span class=\"p\">[]<\/span><span class=\"kt\">string<\/span>\n    <span class=\"n\">teamName<\/span>    <span class=\"kt\">string<\/span>\n    <span class=\"n\">want<\/span>        <span class=\"kt\">string<\/span>\n<span class=\"p\">}{<\/span>\n    <span class=\"s\">\"four names\"<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"p\">[]<\/span><span class=\"kt\">string<\/span><span class=\"p\">{<\/span><span class=\"s\">\"Alice\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Bob\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"Carol\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"David\"<\/span><span class=\"p\">},<\/span>\n        <span class=\"s\">\"Subteam 1\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"s\">\"## Subteam 1<\/span><span class=\"se\">\\n<\/span><span class=\"s\">Carol<\/span><span class=\"se\">\\n<\/span><span class=\"s\">Bob<\/span><span class=\"se\">\\n<\/span><span class=\"s\">Alice<\/span><span class=\"se\">\\n<\/span><span class=\"s\">David<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">},<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I would then iterate through the map, setting up a testing harness upon each iteration. Inside the testing harness, I call the function being tested, and check if the return result of <code>shuffleTeam(tests[\"four names\"].teamMembers, tests[\"four names\"].teamName)<\/code> matches <code>tests[\"four names\"].want<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">for<\/span> <span class=\"n\">name<\/span><span class=\"p\">,<\/span> <span class=\"n\">tt<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">tests<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Run<\/span><span class=\"p\">(<\/span><span class=\"n\">name<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span> <span class=\"o\">*<\/span><span class=\"n\">testing<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">rand<\/span><span class=\"o\">.<\/span><span class=\"n\">Seed<\/span><span class=\"p\">(<\/span><span class=\"m\">0<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">got<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">shuffleTeam<\/span><span class=\"p\">(<\/span><span class=\"n\">tt<\/span><span class=\"o\">.<\/span><span class=\"n\">teamMembers<\/span><span class=\"p\">,<\/span> <span class=\"n\">tt<\/span><span class=\"o\">.<\/span><span class=\"n\">teamName<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">if<\/span> <span class=\"n\">got<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">tt<\/span><span class=\"o\">.<\/span><span class=\"n\">want<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Errorf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%s: got %s, want %s\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">name<\/span><span class=\"p\">,<\/span> <span class=\"n\">got<\/span><span class=\"p\">,<\/span> <span class=\"n\">tt<\/span><span class=\"o\">.<\/span><span class=\"n\">want<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Dave Cheney wrote an <a href=\"https:\/\/dave.cheney.net\/2019\/05\/07\/prefer-table-driven-tests\" rel=\"noopener noreferrer\">excellent blogpost about table-driven testing<\/a> where he recommends using a map for storing the test cases for 2 reasons:<\/p>\n\n<ol>\n<li>The map keys indicate the name of a test case, so you can easily find which case failed.<\/li>\n<li>\n<a href=\"https:\/\/golang.org\/ref\/spec#For_statements\" rel=\"noopener noreferrer\">Map iteration order is undefined in Go<\/a>, which means using a map can help you sniff out conditions where tests only pass in a defined order.<\/li>\n<\/ol>\n\n<p>All the unit tests can simply be run with <code>go test<\/code> - no 3rd-party tooling required. There's an even cooler tool built into the language runtime, though: test coverage. Rob Pike wrote about <a href=\"https:\/\/blog.golang.org\/cover\" rel=\"noopener noreferrer\">test coverage tooling in Go<\/a> that's simply brilliant in design and function. The tl;dr is that you can run:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ go test -coverprofile=coverage.out\nPASS\ncoverage: 55.8% of statements\nok      github.com\/jidicula\/random-standup  0.030s\n$ go tool cover -html=coverage.out\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The second command will open a browser window displaying the program's source code, colour-coded by coverage:<br>\n<a href=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2F8b16961bc26d1c21cb840073618bbe26%2F7d769%2Fgo-cover.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2F8b16961bc26d1c21cb840073618bbe26%2F7d769%2Fgo-cover.png\" alt=\"go test coverage\"><\/a><\/p>\n\n<p>If you run<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ go test -covermode=count -coverprofile=count.out\nPASS\ncoverage: 55.8% of statements\nok      github.com\/jidicula\/random-standup  0.010s\n$ go tool cover -html=count.out\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>you get a heatmap of test coverage, where colour intensity indicates how many times a line is covered by unit testing:<br>\n<a href=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2F2ddd297ec80171993a9ef1409e5a8e82%2F7d769%2Fgo-cover-heatmap.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforcepush.tech%2Fstatic%2F2ddd297ec80171993a9ef1409e5a8e82%2F7d769%2Fgo-cover-heatmap.png\" alt=\"go test coverage heatmap\"><\/a><\/p>\n\n<p>(Of course, you can get textual output for coverage as well.)<\/p>\n\n<p>Python unfortunately doesn't have great test support built-in (it has <code>unittest<\/code>, but it's a bit cumbersome), which has led to the rise of the 3rd-party (noticing a pattern?) tool <a href=\"https:\/\/docs.pytest.org\/en\/latest\/contents.html\" rel=\"noopener noreferrer\">pytest<\/a> as the de facto testing standard. pytest is quite easy to set up tests for, though. The test discovery rules are specified <a href=\"https:\/\/docs.pytest.org\/en\/latest\/explanation\/goodpractices.html#test-discovery\" rel=\"noopener noreferrer\">here<\/a> but essentially pytest will run any function with the <code>test<\/code> prefix in any file that matches <code>test_*.py<\/code> or <code>*_test.py<\/code>. In those test functions, <code>assert<\/code> statements are used for defining and checking test cases - if they fail, the entire test fails:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"k\">def<\/span> <span class=\"nf\">test_standup_cli<\/span><span class=\"p\">():<\/span>\n    <span class=\"n\">runner<\/span> <span class=\"o\">=<\/span> <span class=\"nc\">CliRunner<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">result<\/span> <span class=\"o\">=<\/span> <span class=\"n\">runner<\/span><span class=\"p\">.<\/span><span class=\"nf\">invoke<\/span><span class=\"p\">(<\/span><span class=\"n\">standup<\/span><span class=\"p\">,<\/span> <span class=\"p\">[<\/span><span class=\"sh\">\"<\/span><span class=\"s\">example-roster.toml<\/span><span class=\"sh\">\"<\/span><span class=\"p\">])<\/span>\n    <span class=\"k\">assert<\/span> <span class=\"n\">result<\/span><span class=\"p\">.<\/span><span class=\"n\">exit_code<\/span> <span class=\"o\">==<\/span> <span class=\"mi\">0<\/span>\n    <span class=\"k\">assert<\/span> <span class=\"nf\">str<\/span><span class=\"p\">(<\/span><span class=\"n\">date<\/span><span class=\"p\">.<\/span><span class=\"nf\">today<\/span><span class=\"p\">())<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">result<\/span><span class=\"p\">.<\/span><span class=\"n\">output<\/span>\n    <span class=\"k\">assert<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">## Subteam-1<\/span><span class=\"sh\">\"<\/span> <span class=\"ow\">in<\/span> <span class=\"n\">result<\/span><span class=\"p\">.<\/span><span class=\"n\">output<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This test is invoked by running:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ pytest\n============================= test session starts ==============================\nplatform darwin -- Python 3.8.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1\nrootdir: \/Users\/johanan\/prog\/random-standup-py\ncollected 1 item                                                               \n\ntests\/test_random_standup.py .                                           [100%]\n\n============================== 1 passed in 0.07s ===============================\n\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>For the Python implementation of this program, this was the only test I included - it's incomplete and doesn't test the logic thoroughly, but I wasn't interesting in achieving better coverage for a simple exercise. The interesting portion is that I'm doing a black-box test here - I'm capturing the output of the entire program, not testing specific units within it. I didn't attempt this in Go, but it would be interesting to investigate if\/how that can be done.<\/p>\n\n<p>Getting the test coverage isn't as straightforward as it is in Go - although I didn't attempt it for this program, on <a href=\"https:\/\/github.com\/shimming-toolbox\/shimming-toolbox\" rel=\"noopener noreferrer\">other projects<\/a> I've used additional 3rd-party services like <a href=\"https:\/\/coveralls.io\" rel=\"noopener noreferrer\">Coveralls<\/a> that have <a href=\"https:\/\/docs.coveralls.io\/python\" rel=\"noopener noreferrer\">3rd-party packages<\/a> for computing coverage.<\/p>\n\n<h2>\n  \n  \n  CLI Setup\n<\/h2>\n\n<p>Go has 2 builtin options for building a CLI interface: <code>os.Args<\/code>, which is a variable in the <code>os<\/code> package that holds a slice of strings representing the CLI arguments to the program, or the <code>flag<\/code> package, which provides some convenience utilities for parsing CLI flags as well as arguments. For example, <code>flag.Arg(0)<\/code> prints the first non-flag argument passed to the program. <code>flag<\/code> also has a <code>Usage()<\/code> function that can be shadowed for a custom help output that's printed to <code>stdout<\/code> when passing the flags <code>-h<\/code> or <code>--help<\/code> (<code>usage<\/code> is defined outside the <code>main()<\/code> function here):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">flag<\/span><span class=\"o\">.<\/span><span class=\"n\">Usage<\/span> <span class=\"o\">=<\/span> <span class=\"k\">func<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Fprintf<\/span><span class=\"p\">(<\/span><span class=\"n\">os<\/span><span class=\"o\">.<\/span><span class=\"n\">Stderr<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"%s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">usage<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"n\">flag<\/span><span class=\"o\">.<\/span><span class=\"n\">Parse<\/span><span class=\"p\">()<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">flag<\/span><span class=\"o\">.<\/span><span class=\"n\">NArg<\/span><span class=\"p\">()<\/span> <span class=\"o\">&lt;<\/span> <span class=\"m\">1<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">flag<\/span><span class=\"o\">.<\/span><span class=\"n\">Usage<\/span><span class=\"p\">()<\/span>\n        <span class=\"n\">os<\/span><span class=\"o\">.<\/span><span class=\"n\">Exit<\/span><span class=\"p\">(<\/span><span class=\"m\">1<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"n\">file<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">flag<\/span><span class=\"o\">.<\/span><span class=\"n\">Arg<\/span><span class=\"p\">(<\/span><span class=\"m\">0<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"c\">\/\/ rest of main()<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Python has a few builtin options for a CLI: <code>sys.argv<\/code>, which is analogous to Go's <code>os.Args<\/code> and is rather low-level in functionality, or <code>argparse<\/code>, which can handle flag and argument parsing. <a href=\"https:\/\/click.palletsprojects.com\" rel=\"noopener noreferrer\">Click<\/a> is another 3rd-party package that simplifies CLI setup using decorators to the function representing the CLI command:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight python\"><code><span class=\"nd\">@click.command<\/span><span class=\"p\">()<\/span>\n<span class=\"nd\">@click.argument<\/span><span class=\"p\">(<\/span><span class=\"sh\">\"<\/span><span class=\"s\">rosterfile<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span>\n<span class=\"k\">def<\/span> <span class=\"nf\">standup<\/span><span class=\"p\">(<\/span><span class=\"n\">rosterfile<\/span><span class=\"p\">):<\/span>\n    <span class=\"sh\">\"\"\"<\/span><span class=\"s\">random-standup is a tool for randomizing the order of team member\n    updates in a standup meeting.\n    <\/span><span class=\"sh\">\"\"\"<\/span>\n    <span class=\"nf\">print<\/span><span class=\"p\">(<\/span><span class=\"n\">date<\/span><span class=\"p\">.<\/span><span class=\"nf\">today<\/span><span class=\"p\">())<\/span>\n    <span class=\"k\">with<\/span> <span class=\"nf\">open<\/span><span class=\"p\">(<\/span><span class=\"n\">rosterfile<\/span><span class=\"p\">,<\/span> <span class=\"sh\">\"<\/span><span class=\"s\">r<\/span><span class=\"sh\">\"<\/span><span class=\"p\">)<\/span> <span class=\"k\">as<\/span> <span class=\"n\">f<\/span><span class=\"p\">:<\/span>\n        <span class=\"n\">roster<\/span> <span class=\"o\">=<\/span> <span class=\"n\">f<\/span><span class=\"p\">.<\/span><span class=\"nf\">read<\/span><span class=\"p\">()<\/span>\n\n    <span class=\"n\">parsed_roster<\/span> <span class=\"o\">=<\/span> <span class=\"nf\">parse<\/span><span class=\"p\">(<\/span><span class=\"n\">roster<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The <code>@click.command()<\/code> decorator turns the <code>standup()<\/code> function into a CLI command, and the <code>@click.argument(\"rosterfile\")<\/code> gives a name to the required input arguments for the command that's injected into the help message. The help message is built from the function's docstring, and can be invoked using the <code>-h<\/code> or <code>--help<\/code> flags:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>$ standup --help\nUsage: standup [OPTIONS] ROSTERFILE\n\n  random-standup is a tool for randomizing the order of team member updates\n  in a standup meeting.\n\nOptions:\n  --help  Show this message and exit.\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If further options were added using the <code>@click.option()<\/code> decorator, those would also appear in the <code>Options:<\/code> list in the help message.<\/p>\n\n<p>Click also provides a nice interface for CLI black-box testing, as we saw earlier in Testing.<\/p>\n\n<h1>\n  \n  \n  Summary\n<\/h1>\n\n<p>Overall, I've been very impressed with Go's offerings for building a simple CLI tool. I've structured this piece to showcase how its builtin options are as good or better than Python's, where you often have to reach for many 3rd-party packages to simplify structure or get basic functionality. Go also clearly excels in tooling: its core support for testing, dependency management, and cross-compilation is miles ahead of anything that Python has.<\/p>\n\n<p>The main benefit of having quality functionality built into the language is being able to reduce the dependency surface for a simple program, which greatly simplifies maintainability. For my Go program, I only have one 3rd-party dependency for parsing TOML (<code>go-toml<\/code>), which itself only depends on <code>go-spew<\/code> for pretty-printing its tree-based data structures. The dependency graph for the Python implementation is far more complex, even though it has only 3 core (there are more for formatting and linting) 3rd-party dependencies: Click, pytest, and TOML kit.<\/p>\n\n<p>The biggest downside I see for using Go for writing a simple CLI program are its rather imperative semantics - it's still faster for me to go from thought to code in Python, as evidenced by my first script for shuffling a list. However, as program size, complexity, or performance needs increases, or if you want to even a few quality-of-life improvements like testing or multiplatform support, I see Go pulling far ahead of Python.<\/p>\n\n<p>Did you find this post useful? Buy me a beverage or sponsor me <a href=\"https:\/\/github.com\/sponsors\/jidicula\" rel=\"noopener noreferrer\">here<\/a>!<\/p>\n\n","category":["python","go","cli"]}]}}