{"@attributes":{"version":"2.0"},"channel":{"title":"DEV Community: Forest Hoffman","description":"The latest articles on DEV Community by Forest Hoffman (@foresthoffman).","link":"https:\/\/dev.to\/foresthoffman","image":{"url":"https:\/\/media2.dev.to\/dynamic\/image\/width=90,height=90,fit=cover,gravity=auto,format=auto\/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F998%2F7031a597-4548-4326-85ef-3046755e0925.jpg","title":"DEV Community: Forest Hoffman","link":"https:\/\/dev.to\/foresthoffman"},"language":"en","item":[{"title":"Combining GitHub Secrets and Actions","pubDate":"Thu, 19 Aug 2021 21:33:39 +0000","link":"https:\/\/dev.to\/foresthoffman\/combining-github-secrets-and-actions-4081","guid":"https:\/\/dev.to\/foresthoffman\/combining-github-secrets-and-actions-4081","description":"<p>While searching for a <a href=\"https:\/\/docs.microsoft.com\/en-us\/devops\/develop\/what-is-continuous-integration\" rel=\"noopener noreferrer\">CI<\/a>\/<a href=\"https:\/\/azure.microsoft.com\/en-us\/overview\/continuous-delivery-vs-continuous-deployment\/\" rel=\"noopener noreferrer\">CD<\/a> solution for a side project of mine, I discovered the <a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/encrypted-secrets\" rel=\"noopener noreferrer\">GitHub Secrets<\/a> feature. Using <a href=\"https:\/\/docs.github.com\/en\/actions\" rel=\"noopener noreferrer\">GitHub Actions<\/a>, you can setup automated builds triggered in various customizable ways. If part of your automation pipeline includes deployment or delivery to a third-party, you\u2019re probably going to be using some kind of authorization token. Common knowledge dictates that storing super secret authorization files in plain text is a big no-no, so in comes GitHub Secrets to save the day!<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fakm4zr1b5iwn03dh34i5.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fakm4zr1b5iwn03dh34i5.gif\" alt=\"GIF of a cat dressed as a musketeer\" width=\"280\" height=\"222\"><\/a><\/p>\n\n<h2>\n  \n  \n  Table of Contents\n<\/h2>\n\n<ul>\n<li>Actions<\/li>\n<li>Secrets<\/li>\n<li>Credits<\/li>\n<\/ul>\n\n<h2>\n  \n  \n  Actions\n<\/h2>\n\n<p>GitHub Actions can be as simple or as complicated as you need them to be, but here\u2019s an Action that runs a Bash script from the target repository whenever a commit is directly made to the <code>main<\/code> branch:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Deploy Instance<\/span>\n\n<span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span> <span class=\"nv\">main<\/span> <span class=\"pi\">]<\/span>\n\n<span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">deploy<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s\">ubuntu-18.04<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Deploy<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">.\/script\/deploy<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>In this case, the Action runs a git-checkout on the repository and then executes the deploy script.<\/p>\n\n<h2>\n  \n  \n  Secrets\n<\/h2>\n\n<p>In order to define a GitHub Secret, you must navigate to the <em>Settings &gt; Secrets &gt; Actions<\/em> Page for either a specific repository, or if you\u2019re logged in as an organization, all repositories under that organization. The naming convention for this page is a bit odd. GitHub calls it, \u201cActions secrets\u201d, but the URL points to <code>&lt;GitHub repository url&gt;\/settings\/secrets\/actions<\/code>. \ud83e\udd14<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5znhchq7vvo94zfjzg3c.jpg\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5znhchq7vvo94zfjzg3c.jpg\" alt=\"A screenshot of the Secret creation form\" width=\"477\" height=\"476\"><\/a><\/p>\n\n<p>The documentation is quite clear regarding the rules regarding Secrets, such as:<\/p>\n\n<ul>\n<li>Names may only contain alphanumeric characters and underscores (e.g. <code>A-Z<\/code>, <code>a-z<\/code>, <code>0-9<\/code> or <code>_<\/code>)<\/li>\n<li>Names may not start with <code>GITHUB_<\/code>, as this prefix is reserved<\/li>\n<li>Secret values may not exceed 64KB\nThere are a few other rules, but they\u2019re not relevant to this post.<\/li>\n<\/ul>\n\n<p>In order to use a Secret in an Action workflow (pipeline), the format is: <code>${{ secrets.NAME }}<\/code>, where the <code>NAME<\/code> is whatever you called the Secret.<\/p>\n\n<p>For example, if you create a repository secret named, <code>MY_SECRET<\/code>, and give it a value of, <code>helloworld<\/code>, the value output by <code>${{ secrets.MY_SECRET }}<\/code> in any Action would be <code>helloworld<\/code>.<\/p>\n\n<p>Since Secrets are censored in log output on GitHub, you can\u2019t explicitly see that value being interpreted when the workflow is run. However, by executing an <code>if<\/code> statement in a Bash script run by the workflow, the actual value of the Secret can be confirmed. \ud83d\ude0e<\/p>\n\n<p>See, <code>check.sh<\/code> below:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">#!\/usr\/bin\/env bash<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$MY_SECRET<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">==<\/span> <span class=\"s2\">\"helloworld\"<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"that's the one!\"<\/span>\n<span class=\"k\">else\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"secret does not match\"<\/span>\n<span class=\"k\">fi<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>By committing the above Bash script, the Action workflow from previously can be updated to use the new Secret and check that its value is as expected. See <code>check.yml<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check<\/span>\n\n<span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span> <span class=\"nv\">main<\/span> <span class=\"pi\">]<\/span>\n\n<span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">check<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s\">ubuntu-18.04<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">chmod<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">chmod +x .\/scripts\/*<\/span>\n\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">MY_SECRET=\"${{ secrets.MY_SECRET }}\" .\/scripts\/check.sh<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>After committing the above workflow to the <code>main<\/code> branch, the Action should be triggered, and it will pass the current Secret value to the <code>check.sh<\/code> script for validation. Since the Secret value is simple, the check should be a success!<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbh8lhdnic3vkww5y1hjg.jpg\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbh8lhdnic3vkww5y1hjg.jpg\" alt=\"A screenshot of the workflow log, showing a successful check of the Secret\" width=\"363\" height=\"106\"><\/a><\/p>\n\n<p>Now, let\u2019s spice it up a bit. Say you\u2019re trying to pass along some authentication key, e.g. an SSH key, a Azure DevOps\/Google Cloud Authentication token, a password, etc. Does the check succeed the same way? Let\u2019s see\u2026<\/p>\n\n<p>If the value of <code>MY_SECRET<\/code> is reassigned to something more complex like, <code>apple_microsoft_$123<\/code>, the <code>check.sh<\/code> script should be updated accordingly:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">#!\/usr\/bin\/env bash<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$MY_SECRET<\/span><span class=\"s2\">\"<\/span> <span class=\"o\">==<\/span> <span class=\"s1\">'apple_microsoft_$123'<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"that's the one!\"<\/span>\n<span class=\"k\">else\n  <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"secret does not match\"<\/span>\n<span class=\"k\">fi<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Remember that the quotes for the new value in <code>check.sh<\/code> should be single quotes (<code>\u2019<\/code>) so that the script won\u2019t interpret <code>$123<\/code> as a variable instead of part of the full value.<\/p>\n\n<p>Oops! Here we see that the check failed. \ud83d\ude15 The Secret is <em>not<\/em> equal to the value that it was reassigned to.<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F68wpawv6u5fm7s02wkw4.jpg\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F68wpawv6u5fm7s02wkw4.jpg\" alt=\"A screenshot of the workflow log, showing an unsuccessful check of the Secret\" width=\"362\" height=\"104\"><\/a><\/p>\n\n<p>So, what happened? Well, it turns out that Secrets embedded in workflows are actually interpreted separate from the job commands. So, when <code>${{ secrets.MY_SECRET }}<\/code> is run GitHub has already interpreted the value of the Secret and inserted it into the job command. The job ends up running, <code>MY_SECRET=\u201dapple_microsoft$123\u201d .\/scripts\/check.sh<\/code> instead of <code>MY_SECRET=apple_microsoft$123 .\/scripts\/check.sh<\/code>. Since the Secrets are interpreted first, the same change that was made to <code>check.sh<\/code> needs to be made to the workflow. Use single-quotes instead of double-quotes!<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight yaml\"><code><span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check<\/span>\n\n<span class=\"na\">on<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">push<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">branches<\/span><span class=\"pi\">:<\/span> <span class=\"pi\">[<\/span> <span class=\"nv\">main<\/span> <span class=\"pi\">]<\/span>\n\n<span class=\"na\">jobs<\/span><span class=\"pi\">:<\/span>\n  <span class=\"na\">check<\/span><span class=\"pi\">:<\/span>\n    <span class=\"na\">runs-on<\/span><span class=\"pi\">:<\/span> <span class=\"s\">ubuntu-18.04<\/span>\n    <span class=\"na\">steps<\/span><span class=\"pi\">:<\/span>\n      <span class=\"pi\">-<\/span> <span class=\"na\">uses<\/span><span class=\"pi\">:<\/span> <span class=\"s\">actions\/checkout@v2<\/span>\n\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">chmod<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">chmod +x .\/scripts\/*<\/span>\n\n      <span class=\"pi\">-<\/span> <span class=\"na\">name<\/span><span class=\"pi\">:<\/span> <span class=\"s\">Check<\/span>\n        <span class=\"na\">run<\/span><span class=\"pi\">:<\/span> <span class=\"s\">MY_SECRET='${{ secrets.MY_SECRET }}' .\/scripts\/check.sh<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now, the check should succeed and the special characters in the Secret won\u2019t break the intended behavior!<\/p>\n\n<p>It\u2019s the tiniest change, but wow\u2026I spent many hours figuring that out. \ud83d\ude05 Learn from my mistakes y\u2019all.<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fagpqo2kapm5cdt10o6vj.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fagpqo2kapm5cdt10o6vj.gif\" alt=\"Phew\" width=\"498\" height=\"258\"><\/a><\/p>\n\n<p>If you want to learn more about <a href=\"https:\/\/docs.github.com\/en\/actions\/reference\/encrypted-secrets#limits-for-secrets\" rel=\"noopener noreferrer\">GitHub Secrets and the limits they have<\/a>, check out the bottom of the documentation page, which explains how to encrypt larger secrets with GPG. Until next time, thanks for reading!<\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Cover image by <a href=\"https:\/\/unsplash.com\/@usinglight?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Stefan Steinbauer<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/secret?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>! :D<\/p>\n\n","category":["showdev","git","devops","webdev"]},{"title":"Intercepting RESTful Responses with Middleware","pubDate":"Thu, 14 Jan 2021 15:25:19 +0000","link":"https:\/\/dev.to\/foresthoffman\/intercepting-restful-responses-with-middleware-4b64","guid":"https:\/\/dev.to\/foresthoffman\/intercepting-restful-responses-with-middleware-4b64","description":"<p>When we look at building HTTP servers, we typically jump to a golden example of a client and a server passing back and forth data for creating, updating, reading, and deleting resources (i.e. <a href=\"https:\/\/www.tutorialspoint.com\/restful\/restful_introduction.htm\" rel=\"noopener noreferrer\">RESTful services<\/a>). This could be text files, binary data (like .mov, and .mp4 files), and even pre-formatted data (JSON, XML, etc.) for future use by another program. Multiple clients may communicate with the one server at a time, and the server always provides consistent data without any need for intervention. <\/p>\n\n<p>The golden example is contrived to simplify how we talk about the communication between clients and the server. More complex solutions, which tend to appear more frequently than most would think, are left out of the conversation. My particular focus in this article, is on the usage of third-party <a href=\"https:\/\/www.mulesoft.com\/resources\/api\/what-is-an-api\" rel=\"noopener noreferrer\">API<\/a>s in-house, and how to leverage the functionality that they provide, while still maintaining control over the responses clients receive. <\/p>\n\n<p>Assuming that an in-house server is not simply a middleman for the communication between the client and a third-party API, there should be some sort of quality control over the responses clients receive. The standard Go <a href=\"https:\/\/golang.org\/pkg\/net\/http\/\" rel=\"noopener noreferrer\"><code>net\/http<\/code><\/a> package provides some functionality to intercept incoming <em>requests<\/em> before they reach their intended target (<a href=\"https:\/\/golang.org\/pkg\/net\/http\/#Handler\" rel=\"noopener noreferrer\"><code>Handler<\/code><\/a>). However, there is no pre-baked method of intercepting outgoing <em>responses<\/em> before they reach the client.<\/p>\n\n<h2>\n  \n  \n  Table of Contents\n<\/h2>\n\n<ul>\n<li>Receiving Requests from Browser Clients<\/li>\n<li>Receiving Requests from Programmatic Clients<\/li>\n<li>Using third-party APIs<\/li>\n<li>Intercepting Requests<\/li>\n<li>Intercepting Responses<\/li>\n<li>Credits<\/li>\n<\/ul>\n\n<h2>\n  \n  \n  Receiving Requests from Browser Clients\n<\/h2>\n\n<p>To save time, let us talk about the most used request method, \u201cGET\u201d. A <a href=\"https:\/\/www.smashingmagazine.com\/2018\/01\/understanding-using-rest-api\/\" rel=\"noopener noreferrer\">GET<\/a> request, does what is says on the tin, it retrieves some data from storage (e.g. a post from DEV, a profile from GitHub, a feed from Twitter, etc.). When you enter a URL into your browser, like <a href=\"https:\/\/foresthoffman.com\" rel=\"noopener noreferrer\">https:\/\/foresthoffman.com<\/a>, your browser attempts to GET it by default. That is how we read things online.<\/p>\n\n<p>A server\u2019s job is to fulfill client requests. So, when a browser makes a GET request for <a href=\"https:\/\/foresthoffman.com\" rel=\"noopener noreferrer\">https:\/\/foresthoffman.com<\/a> it generates a request that gets routed to that server and waits for a response. In my case, my website responds with an HTML file, which requires a few other external files, which in turn causes the browser to make more GET requests, until it has completely loaded the website.<\/p>\n\n<p>For a server that receives GET requests programmatically, i.e. code talking to code, things get more interesting.<\/p>\n\n<h2>\n  \n  \n  Receiving Requests from Programmatic Clients\n<\/h2>\n\n<p>Assume we have a server expecting GET requests to a static endpoint, programmatic clients might pull from this endpoint and parse our response however they want. Here is an example of a localhost server hosting an <code>\/example<\/code> endpoint:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"s\">\"net\/http\"<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">exampleHandler<\/span><span class=\"p\">()<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusOK<\/span><span class=\"p\">)<\/span>\n      <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"hey\"<\/span><span class=\"p\">))<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handle<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\/example\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">exampleHandler<\/span><span class=\"p\">())<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ListenAndServe<\/span><span class=\"p\">(<\/span><span class=\"s\">\":9001\"<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>A client may then GET this endpoint by curling:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> <span class=\"s2\">\"GET\"<\/span> http:\/\/localhost:9001\/example\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The response body should contain: <code>hey<\/code>. This by itself does not seem complex, but imagine if the response was JSON formatted according to this server response structure:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">Response<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">Message<\/span> <span class=\"kt\">string<\/span> <span class=\"s\">`json:\"message\"`<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The handler would have to change accordingly:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">exampleHandler<\/span><span class=\"p\">()<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"n\">respBytes<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">Marshal<\/span><span class=\"p\">(<\/span><span class=\"n\">Response<\/span><span class=\"p\">{<\/span><span class=\"n\">Message<\/span><span class=\"o\">:<\/span> <span class=\"s\">\"hey\"<\/span><span class=\"p\">})<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusOK<\/span><span class=\"p\">)<\/span>\n      <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">respBytes<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The curl output should then be: <code>{\"message\":\"hey\"}<\/code>. Assuming the client can parse this JSON, a mirrored object structure could be created on the client-side. Cool!<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2o9krcmzy20yfutr9198.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F2o9krcmzy20yfutr9198.gif\" alt=\"Gif of a kid in front of a computer giving a thumbs up\" width=\"320\" height=\"240\"><\/a><\/p>\n\n<h2>\n  \n  \n  Using third-party APIs\n<\/h2>\n\n<p>For some tasks, it makes more sense to rely on functionality supported by third-party APIs, e.g. <a href=\"https:\/\/cloud.google.com\/apis\" rel=\"noopener noreferrer\">Google Cloud<\/a>, <a href=\"https:\/\/www.mongodb.com\/\" rel=\"noopener noreferrer\">MongoDB<\/a>, <a href=\"https:\/\/tus.io\/\" rel=\"noopener noreferrer\">Tus.io<\/a>, etc. This of course adds its own complexity to our server, since we do not control the responses provided by these third-party APIs. Let us add a \u201cthird-party\u201d handler now.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">thirdPartyHandler<\/span><span class=\"p\">()<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"n\">respBytes<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">Marshal<\/span><span class=\"p\">(<\/span><span class=\"k\">struct<\/span><span class=\"p\">{<\/span>\n         <span class=\"n\">Timestamp<\/span> <span class=\"kt\">string<\/span> <span class=\"s\">`json:\"timestamp\"`<\/span>\n      <span class=\"p\">}{<\/span>\n         <span class=\"n\">Timestamp<\/span><span class=\"o\">:<\/span> <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Sprint<\/span><span class=\"p\">(<\/span><span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Now<\/span><span class=\"p\">()<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()),<\/span>\n      <span class=\"p\">})<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusOK<\/span><span class=\"p\">)<\/span>\n      <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">respBytes<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This \u201cthird-party\u201d handler takes a request, gets the current date and formats it as a string of the nanoseconds since January 1st 1970. While this timestamp is handy, it does not conform to the structure we have implemented for our own handler.<\/p>\n\n<p>We can see the difference in the response structures after giving the third-party handler its own endpoint in <code>main<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handle<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\/third\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">thirdPartyHandler<\/span><span class=\"p\">())<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now, when a client makes a GET request for the new endpoint, they will see: <code>{\"timestamp\":\"1610378453927873100\"}<\/code>. Of course, the timestamp is based on the time at which the request was made, so the timestamp will change.<\/p>\n\n<p>We can see here that the handy third-party response does what we need, but could confuse clients, since the <code>{\"message\":\"hey\"}<\/code> and <code>{\"timestamp\":\"1610378453927873100\"}<\/code> structures do not line up. This is where HTTP middleware come into play.<\/p>\n\n<h2>\n  \n  \n  Intercepting Requests\n<\/h2>\n\n<p>A \u201cmiddleware\u201d, at its core is a HTTP handler. It has an input, the request, and an <em>optional<\/em> output, the response. The output is <em>optional<\/em>, because middlewares are not intended to be the end target of a request. They are middlemen, so they normally modify the request and pass it along to the next middleware or handler.<\/p>\n\n<p>They look like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">exampleMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">h<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span><span class=\"p\">)<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"n\">h<\/span><span class=\"o\">.<\/span><span class=\"n\">ServeHTTP<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This middleware returns a wrapping handler which calls any handler passed into it, so a new endpoint using this middleware could be created like so:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handle<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\/fourth\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">exampleMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">thirdPartyHandler<\/span><span class=\"p\">()))<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Curling this new endpoint should reveal the response we would expect from the <code>\/third<\/code> endpoint:<br>\n<code>{\"timestamp\":\"1610381096848493300\"}<\/code>. So, here we have our example of intercepting an HTTP request. We did not modify the request at all, but we did pass it along to a different handler.<\/p>\n\n<p>Let us explore modifying a request. This is very powerful, because it allows us to restrict access to certain handlers if need be.<\/p>\n\n<p>Here is a middleware that expects to find a secret key in the URL parameters (e.g. website.com?key=corgis-are-cute):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">restrictMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">h<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span><span class=\"p\">)<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">URL<\/span><span class=\"o\">.<\/span><span class=\"n\">Query<\/span><span class=\"p\">()<\/span><span class=\"o\">.<\/span><span class=\"n\">Get<\/span><span class=\"p\">(<\/span><span class=\"s\">\"super-secret-key\"<\/span><span class=\"p\">)<\/span> <span class=\"o\">==<\/span> <span class=\"s\">\"\"<\/span> <span class=\"p\">{<\/span>\n         <span class=\"n\">respBytes<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">Marshal<\/span><span class=\"p\">(<\/span><span class=\"n\">Response<\/span><span class=\"p\">{<\/span><span class=\"n\">Message<\/span><span class=\"o\">:<\/span> <span class=\"s\">\"key must not be empty\"<\/span><span class=\"p\">})<\/span>\n         <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n         <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusBadRequest<\/span><span class=\"p\">)<\/span>\n         <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">w<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">respBytes<\/span><span class=\"p\">)<\/span>\n         <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n         <span class=\"k\">return<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">h<\/span><span class=\"o\">.<\/span><span class=\"n\">ServeHTTP<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This middleware\u2019s new endpoint will wrap the first handler we created:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handle<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\/restrict\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">restrictMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">exampleHandler<\/span><span class=\"p\">()))<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now, when a client curls this endpoint without a secret key, they will be met with: <code>{\"message\":\"key must not be empty\"}<\/code>.<\/p>\n\n<p>To provide a key, clients must format their curl request like so:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>curl <span class=\"nt\">-X<\/span> <span class=\"s2\">\"GET\"<\/span> http:\/\/localhost:9001\/restrict?super-secret-key<span class=\"o\">=<\/span>secret\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, clients will be able to see the expected response: <code>{\"message\":\"hey\"}<\/code>!<\/p>\n\n<p>With this example, we can see that a server may restrict access to certain endpoints by preventing the target handler from ever handling the request.<\/p>\n\n<h2>\n  \n  \n  Intercepting Responses\n<\/h2>\n\n<p>We have already seen how useful middlewares can be when processing requests, but when it comes to unexpectedly abrupt responses from third-party handlers, they cannot help us. At least, not by default.<\/p>\n\n<p>In order to intercept a response from a chunk of code that we do not control, we must capture the response in memory while maintaining the response\u2019s integrity. We must ensure that the response is not sent to the client before we are done with it. Otherwise, any modifications we may make will be useless.<\/p>\n\n<p>By default, the <a href=\"https:\/\/golang.org\/pkg\/net\/http\/#ResponseWriter\" rel=\"noopener noreferrer\"><code>http.ResponseWriter<\/code><\/a> interface does not support reading, as it only features methods for writing. We can achieve the functionality we desire by creating a custom struct that implements the <code>http.ResponseWriter<\/code> interface. However, we do not want to completely throw away the features that the <code>http.ResponseWriter<\/code> provides, so we\u2019ll have to include the response writer as a field in our custom struct like so:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">responseSkimmer<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span>\n   <span class=\"n\">body<\/span> <span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">Buffer<\/span>\n   <span class=\"n\">header<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Header<\/span>\n   <span class=\"n\">status<\/span> <span class=\"kt\">int<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">rs<\/span> <span class=\"o\">*<\/span><span class=\"n\">responseSkimmer<\/span><span class=\"p\">)<\/span> <span class=\"n\">Header<\/span><span class=\"p\">()<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Header<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">rs<\/span><span class=\"o\">.<\/span><span class=\"n\">header<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">rs<\/span> <span class=\"o\">*<\/span><span class=\"n\">responseSkimmer<\/span><span class=\"p\">)<\/span> <span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">b<\/span> <span class=\"p\">[]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">)<\/span> <span class=\"p\">(<\/span><span class=\"kt\">int<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">rs<\/span><span class=\"o\">.<\/span><span class=\"n\">body<\/span><span class=\"o\">.<\/span><span class=\"n\">Reset<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">rs<\/span><span class=\"o\">.<\/span><span class=\"n\">body<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">b<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">rs<\/span> <span class=\"o\">*<\/span><span class=\"n\">responseSkimmer<\/span><span class=\"p\">)<\/span> <span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">code<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">rs<\/span><span class=\"o\">.<\/span><span class=\"n\">status<\/span> <span class=\"o\">=<\/span> <span class=\"n\">code<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Next, we can implement this \u201cskimmer\u201d in a middleware that will receive the third-party handler as a parameter. This will showcase how we can capture the response from the third-party handler, manipulate it, and respond with the new body as though nothing happened.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">func<\/span> <span class=\"n\">responseMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">h<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span><span class=\"p\">)<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handler<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">HandlerFunc<\/span><span class=\"p\">(<\/span><span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">w<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span> <span class=\"o\">*<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Request<\/span><span class=\"p\">){<\/span>\n      <span class=\"c\">\/\/ Intercept response.<\/span>\n      <span class=\"n\">skimmer<\/span> <span class=\"o\">:=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">responseSkimmer<\/span><span class=\"p\">{<\/span>\n         <span class=\"n\">ResponseWriter<\/span><span class=\"o\">:<\/span> <span class=\"n\">w<\/span><span class=\"p\">,<\/span>\n         <span class=\"n\">body<\/span><span class=\"o\">:<\/span> <span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">Buffer<\/span><span class=\"p\">{},<\/span>\n         <span class=\"n\">header<\/span><span class=\"o\">:<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Header<\/span><span class=\"p\">{},<\/span>\n         <span class=\"n\">status<\/span><span class=\"o\">:<\/span> <span class=\"m\">0<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">h<\/span><span class=\"o\">.<\/span><span class=\"n\">ServeHTTP<\/span><span class=\"p\">(<\/span><span class=\"n\">skimmer<\/span><span class=\"p\">,<\/span> <span class=\"n\">r<\/span><span class=\"p\">)<\/span>\n\n      <span class=\"c\">\/\/ Read.<\/span>\n      <span class=\"k\">if<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">body<\/span><span class=\"o\">.<\/span><span class=\"n\">Bytes<\/span><span class=\"p\">())<\/span> <span class=\"o\">==<\/span> <span class=\"m\">0<\/span> <span class=\"p\">{<\/span>\n         <span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">StatusNoContent<\/span><span class=\"p\">)<\/span>\n         <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\"<\/span><span class=\"p\">))<\/span>\n         <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n         <span class=\"k\">return<\/span>\n      <span class=\"p\">}<\/span>\n\n      <span class=\"c\">\/\/ Transform.<\/span>\n      <span class=\"n\">respBytes<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">Marshal<\/span><span class=\"p\">(<\/span><span class=\"n\">Response<\/span><span class=\"p\">{<\/span><span class=\"n\">Message<\/span><span class=\"o\">:<\/span> <span class=\"kt\">string<\/span><span class=\"p\">(<\/span><span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">body<\/span><span class=\"o\">.<\/span><span class=\"n\">Bytes<\/span><span class=\"p\">())})<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n\n      <span class=\"c\">\/\/ Write.<\/span>\n      <span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteHeader<\/span><span class=\"p\">(<\/span><span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">status<\/span><span class=\"p\">)<\/span>\n      <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">skimmer<\/span><span class=\"o\">.<\/span><span class=\"n\">ResponseWriter<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">(<\/span><span class=\"n\">respBytes<\/span><span class=\"p\">)<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n         <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"k\">return<\/span>\n   <span class=\"p\">})<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Lastly, we need to add an endpoint and pass along the new middleware and the existing handler.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Handle<\/span><span class=\"p\">(<\/span><span class=\"s\">\"\/intercept\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">responseMiddleware<\/span><span class=\"p\">(<\/span><span class=\"n\">thirdPartyHandler<\/span><span class=\"p\">()))<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Curling the new endpoint we should see the same response we saw from the third-party handler endpoint, <code>\/third<\/code>, but the difference is that it has been wrapped by the response structure we created for the <code>\/example<\/code> endpoint: <code>{\"message\":\"{\\\"timestamp\\\":\\\"1610599055718152700\\\"}\"}<\/code>.<\/p>\n\n<p>And there we have it. A middleware that works for outgoing responses, and not just incoming requests! Woo! <\/p>\n\n<p>If you made it this far, thank you for reading! Hopefully, you found a useful nugget of knowledge here. Cheers!<\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Cover image by <a href=\"https:\/\/unsplash.com\/@flo_stk?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Florian Steciuk<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/highway?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>! :D<\/p>\n\n","category":["showdev","go","webdev","tutorial"]},{"title":"On Mocking Functionality in Go Tests","pubDate":"Thu, 29 Oct 2020 18:53:21 +0000","link":"https:\/\/dev.to\/foresthoffman\/on-mocking-functionality-in-go-tests-1he0","guid":"https:\/\/dev.to\/foresthoffman\/on-mocking-functionality-in-go-tests-1he0","description":"<p>In my recent years of working with <a href=\"https:\/\/golang.org\" rel=\"noopener noreferrer\">Go<\/a>, I\u2019ve noticed a significant understanding barrier for newcomers, with regard to mocking third-party and internal logic when designing tests in Go.<\/p>\n\n<p>With some strictly Object-Oriented (OO) programming languages, we have access to class hierarchies and inheritance features that allow for overwriting certain logical dependencies associated with any class or program that we wish to test. In Go it\u2019s a bit more vague, because Go doesn\u2019t strictly subscribe to the Object-Oriented paradigm.<\/p>\n\n<p>The back-bone of Go is composed of Interfaces and Structures. Interfaces allow for grouping supported functionality into logical groups, much like Classes in other languages. Defining what a logical group <em>might<\/em> do is the job of an Interface. Whereas Structures house the actual implementation of the functionality. Structures describe what a logical group <em>actually<\/em> does, within the context of its Interface. The caveat is, Go does not support explicitly defining relationships between Interfaces and Structures.<\/p>\n\n<p>The lack of explicit class inheritance sounds convenient and less clunky than the typical Object-Oriented paradigm, but it does open us up to some potential hiccups. We could hard code the relationships between structures by avoiding using Interfaces and only using Structures, but our code would likely become rigid. Rigid code leads to rigid tests, which can allow bugs to squeeze through the cracks.<\/p>\n\n<p>By designing our features in a modular way, with appropriate use of Interfaces, we can write Go code that represents relationships between features in broader terms. By obfuscating each feature\u2019s logic from the next, each module becomes responsible for its own input and output rather than the program as a whole. Modular tests can then be expanded as needed to account for new standards of user input and potentially destructive input that the program must anticipate and handle appropriately. Generally, the more complex a program gets, the more modules are needed, third-party or otherwise.<\/p>\n\n<p>That\u2019s where Mocks come into play.<\/p>\n\n<h2>\n  \n  \n  What are Mocks?\n<\/h2>\n\n<p>Mocking is method by which we can override specific bits of code with fake logic, in order to test other aspects of our code without worrying about the reliability of the logic that we have faked.<\/p>\n\n<h2>\n  \n  \n  Why do we use Mocks?\n<\/h2>\n\n<p>We mock in our tests, to prevent having live input\/output\/etc. flowing through our program, when our test is only concerned with a small portion of our program\u2019s logic.<\/p>\n\n<p>Mocks are particularly good for omitting actions that would update live external sources, such as a database, a server, or a code repository.<\/p>\n\n<h2>\n  \n  \n  How can we design our programs to support Mocks?\n<\/h2>\n\n<p>As I described in the big wall of text at the start of the article, Mocks are best supported by a modular or functional design. Either paradigm avoids rigid relationships between pieces of logic in our programs, making our lives much easier.<\/p>\n\n<h2>\n  \n  \n  Our Program\n<\/h2>\n\n<p>Here\u2019s an example program that pulls recent articles from the <a href=\"https:\/\/dev.to\/api\">DEV API<\/a> and lists the date they were published as well as the title:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"bytes\"<\/span>\n   <span class=\"s\">\"encoding\/json\"<\/span>\n   <span class=\"s\">\"fmt\"<\/span>\n   <span class=\"s\">\"io\/ioutil\"<\/span>\n   <span class=\"s\">\"net\/http\"<\/span>\n   <span class=\"s\">\"sort\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n   <span class=\"c\">\/\/ Make the request.<\/span>\n   <span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Get<\/span><span class=\"p\">(<\/span><span class=\"s\">\"https:\/\/dev.to\/api\/articles?per_page=20\"<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Read the bytes from the response.<\/span>\n   <span class=\"n\">body<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ioutil<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n   <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Transform into usable JSON list.<\/span>\n   <span class=\"n\">decoder<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">NewDecoder<\/span><span class=\"p\">(<\/span><span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">body<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">PublishedAt<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Time<\/span> <span class=\"s\">`json:\"published_at\"`<\/span>\n      <span class=\"n\">Title<\/span>       <span class=\"kt\">string<\/span>    <span class=\"s\">`json:\"title\"`<\/span>\n   <span class=\"p\">}<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decoder<\/span><span class=\"o\">.<\/span><span class=\"n\">Decode<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Sort the list, newest to oldest.<\/span>\n   <span class=\"n\">sort<\/span><span class=\"o\">.<\/span><span class=\"n\">Slice<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">j<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"kt\">bool<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span>\n   <span class=\"p\">})<\/span>\n\n   <span class=\"c\">\/\/ Print the list.<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%v -- %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The output we see when running the above is something like:<\/p>\n\n<blockquote>\n<p>2020-10-27 17:58:59 +0000 UTC -- The 7 Most Popular DEV Posts from the Past Week<br>\n2020-10-27 17:40:25 +0000 UTC -- My Online Portfolio<br>\n2020-10-27 16:46:39 +0000 UTC -- Pairing with Community Member Rachael Wright-Munn<br>\n2020-10-27 14:37:07 +0000 UTC -- I\u2019m Levi Sharpe, Senior Podcast Producer at DEV\/Forem- AMA<br>\n2020-10-27 14:12:05 +0000 UTC -- A Sinatra Project<br>\n2020-10-27 13:21:46 +0000 UTC -- Projects to build that would get you hired as a beginner.<br>\n2020-10-27 13:18:16 +0000 UTC -- Create a Landing page in less than 100 lines (incl. CSS) \ufffd\ufffd<br>\n2020-10-27 13:17:06 +0000 UTC -- Forget pay cut, give me a raise to work remotely<br>\n2020-10-27 12:33:29 +0000 UTC -- How to hide API KEY in GitHub repo<br>\n2020-10-27 12:17:41 +0000 UTC -- How to Build a Secret Dark Mode Toggle for Your Blog<br>\n2020-10-27 11:11:57 +0000 UTC -- Open Hacktoberfest Issues on Scaffolder<br>\n2020-10-27 09:28:30 +0000 UTC -- Top 5 Free Awesome React.JS Material-UI Admin Dashboard Templates<br>\n2020-10-27 07:19:36 +0000 UTC -- 5 Productivity Tools for Mobile App Developers<br>\n2020-10-27 06:07:56 +0000 UTC -- Five things you should never say in a software developer interview<br>\n2020-10-27 04:31:29 +0000 UTC -- How to log user activities using the Beacon Web API?<br>\n2020-10-27 03:57:36 +0000 UTC -- An Intro to JSX<br>\n2020-10-27 02:33:43 +0000 UTC -- How do I work on multiple projects simultaneously without losing my mind<br>\n2020-10-26 23:34:37 +0000 UTC -- JavaScript Challenge 6: Convert string to camel case<br>\n2020-10-26 21:28:26 +0000 UTC -- Final week of Hacktoberfest!<br>\n2020-10-26 20:26:02 +0000 UTC -- React Hooks: Managing State With useState Hook<\/p>\n<\/blockquote>\n\n<h2>\n  \n  \n  Adding Complexity\n<\/h2>\n\n<p>Now this is a fairly simple program that will <a href=\"https:\/\/gobyexample.com\/panic\" rel=\"noopener noreferrer\"><code>panic<\/code><\/a> if it critically fails. So, simply building and running it will give a good indication of the health of the program. However, if we want to achieve some more complicated results, things get more interesting.<\/p>\n\n<p>We are already sorting the DEV articles by publishing date, in ascending order. Let\u2019s filter the results a bit. See the line starting with <code>\/** UPDATE:<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"bytes\"<\/span>\n   <span class=\"s\">\"encoding\/json\"<\/span>\n   <span class=\"s\">\"fmt\"<\/span>\n   <span class=\"s\">\"io\/ioutil\"<\/span>\n   <span class=\"s\">\"net\/http\"<\/span>\n   <span class=\"s\">\"sort\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n   <span class=\"c\">\/\/ Make the request.<\/span>\n   <span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Get<\/span><span class=\"p\">(<\/span><span class=\"s\">\"https:\/\/dev.to\/api\/articles?per_page=20\"<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Read the bytes from the response.<\/span>\n   <span class=\"n\">body<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ioutil<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n   <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Transform into usable JSON list.<\/span>\n   <span class=\"n\">decoder<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">NewDecoder<\/span><span class=\"p\">(<\/span><span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">body<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">PublishedAt<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Time<\/span> <span class=\"s\">`json:\"published_at\"`<\/span>\n      <span class=\"n\">Title<\/span>       <span class=\"kt\">string<\/span>    <span class=\"s\">`json:\"title\"`<\/span>\n   <span class=\"p\">}<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decoder<\/span><span class=\"o\">.<\/span><span class=\"n\">Decode<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Sort the list, newest to oldest.<\/span>\n   <span class=\"n\">sort<\/span><span class=\"o\">.<\/span><span class=\"n\">Slice<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">j<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"kt\">bool<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span>\n   <span class=\"p\">})<\/span>\n\n   <span class=\"c\">\/\/ Print the list.<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"c\">\/** UPDATE: filter our results by only showing posts after 12PM GMT (noon). **\/<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Hour<\/span><span class=\"p\">()<\/span> <span class=\"o\">&lt;<\/span> <span class=\"m\">13<\/span> <span class=\"p\">{<\/span>\n         <span class=\"k\">continue<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%v -- %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Running this again, we should only see articles after 12PM:<\/p>\n\n<blockquote>\n<p>2020-10-27 17:58:59 +0000 UTC -- The 7 Most Popular DEV Posts from the Past Week<br>\n2020-10-27 17:40:25 +0000 UTC -- My Online Portfolio<br>\n2020-10-27 16:46:39 +0000 UTC -- Pairing with Community Member Rachael Wright-Munn<br>\n2020-10-27 14:37:07 +0000 UTC -- I\u2019m Levi Sharpe, Senior Podcast Producer at DEV\/Forem- AMA<br>\n2020-10-27 14:12:05 +0000 UTC -- A Sinatra Project<br>\n2020-10-27 13:21:46 +0000 UTC -- Projects to build that would get you hired as a beginner.<br>\n2020-10-27 13:18:16 +0000 UTC -- Create a Landing page in less than 100 lines (incl. CSS) \ufffd\ufffd<br>\n2020-10-27 13:17:06 +0000 UTC -- Forget pay cut, give me a raise to work remotely<br>\n2020-10-26 23:34:37 +0000 UTC -- JavaScript Challenge 6: Convert string to camel case<br>\n2020-10-26 21:28:26 +0000 UTC -- Final week of Hacktoberfest!<br>\n2020-10-26 20:26:02 +0000 UTC -- React Hooks: Managing State With useState Hook<\/p>\n<\/blockquote>\n\n<h2>\n  \n  \n  Writing A Rigid Test\n<\/h2>\n\n<p>Viola!<\/p>\n\n<p>So now what?...Well we want to make sure there aren\u2019t any unseen failure states. So, we should write a test for our new logic. That\u2019s going to require a bit of reorganization. Like so:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"bytes\"<\/span>\n   <span class=\"s\">\"encoding\/json\"<\/span>\n   <span class=\"s\">\"fmt\"<\/span>\n   <span class=\"s\">\"io\/ioutil\"<\/span>\n   <span class=\"s\">\"net\/http\"<\/span>\n   <span class=\"s\">\"sort\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">Article<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">PublishedAt<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Time<\/span> <span class=\"s\">`json:\"published_at\"`<\/span>\n   <span class=\"n\">Title<\/span>       <span class=\"kt\">string<\/span>    <span class=\"s\">`json:\"title\"`<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">GetArticles<\/span><span class=\"p\">()<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">fetch<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Filter our results by only showing posts after 12PM GMT (noon).<\/span>\n   <span class=\"n\">articles<\/span> <span class=\"o\">=<\/span> <span class=\"n\">FilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n\n   <span class=\"c\">\/\/ Sort the list, newest to oldest.<\/span>\n   <span class=\"n\">sort<\/span><span class=\"o\">.<\/span><span class=\"n\">Slice<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">j<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"kt\">bool<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span>\n   <span class=\"p\">})<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">FilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span><span class=\"p\">)<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">filtered<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Hour<\/span><span class=\"p\">()<\/span> <span class=\"o\">&lt;<\/span> <span class=\"m\">13<\/span> <span class=\"p\">{<\/span>\n         <span class=\"k\">continue<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">filtered<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">filtered<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">filtered<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">ListArticles<\/span><span class=\"p\">()<\/span> <span class=\"kt\">error<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">fetch<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Print the list.<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%v -- %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">fetch<\/span><span class=\"p\">()<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span>\n\n   <span class=\"c\">\/\/ Make the request.<\/span>\n   <span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Get<\/span><span class=\"p\">(<\/span><span class=\"s\">\"https:\/\/dev.to\/api\/articles?per_page=20\"<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Read the bytes from the response.<\/span>\n   <span class=\"n\">body<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ioutil<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n   <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Transform into usable JSON list.<\/span>\n   <span class=\"n\">decoder<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">NewDecoder<\/span><span class=\"p\">(<\/span><span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">body<\/span><span class=\"p\">))<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decoder<\/span><span class=\"o\">.<\/span><span class=\"n\">Decode<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ListArticles<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now that we have broken our logic out into functions, we can write our first test! Let\u2019s create a test for the <code>FilterArticles<\/code> function in a new <code>main_test.go<\/code> file:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"strings\"<\/span>\n   <span class=\"s\">\"testing\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">var<\/span> <span class=\"n\">testCases<\/span> <span class=\"o\">=<\/span> <span class=\"k\">map<\/span><span class=\"p\">[<\/span><span class=\"kt\">string<\/span><span class=\"p\">][]<\/span><span class=\"n\">Article<\/span><span class=\"p\">{<\/span>\n   <span class=\"s\">\"general\"<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603817199<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Pairing with Community Member Rachael Wright-Munn\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603809427<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"I\u2019m Levi Sharpe, Senior Podcast Producer at DEV\/Forem- AMA\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603807925<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"A Sinatra Project\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603804906<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Projects to build that would get you hired as a beginner.\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603804696<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Create a Landing page in less than 100 lines (incl. CSS) \ufffd\ufffd\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603755277<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"JavaScript Challenge 6: Convert string to camel case\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603751738<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Ruby CLI application: scraping, object relationships and single source of truth\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603747706<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Final week of Hacktoberfest!\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603743962<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"React Hooks: Managing State With useState Hook\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603642241<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"What are your favorite Kotlin resources?\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n      <span class=\"p\">{<\/span>\n         <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603470582<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n         <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"How to accelerate application performance with smart SQL queries.\"<\/span><span class=\"p\">,<\/span>\n      <span class=\"p\">},<\/span>\n   <span class=\"p\">},<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">TestFilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span> <span class=\"o\">*<\/span><span class=\"n\">testing<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">testCase<\/span><span class=\"p\">,<\/span> <span class=\"n\">want<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">testCases<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Run<\/span><span class=\"p\">(<\/span><span class=\"n\">testCase<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span> <span class=\"o\">*<\/span><span class=\"n\">testing<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n         <span class=\"n\">got<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">GetArticles<\/span><span class=\"p\">()<\/span>\n         <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Error<\/span><span class=\"p\">(<\/span><span class=\"s\">\"failed to get articles\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n\n         <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">wantArticle<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">want<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">gotArticle<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">got<\/span> <span class=\"p\">{<\/span>\n               <span class=\"k\">if<\/span> <span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">Trim<\/span><span class=\"p\">(<\/span><span class=\"n\">gotArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">,<\/span> <span class=\"s\">\" \"<\/span><span class=\"p\">)<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">Trim<\/span><span class=\"p\">(<\/span><span class=\"n\">wantArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">,<\/span> <span class=\"s\">\" \"<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                  <span class=\"k\">continue<\/span>\n               <span class=\"p\">}<\/span>\n               <span class=\"k\">if<\/span> <span class=\"n\">gotArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">()<\/span> <span class=\"o\">==<\/span> <span class=\"n\">wantArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n                  <span class=\"k\">break<\/span>\n               <span class=\"p\">}<\/span>\n               <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Errorf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"article not found: %v\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">wantArticle<\/span><span class=\"p\">)<\/span>\n            <span class=\"p\">}<\/span>\n         <span class=\"p\">}<\/span>\n      <span class=\"p\">})<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>So here, we have a test to see if all our previously seen (and filtered) articles are accounted for and that their timestamps and titles are the same. Yay!<\/p>\n\n<h2>\n  \n  \n  Improving Test Stability\n<\/h2>\n\n<p>However, if we run this test enough times, we\u2019ll find that at some point it <em>will<\/em> fail. After a few hours or perhaps a day, we\u2019ll likely have a completely different article set. That\u2019s because we are hitting the live DEV API!<\/p>\n\n<p>We want our tests to be reliable indicators of our code\u2019s health. Whether or not the DEV API is sending correctly formatted data, or whether every article is a duplicate, is of no consequence to us \u2014 <em>at least not as far as our unit tests are concerned<\/em>.<\/p>\n\n<p>Thankfully, we can use a Mock to fix this! We have yet to introduce an Interface to encapsulate all the functionality we\u2019ve provided to the main function. Let\u2019s add an Interface that is going to describe a struct that will handle all this logic when it\u2019s created:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"bytes\"<\/span>\n   <span class=\"s\">\"encoding\/json\"<\/span>\n   <span class=\"s\">\"fmt\"<\/span>\n   <span class=\"s\">\"io\/ioutil\"<\/span>\n   <span class=\"s\">\"net\/http\"<\/span>\n   <span class=\"s\">\"sort\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">Fetcher<\/span> <span class=\"k\">interface<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">Fetch<\/span><span class=\"p\">()<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">ArticleFetcher<\/span> <span class=\"k\">struct<\/span><span class=\"p\">{}<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">Article<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">PublishedAt<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Time<\/span> <span class=\"s\">`json:\"published_at\"`<\/span>\n   <span class=\"n\">Title<\/span>       <span class=\"kt\">string<\/span>    <span class=\"s\">`json:\"title\"`<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"c\">\/\/ Compile-time check to ensure that our struct implements the interface.<\/span>\n<span class=\"k\">var<\/span> <span class=\"n\">_<\/span> <span class=\"n\">Fetcher<\/span> <span class=\"o\">=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">ArticleFetcher<\/span><span class=\"p\">{}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">a<\/span> <span class=\"o\">*<\/span><span class=\"n\">ArticleFetcher<\/span><span class=\"p\">)<\/span> <span class=\"n\">Fetch<\/span><span class=\"p\">()<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span>\n\n   <span class=\"c\">\/\/ Make the request.<\/span>\n   <span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">http<\/span><span class=\"o\">.<\/span><span class=\"n\">Get<\/span><span class=\"p\">(<\/span><span class=\"s\">\"https:\/\/dev.to\/api\/articles?per_page=20\"<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Read the bytes from the response.<\/span>\n   <span class=\"n\">body<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ioutil<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadAll<\/span><span class=\"p\">(<\/span><span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"p\">)<\/span>\n   <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">Body<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Transform into usable JSON list.<\/span>\n   <span class=\"n\">decoder<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">NewDecoder<\/span><span class=\"p\">(<\/span><span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">body<\/span><span class=\"p\">))<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">decoder<\/span><span class=\"o\">.<\/span><span class=\"n\">Decode<\/span><span class=\"p\">(<\/span><span class=\"o\">&amp;<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">GetArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span> <span class=\"n\">Fetcher<\/span><span class=\"p\">)<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">f<\/span><span class=\"o\">.<\/span><span class=\"n\">Fetch<\/span><span class=\"p\">()<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Filter our results by only showing posts after 12PM GMT (noon).<\/span>\n   <span class=\"n\">articles<\/span> <span class=\"o\">=<\/span> <span class=\"n\">FilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">)<\/span>\n\n   <span class=\"c\">\/\/ Sort the list, newest to oldest.<\/span>\n   <span class=\"n\">sort<\/span><span class=\"o\">.<\/span><span class=\"n\">Slice<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">i<\/span><span class=\"p\">,<\/span> <span class=\"n\">j<\/span> <span class=\"kt\">int<\/span><span class=\"p\">)<\/span> <span class=\"kt\">bool<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">i<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span> <span class=\"o\">&gt;<\/span> <span class=\"n\">articles<\/span><span class=\"p\">[<\/span><span class=\"n\">j<\/span><span class=\"p\">]<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">UnixNano<\/span><span class=\"p\">()<\/span>\n   <span class=\"p\">})<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">FilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">articles<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span><span class=\"p\">)<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">var<\/span> <span class=\"n\">filtered<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">if<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Hour<\/span><span class=\"p\">()<\/span> <span class=\"o\">&lt;<\/span> <span class=\"m\">13<\/span> <span class=\"p\">{<\/span>\n         <span class=\"k\">continue<\/span>\n      <span class=\"p\">}<\/span>\n      <span class=\"n\">filtered<\/span> <span class=\"o\">=<\/span> <span class=\"nb\">append<\/span><span class=\"p\">(<\/span><span class=\"n\">filtered<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">filtered<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">ListArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span> <span class=\"n\">Fetcher<\/span><span class=\"p\">)<\/span> <span class=\"kt\">error<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">articles<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">GetArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">)<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"k\">return<\/span> <span class=\"n\">err<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"c\">\/\/ Print the list.<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">articles<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"%v -- %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"p\">,<\/span> <span class=\"n\">article<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n\n   <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ListArticles<\/span><span class=\"p\">(<\/span><span class=\"nb\">new<\/span><span class=\"p\">(<\/span><span class=\"n\">ArticleFetcher<\/span><span class=\"p\">))<\/span>\n   <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n      <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Here, we\u2019ve thrown the <code>fetch()<\/code> logic into a Structure method and assigned that Structure (<code>ArticleFetcher<\/code>) to an Interface (<code>Fetcher<\/code>) that we can use to create our Mock.<br>\nLikewise, the test will have to change:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n   <span class=\"s\">\"strings\"<\/span>\n   <span class=\"s\">\"testing\"<\/span>\n   <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">MockArticleFetcher<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n   <span class=\"n\">want<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">var<\/span> <span class=\"p\">(<\/span>\n   <span class=\"n\">_<\/span>         <span class=\"n\">Fetcher<\/span> <span class=\"o\">=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">MockArticleFetcher<\/span><span class=\"p\">{}<\/span>\n   <span class=\"n\">testCases<\/span>         <span class=\"o\">=<\/span> <span class=\"k\">map<\/span><span class=\"p\">[<\/span><span class=\"kt\">string<\/span><span class=\"p\">][]<\/span><span class=\"n\">Article<\/span><span class=\"p\">{<\/span>\n      <span class=\"s\">\"general\"<\/span><span class=\"o\">:<\/span> <span class=\"p\">{<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603817199<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Pairing with Community Member Rachael Wright-Munn\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603809427<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"I\u2019m Levi Sharpe, Senior Podcast Producer at DEV\/Forem- AMA\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603807925<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"A Sinatra Project\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603804906<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Projects to build that would get you hired as a beginner.\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603804696<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Create a Landing page in less than 100 lines (incl. CSS) \ufffd\ufffd\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603755277<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"JavaScript Challenge 6: Convert string to camel case\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603751738<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Ruby CLI application: scraping, object relationships and single source of truth\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603747706<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"Final week of Hacktoberfest!\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603743962<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"React Hooks: Managing State With useState Hook\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603642241<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"What are your favorite Kotlin resources?\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n         <span class=\"p\">{<\/span>\n            <span class=\"n\">PublishedAt<\/span><span class=\"o\">:<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">(<\/span><span class=\"m\">1603470582<\/span><span class=\"p\">,<\/span> <span class=\"m\">0<\/span><span class=\"p\">),<\/span>\n            <span class=\"n\">Title<\/span><span class=\"o\">:<\/span>       <span class=\"s\">\"How to accelerate application performance with smart SQL queries.\"<\/span><span class=\"p\">,<\/span>\n         <span class=\"p\">},<\/span>\n      <span class=\"p\">},<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">NewMockArticleFetcher<\/span><span class=\"p\">(<\/span><span class=\"n\">want<\/span> <span class=\"p\">[]<\/span><span class=\"n\">Article<\/span><span class=\"p\">)<\/span> <span class=\"o\">*<\/span><span class=\"n\">MockArticleFetcher<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">MockArticleFetcher<\/span><span class=\"p\">{<\/span><span class=\"n\">want<\/span><span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">m<\/span> <span class=\"o\">*<\/span><span class=\"n\">MockArticleFetcher<\/span><span class=\"p\">)<\/span> <span class=\"n\">Fetch<\/span><span class=\"p\">()<\/span> <span class=\"p\">([]<\/span><span class=\"n\">Article<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">return<\/span> <span class=\"n\">m<\/span><span class=\"o\">.<\/span><span class=\"n\">want<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">TestFilterArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span> <span class=\"o\">*<\/span><span class=\"n\">testing<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n   <span class=\"k\">for<\/span> <span class=\"n\">testCase<\/span><span class=\"p\">,<\/span> <span class=\"n\">want<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">testCases<\/span> <span class=\"p\">{<\/span>\n      <span class=\"n\">mf<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">NewMockArticleFetcher<\/span><span class=\"p\">(<\/span><span class=\"n\">want<\/span><span class=\"p\">)<\/span>\n      <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Run<\/span><span class=\"p\">(<\/span><span class=\"n\">testCase<\/span><span class=\"p\">,<\/span> <span class=\"k\">func<\/span><span class=\"p\">(<\/span><span class=\"n\">t<\/span> <span class=\"o\">*<\/span><span class=\"n\">testing<\/span><span class=\"o\">.<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n         <span class=\"n\">got<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">GetArticles<\/span><span class=\"p\">(<\/span><span class=\"n\">mf<\/span><span class=\"p\">)<\/span>\n         <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">!=<\/span> <span class=\"no\">nil<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Error<\/span><span class=\"p\">(<\/span><span class=\"s\">\"failed to get articles\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n         <span class=\"k\">if<\/span> <span class=\"nb\">len<\/span><span class=\"p\">(<\/span><span class=\"n\">got<\/span><span class=\"p\">)<\/span> <span class=\"o\">==<\/span> <span class=\"m\">0<\/span> <span class=\"p\">{<\/span>\n            <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Error<\/span><span class=\"p\">(<\/span><span class=\"s\">\"got empty article list\"<\/span><span class=\"p\">)<\/span>\n         <span class=\"p\">}<\/span>\n\n         <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">wantArticle<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">want<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">for<\/span> <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">gotArticle<\/span> <span class=\"o\">:=<\/span> <span class=\"k\">range<\/span> <span class=\"n\">got<\/span> <span class=\"p\">{<\/span>\n               <span class=\"k\">if<\/span> <span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">Trim<\/span><span class=\"p\">(<\/span><span class=\"n\">gotArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">,<\/span> <span class=\"s\">\" \"<\/span><span class=\"p\">)<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">Trim<\/span><span class=\"p\">(<\/span><span class=\"n\">wantArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">Title<\/span><span class=\"p\">,<\/span> <span class=\"s\">\" \"<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n                  <span class=\"k\">continue<\/span>\n               <span class=\"p\">}<\/span>\n               <span class=\"k\">if<\/span> <span class=\"n\">gotArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">()<\/span> <span class=\"o\">==<\/span> <span class=\"n\">wantArticle<\/span><span class=\"o\">.<\/span><span class=\"n\">PublishedAt<\/span><span class=\"o\">.<\/span><span class=\"n\">Unix<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n                  <span class=\"k\">break<\/span>\n               <span class=\"p\">}<\/span>\n               <span class=\"n\">t<\/span><span class=\"o\">.<\/span><span class=\"n\">Errorf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"article not found: %v\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">wantArticle<\/span><span class=\"p\">)<\/span>\n            <span class=\"p\">}<\/span>\n         <span class=\"p\">}<\/span>\n      <span class=\"p\">})<\/span>\n   <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now we can ensure that no matter how our third-party connector is functioning, we can always trust that in this version of our program, our test will behave consistently.<\/p>\n\n<h2>\n  \n  \n  Recap\n<\/h2>\n\n<p>We took our simplistic design and expanded the functionality of our \u201cthird-party\u201d feature (requesting data from the live DEV API) to, make testing our current filtering logic reliable, and make testing future features easier as well.<\/p>\n\n<p>That\u2019s all, thanks for reading!<\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Header photo credit: <a href=\"https:\/\/unsplash.com\/@philhearing?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Phil Hearing<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/fake?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a><\/p>\n\n","category":["showdev","go","testing","tutorial"]},{"title":"On Maintaining a Third-party VS Code Extension Marketplace","pubDate":"Mon, 01 Jul 2019 15:30:17 +0000","link":"https:\/\/dev.to\/foresthoffman\/on-maintaining-a-third-party-vs-code-extension-marketplace-2m9l","guid":"https:\/\/dev.to\/foresthoffman\/on-maintaining-a-third-party-vs-code-extension-marketplace-2m9l","description":"<p>Amongst the various things that I worked on at <a href=\"https:\/\/coder.com\" rel=\"noopener noreferrer\">Coder.com<\/a>, one of the most interesting projects was that of the internal extension marketplace. This marketplace interfaced with <a href=\"https:\/\/github.com\/foresthoffman\/code-server\" rel=\"noopener noreferrer\">code-server<\/a> clients, allowing users to install extensions on their server-powered VS Code instances. Since the official <a href=\"https:\/\/marketplace.visualstudio.com\/vscode\" rel=\"noopener noreferrer\">VS Code Marketplace<\/a> was not technically or <a href=\"https:\/\/code.visualstudio.com\/license\" rel=\"noopener noreferrer\">legally<\/a> usable for third-parties (it still isn't), we had to roll our own!<\/p>\n\n<p>Our internal marketplace leveraged the Google Cloud Storage platform to support over 4.5 thousand extensions for code-server users.<\/p>\n\n<p>That's fantastic, however this project was particularly challenging for three reasons:<\/p>\n\n<ol>\n<li>The official Marketplace could not be used directly<\/li>\n<li>The official Marketplace did not appear curated in any major way<\/li>\n<li>Some extensions were not as...easily supported...as others<\/li>\n<\/ol>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fapx6f29x7ibau69ccxs4.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fapx6f29x7ibau69ccxs4.gif\" alt=\"Coding with flames gif\" width=\"480\" height=\"270\"><\/a><\/p>\n\n<h6>\n  \n  \n  Side-note as I'm writing this: let's see how many times I can say \"official\" or \"officially\".\n<\/h6>\n\n<h3>\n  \n  \n  1. \ud83d\udea7 Officially Off-Limits \ud83d\udea7\n<\/h3>\n\n<p>Due to the official Marketplace being off limits, the internal system had to supply valid extensions including all their metadata. Such as:<\/p>\n\n<ul>\n<li>Manifest\n\n<ul>\n<li>Author, Repository, Version, etc.<\/li>\n<\/ul>\n\n\n<\/li>\n\n<li>Icons<\/li>\n\n<li>Description<\/li>\n\n<li>License<\/li>\n\n<li>Extension Archive<\/li>\n\n<li>Miscellaneous Files\n\n<ul>\n<li>Changelog<\/li>\n<\/ul>\n\n\n<\/li>\n\n<\/ul>\n\n<p>Obviously this is a lot of data to process manually, so we had a secondary service to collect all of it. That's where the extension scraper came in. This service was in charge of pulling open source extension repositories and packaging them for distribution.<\/p>\n\n<h3>\n  \n  \n  2. \ud83d\udd75\ufe0f Officially Mysterious \ud83d\udd75\ufe0f\n<\/h3>\n\n<p>Reliability is the name of the game, so the scraper's job was not only to pull the sources down, package them, and dump them out into the storage bucket. It needed to ensure that the extensions were: valid according to the official standards; and functioned.<\/p>\n\n<p>The official standards can technically be checked by Microsoft's <a href=\"https:\/\/www.npmjs.com\/package\/vsce\" rel=\"noopener noreferrer\">vsce<\/a> CLI tool. This tool is actually what outputs the <code>*.VSIX<\/code> Zip archives. If it encounters any abnormalities, it won't package the extension.<\/p>\n\n<p>Having run this command manually and programmatically on several thousand extensions, I can safely say that the most common error I've seen among the extensions that don't make the cut is, when the extension includes the <a href=\"https:\/\/www.npmjs.com\/package\/vscode\" rel=\"noopener noreferrer\">vscode<\/a> module as a dependency. The vscode module is provided by the VS Code runtime environment, so it's only needed as a dev-dependency. This occurs with extensions that are actually on the official Marketplace at this moment.<\/p>\n\n<p>So, taking other errors into consideration, especially sketchy repository URLs, or incomplete metadata, my only conclusion is that the Marketplace doesn't have any curation. However, this is only an issue for less popular extensions. This isn't something that becomes apparent with any of the officially supported and maintained extensions from Microsoft. Think <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-python.python\" rel=\"noopener noreferrer\">Python<\/a> or <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=ms-vscode.PowerShell\" rel=\"noopener noreferrer\">PowerShell<\/a>. Or, even extensions with strong communities backing them, like <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=vscodevim.vim\" rel=\"noopener noreferrer\">Vim<\/a>. Those were all fantastic to work with, and any issues were easily reported and dealt with.<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ft890eqt6vt8ep65bsml1.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ft890eqt6vt8ep65bsml1.gif\" alt=\"Patrick daww gif\" width=\"498\" height=\"280\"><\/a><\/p>\n\n<p>It makes sense. All of the popular extensions used by roughly a million people <em>need<\/em> to be stable and conform to the standards. It's the extensions that are used by smaller, but still significant, groups that are worrisome.<\/p>\n\n<p>In addition to packaging, the scraper needed to ensure that the extensions would function for users. To check this, the scraper loaded each extension source into a Docker container and attempted to load all the extension's dependencies. This needed to include local and global dependencies. Again, for the vast majority this worked great. However, some of the outliers were concerning.<\/p>\n\n<p>There was an extension that was still referencing and attempting to download an old NPM module that experienced severe security issues in the past. I got a hold of the extension maintainer, and it was dealt with. Still, that made me wonder what other potential security risks there are with the overlooked extensions on the Marketplace.<\/p>\n\n<p>It would be nice if there was some tool or automated submission pipeline for extensions. Ideally such a pipeline would ensure that the extension being submitted conformed to the same standards that we would expect of the incredibly popular extensions I mentioned previously.<\/p>\n\n<h3>\n  \n  \n  3. \ud83d\udd25 Caution: Flammable \ud83d\udd25\n<\/h3>\n\n<p>One of the unfortunate drawbacks of creating a third-party fill for a third-party fork of a widely used open source project is that you have to roll with the punches. You're at the whim of the official maintainers, and you're at the whim of all the extension maintainers that you want to support on your platform.<\/p>\n\n<p>So, in some cases like <a href=\"https:\/\/marketplace.visualstudio.com\/items?itemName=Shan.code-settings-sync\" rel=\"noopener noreferrer\">Settings Sync<\/a> (sidenote: I personally love it \ud83d\ude0d) the extension maintainers were blocked by a lack of custom functionality in VS Code itself. Which means then that our users either had to cross their fingers and hope that the functionality was delivered, or take some convoluted steps to support it! Neither were ideal.<\/p>\n\n<p>There were some repositories, such as the <a href=\"https:\/\/github.com\/Microsoft\/vscode-loc\" rel=\"noopener noreferrer\">localization extension pack<\/a>, that contain extensions that have extraordinarily aggressive versioning conventions. Every time <a href=\"https:\/\/code.visualstudio.com\/insiders\/\" rel=\"noopener noreferrer\">VS Code Insiders<\/a> updated, the extensions in question would update their version number to a version of VS Code we obviously couldn't support!<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fakgof7wyi22ej6x1wdlt.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fakgof7wyi22ej6x1wdlt.gif\" alt=\"\" width=\"374\" height=\"379\"><\/a><\/p>\n\n<p>That lead me to create version constraints for the extension scraper. Such that aggressively versioned extensions, or extensions that needed to be rolled back due to incompatibilities (e.g. <a href=\"https:\/\/github.com\/VSCodeVim\/Vim\/issues\/3522\" rel=\"noopener noreferrer\">https:\/\/github.com\/VSCodeVim\/Vim\/issues\/3522<\/a>), could continue functioning as expected for our users.<\/p>\n\n\n\n\n<p>Overall, it was a lot of fun to work on! Thanks for reading.<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdp6jkzk2f6e03xulbi9g.gif\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fdp6jkzk2f6e03xulbi9g.gif\" alt=\"Thumbs up\" width=\"300\" height=\"320\"><\/a><\/p>\n\n<h6>\n  \n  \n  Looks like a total of 9 instances of \"official\" and 4 instances of \"officially\"! \ud83e\udd23\n<\/h6>\n\n\n\n\n<p>Header photo Credit: Photo by <a href=\"https:\/\/unsplash.com\/@bernardhermant?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Bernard Hermant<\/a> on <a href=\"https:\/\/unsplash.com\/search\/photos\/market?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a><\/p>\n\n","category":["showdev","typescript","vscode","docker"]},{"title":"Building a Twitch.tv Chat Bot with Go - Part 2","pubDate":"Wed, 27 Dec 2017 03:35:08 +0000","link":"https:\/\/dev.to\/foresthoffman\/building-a-twitchtv-chat-bot-with-go---part-2-52jo","guid":"https:\/\/dev.to\/foresthoffman\/building-a-twitchtv-chat-bot-with-go---part-2-52jo","description":"<p>EDIT (5\/13\/2019): Since the time of writing the package has been renamed from \"twitchbot\" to \"bot\". However, the tagged releases used for this tutorial still use the old name.<\/p>\n\n<p>All of the disclaimers and warnings at the beginning of Part 1 apply here.<\/p>\n\n<h2>\n  \n  \n  Step 1\n<\/h2>\n\n<p>Go ahead and jump to Step 1 via the following:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> step-1\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>In the previous step we laid out the data types that the <code>BasicBot<\/code> struct would need to function properly. Now, we'll lay out the behavior for the bot. To do that we'll start by filling out the <code>TwitchBot<\/code> interface.<\/p>\n\n<p>From the notes just below the import block, we need to make functions that do the following:<\/p>\n\n<ul>\n<li>Connect to the Twitch IRC server<\/li>\n<li>Disconnect from the Twitch IRC server<\/li>\n<li>Parse and react to messages from the chat<\/li>\n<li>Join a specific channel once connected<\/li>\n<li>Read from the super-top-secret credentials file to get the bot's password<\/li>\n<li>Send messages to the current chat channel<\/li>\n<\/ul>\n\n<p>So, in Go terms:<\/p>\n\n<ul>\n<li>Connect()<\/li>\n<li>Disconnect()<\/li>\n<li>HandleChat() error<\/li>\n<li>JoinChannel()<\/li>\n<li>ReadCredentials() (*OAuthCred, error)<\/li>\n<li>Say(msg string) error<\/li>\n<\/ul>\n\n<p>The <code>ReadCredentials()<\/code> function looks funny, because the <code>OAuthCred<\/code> pointer being returned is a mistake. Don't worry, it gets fixed in the following steps. I thought about omitting it for the sake of the walkthrough, but it's bound to make anyone following along think, \"Huh?\", while turning their head sideways.<\/p>\n\n<p>We'll also need a function to kick everything off, so the <code>Start()<\/code> function will be added. Pop open <code>twitchbot.go<\/code> and make the following changes to the <code>TwitchBot<\/code> interface:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">TwitchBot<\/span> <span class=\"k\">interface<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Connect<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">Disconnect<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">HandleChat<\/span><span class=\"p\">()<\/span> <span class=\"kt\">error<\/span>\n    <span class=\"n\">JoinChannel<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">ReadCredentials<\/span><span class=\"p\">()<\/span> <span class=\"p\">(<\/span><span class=\"o\">*<\/span><span class=\"n\">OAuthCred<\/span><span class=\"p\">,<\/span> <span class=\"kt\">error<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">Say<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span> <span class=\"kt\">string<\/span><span class=\"p\">)<\/span> <span class=\"kt\">error<\/span>\n    <span class=\"n\">Start<\/span><span class=\"p\">()<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Since <code>OAuthCred<\/code> is undefined, we'll add that too. Let's add it above <code>TwitchBot<\/code>.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">OAuthCred<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Password<\/span> <span class=\"kt\">string<\/span> <span class=\"s\">`json:\"password,omitempty\"`<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">TwitchBot<\/span> <span class=\"k\">interface<\/span> <span class=\"p\">{<\/span>\n<span class=\"o\">...<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now that <code>OAuthCred<\/code> has been defined, let's add a pointer to an <code>OAuthCred<\/code> instance to the BasicBot struct, for later. The <code>BasicBot<\/code> definition should now look like:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">BasicBot<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Channel<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">Credentials<\/span> <span class=\"o\">*<\/span><span class=\"n\">OAuthCred<\/span> <span class=\"c\">\/\/ add the pointer field<\/span>\n    <span class=\"n\">MsgRate<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Duration<\/span>\n    <span class=\"n\">Name<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">Port<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">PrivatePath<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">Server<\/span> <span class=\"kt\">string<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Go ahead and build now, if there are no explosions we can move on.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go <span class=\"nb\">fmt<\/span> .\/... <span class=\"o\">&amp;&amp;<\/span> go build\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Step 2\n<\/h2>\n\n<p>Let's run the following to jump to Step 2:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> step-2\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now, we have to make the <code>BasicBot<\/code> implement the functions that we added to the <code>TwitchBot<\/code> interface. Let's start go in order.<\/p>\n\n<h4>\n  \n  \n  Connect()\n<\/h4>\n\n<p><code>BasicBot.Connect()<\/code> is going to require the <a href=\"https:\/\/golang.org\/pkg\/net\/\" rel=\"noopener noreferrer\"><code>net<\/code><\/a> and <a href=\"https:\/\/golang.org\/pkg\/fmt\/\" rel=\"noopener noreferrer\"><code>fmt<\/code><\/a> packages, so we'll add those to the import block:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"fmt\"<\/span>\n    <span class=\"s\">\"net\"<\/span>\n    <span class=\"o\">...<\/span>\n<span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Since we want to see what the bot is doing when it connects, we'll need to write to standard output. Unfortunately that can get a bit unwieldy, so lets add in some helper functions to timestamp the log messages. We'll also add a constant time-format string for <code>timeStamp()<\/code> to use, just after the import block.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n<span class=\"o\">...<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">const<\/span> <span class=\"n\">PSTFormat<\/span> <span class=\"o\">=<\/span> <span class=\"s\">\"Jan 2 15:04:05 PST\"<\/span>\n\n<span class=\"o\">...<\/span>\n\n<span class=\"c\">\/\/ for BasicBot<\/span>\n<span class=\"k\">func<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">()<\/span> <span class=\"kt\">string<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">TimeStamp<\/span><span class=\"p\">(<\/span><span class=\"n\">PSTFormat<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"c\">\/\/ the generic variation, for bots using the TwitchBot interface<\/span>\n<span class=\"k\">func<\/span> <span class=\"n\">TimeStamp<\/span><span class=\"p\">(<\/span><span class=\"n\">format<\/span> <span class=\"kt\">string<\/span><span class=\"p\">)<\/span> <span class=\"kt\">string<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">return<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Now<\/span><span class=\"p\">()<\/span><span class=\"o\">.<\/span><span class=\"n\">Format<\/span><span class=\"p\">(<\/span><span class=\"n\">format<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>And, then the function itself:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Connects the bot to the Twitch IRC server. The bot will continue to try to connect until it<\/span>\n<span class=\"c\">\/\/ succeeds or is manually shutdown.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">Connect<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">var<\/span> <span class=\"n\">err<\/span> <span class=\"kt\">error<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Connecting to %s...<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Server<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"c\">\/\/ makes connection to Twitch IRC server<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">net<\/span><span class=\"o\">.<\/span><span class=\"n\">Dial<\/span><span class=\"p\">(<\/span><span class=\"s\">\"tcp\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Server<\/span><span class=\"o\">+<\/span><span class=\"s\">\":\"<\/span><span class=\"o\">+<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Port<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Cannot connect to %s, retrying.<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Server<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Connect<\/span><span class=\"p\">()<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Connected to %s!<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Server<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">startTime<\/span> <span class=\"o\">=<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Now<\/span><span class=\"p\">()<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Great, now we've got the <code>BasicBot.Connect()<\/code> function, and it's set to create a connection to the Twitch IRC server with all the necessary information. However, we don't have a way of storing the connection, like <code>bb.conn<\/code> assumes within the <code>BasicBot.Connect()<\/code> function. We're also missing a <code>bb.startTime<\/code> field to hold the time at which the bot successfully connected to the server. Let's add them:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">BasicBot<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Channel<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">conn<\/span> <span class=\"n\">net<\/span><span class=\"o\">.<\/span><span class=\"n\">Conn<\/span> <span class=\"c\">\/\/ add this field<\/span>\n    <span class=\"n\">Credentials<\/span> <span class=\"o\">*<\/span><span class=\"n\">OAuthCred<\/span>\n    <span class=\"n\">MsgRate<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Duration<\/span>\n    <span class=\"n\">Name<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">Port<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">PrivatePath<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">Server<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">startTime<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Time<\/span> <span class=\"c\">\/\/ add this field<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h4>\n  \n  \n  Disconnect()\n<\/h4>\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Officially disconnects the bot from the Twitch IRC server.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">Disconnect<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Close<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">upTime<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Now<\/span><span class=\"p\">()<\/span><span class=\"o\">.<\/span><span class=\"n\">Sub<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">startTime<\/span><span class=\"p\">)<\/span><span class=\"o\">.<\/span><span class=\"n\">Seconds<\/span><span class=\"p\">()<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Closed connection from %s! | Live for: %fs<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Server<\/span><span class=\"p\">,<\/span> <span class=\"n\">upTime<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This function does what it says on the tin, and outputs some info on the duration of the bot's connection. Neat!<\/p>\n\n<h4>\n  \n  \n  HandleChat()\n<\/h4>\n\n<p><code>BasicBot.HandleChat()<\/code> is going to be doing the heavy lifting, so there are four packages that are going to be added to the import block:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"bufio\"<\/span>         <span class=\"c\">\/\/ 1<\/span>\n    <span class=\"s\">\"errors\"<\/span>        <span class=\"c\">\/\/ 2<\/span>\n    <span class=\"s\">\"fmt\"<\/span>\n    <span class=\"s\">\"net\"<\/span>\n    <span class=\"s\">\"net\/textproto\"<\/span> <span class=\"c\">\/\/ 3<\/span>\n    <span class=\"s\">\"regexp\"<\/span>        <span class=\"c\">\/\/ 4<\/span>\n    <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>In order for the bot to understand and react to chat messages, we're going to need to parse the raw messages and use context. That's why we've imported the <code>regexp<\/code> package. Let's compile some regular expressions for use within the function later. You can add the following under the import block:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Regex for parsing PRIVMSG strings.<\/span>\n<span class=\"c\">\/\/<\/span>\n<span class=\"c\">\/\/ First matched group is the user's name and the second matched group is the content of the<\/span>\n<span class=\"c\">\/\/ user's message.<\/span>\n<span class=\"k\">var<\/span> <span class=\"n\">msgRegex<\/span> <span class=\"o\">*<\/span><span class=\"n\">regexp<\/span><span class=\"o\">.<\/span><span class=\"n\">Regexp<\/span> <span class=\"o\">=<\/span> <span class=\"n\">regexp<\/span><span class=\"o\">.<\/span><span class=\"n\">MustCompile<\/span><span class=\"p\">(<\/span><span class=\"s\">`^:(\\w+)!\\w+@\\w+\\.tmi\\.twitch\\.tv (PRIVMSG) #\\w+(?: :(.*))?$`<\/span><span class=\"p\">)<\/span>\n\n<span class=\"c\">\/\/ Regex for parsing user commands, from already parsed PRIVMSG strings.<\/span>\n<span class=\"c\">\/\/<\/span>\n<span class=\"c\">\/\/ First matched group is the command name and the second matched group is the argument for the<\/span>\n<span class=\"c\">\/\/ command.<\/span>\n<span class=\"k\">var<\/span> <span class=\"n\">cmdRegex<\/span> <span class=\"o\">*<\/span><span class=\"n\">regexp<\/span><span class=\"o\">.<\/span><span class=\"n\">Regexp<\/span> <span class=\"o\">=<\/span> <span class=\"n\">regexp<\/span><span class=\"o\">.<\/span><span class=\"n\">MustCompile<\/span><span class=\"p\">(<\/span><span class=\"s\">`^!(\\w+)\\s?(\\w+)?`<\/span><span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>You can (and should!) read the <a href=\"https:\/\/dev.twitch.tv\/docs\/irc\" rel=\"noopener noreferrer\">Twitch Docs<\/a> on the various types of IRC messages that get sent, but the gist of it is: <\/p>\n\n<ul>\n<li>there are PING messages that must trigger a PONG response from the bot, otherwise it will be disconnected<\/li>\n<li>there are PRIVMSG messages that are sent as a result of someone (including the bot itself) talking in the current chat channel (despite being called \"PRIVMSG\" messages, they are public to the current chat channel)<\/li>\n<\/ul>\n\n<p>The <code>msgRegex<\/code> variable is going to be used to parse the initial PRIVMSGs and the <code>cmdRegex<\/code> variable will be used for further parsing to catch bot commands in chat.<\/p>\n\n<p>Then we'll have the function itself:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Listens for and logs messages from chat. Responds to commands from the channel owner. The bot<\/span>\n<span class=\"c\">\/\/ continues until it gets disconnected, told to shutdown, or forcefully shutdown.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">HandleChat<\/span><span class=\"p\">()<\/span> <span class=\"kt\">error<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Watching #%s...<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span><span class=\"p\">)<\/span>\n\n    <span class=\"c\">\/\/ reads from connection<\/span>\n    <span class=\"n\">tp<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">textproto<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">bufio<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"p\">))<\/span>\n\n    <span class=\"c\">\/\/ listens for chat messages<\/span>\n    <span class=\"k\">for<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">line<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">tp<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadLine<\/span><span class=\"p\">()<\/span>\n        <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n\n            <span class=\"c\">\/\/ officially disconnects the bot from the server<\/span>\n            <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Disconnect<\/span><span class=\"p\">()<\/span>\n\n            <span class=\"k\">return<\/span> <span class=\"n\">errors<\/span><span class=\"o\">.<\/span><span class=\"n\">New<\/span><span class=\"p\">(<\/span><span class=\"s\">\"bb.Bot.HandleChat: Failed to read line from channel. Disconnected.\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span>\n\n        <span class=\"c\">\/\/ logs the response from the IRC server<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">line<\/span><span class=\"p\">)<\/span>\n\n        <span class=\"k\">if<\/span> <span class=\"s\">\"PING :tmi.twitch.tv\"<\/span> <span class=\"o\">==<\/span> <span class=\"n\">line<\/span> <span class=\"p\">{<\/span>\n\n            <span class=\"c\">\/\/ respond to PING message with a PONG message, to maintain the connection<\/span>\n            <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"PONG :tmi.twitch.tv<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">))<\/span>\n            <span class=\"k\">continue<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n\n            <span class=\"c\">\/\/ handle a PRIVMSG message<\/span>\n            <span class=\"n\">matches<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">msgRegex<\/span><span class=\"o\">.<\/span><span class=\"n\">FindStringSubmatch<\/span><span class=\"p\">(<\/span><span class=\"n\">line<\/span><span class=\"p\">)<\/span>\n            <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">matches<\/span> <span class=\"p\">{<\/span>\n                <span class=\"n\">userName<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"m\">1<\/span><span class=\"p\">]<\/span>\n                <span class=\"n\">msgType<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"m\">2<\/span><span class=\"p\">]<\/span>\n\n                <span class=\"k\">switch<\/span> <span class=\"n\">msgType<\/span> <span class=\"p\">{<\/span>\n                <span class=\"k\">case<\/span> <span class=\"s\">\"PRIVMSG\"<\/span><span class=\"o\">:<\/span>\n                    <span class=\"n\">msg<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">matches<\/span><span class=\"p\">[<\/span><span class=\"m\">3<\/span><span class=\"p\">]<\/span>\n                    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] %s: %s<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">userName<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span><span class=\"p\">)<\/span>\n\n                    <span class=\"c\">\/\/ parse commands from user message<\/span>\n                    <span class=\"n\">cmdMatches<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">cmdRegex<\/span><span class=\"o\">.<\/span><span class=\"n\">FindStringSubmatch<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span><span class=\"p\">)<\/span>\n                    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">cmdMatches<\/span> <span class=\"p\">{<\/span>\n                        <span class=\"n\">cmd<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">cmdMatches<\/span><span class=\"p\">[<\/span><span class=\"m\">1<\/span><span class=\"p\">]<\/span>\n                        <span class=\"c\">\/\/arg := cmdMatches[2]<\/span>\n\n                        <span class=\"c\">\/\/ channel-owner specific commands<\/span>\n                        <span class=\"k\">if<\/span> <span class=\"n\">userName<\/span> <span class=\"o\">==<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span> <span class=\"p\">{<\/span>\n                            <span class=\"k\">switch<\/span> <span class=\"n\">cmd<\/span> <span class=\"p\">{<\/span>\n                            <span class=\"k\">case<\/span> <span class=\"s\">\"tbdown\"<\/span><span class=\"o\">:<\/span>\n                                <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span>\n                                    <span class=\"s\">\"[%s] Shutdown command received. Shutting down now...<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span>\n                                    <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span>\n                                <span class=\"p\">)<\/span>\n\n                                <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Disconnect<\/span><span class=\"p\">()<\/span>\n                                <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n                            <span class=\"k\">default<\/span><span class=\"o\">:<\/span>\n                                <span class=\"c\">\/\/ do nothing<\/span>\n                            <span class=\"p\">}<\/span>\n                        <span class=\"p\">}<\/span>\n                    <span class=\"p\">}<\/span>\n                <span class=\"k\">default<\/span><span class=\"o\">:<\/span>\n                    <span class=\"c\">\/\/ do nothing<\/span>\n                <span class=\"p\">}<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n        <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Sleep<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">MsgRate<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>There's a lot, but the most important bits are where we read from the bot's connection...<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"o\">...<\/span>\n<span class=\"c\">\/\/ reads from connection<\/span>\n<span class=\"n\">tp<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">textproto<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">bufio<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"p\">))<\/span>\n<span class=\"o\">...<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>...and attempt to keep it alive, by responding to PINGs.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"o\">...<\/span>\n<span class=\"k\">if<\/span> <span class=\"s\">\"PING :tmi.twitch.tv\"<\/span> <span class=\"o\">==<\/span> <span class=\"n\">line<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"c\">\/\/ respond to PING message with a PONG message, to maintain the connection<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"PONG :tmi.twitch.tv<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">))<\/span>\n    <span class=\"k\">continue<\/span>\n<span class=\"p\">}<\/span>\n<span class=\"o\">...<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The handling of the chat occurs in an infinite loop and has a slight delay to prevent the bot from breaking the message limits, if it were to send chat messages itself.<\/p>\n\n<h4>\n  \n  \n  JoinChannel()\n<\/h4>\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Makes the bot join its pre-specified channel.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">JoinChannel<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Joining #%s...<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"PASS \"<\/span> <span class=\"o\">+<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Credentials<\/span><span class=\"o\">.<\/span><span class=\"n\">Password<\/span> <span class=\"o\">+<\/span> <span class=\"s\">\"<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">))<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"NICK \"<\/span> <span class=\"o\">+<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Name<\/span> <span class=\"o\">+<\/span> <span class=\"s\">\"<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">))<\/span>\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"s\">\"JOIN #\"<\/span> <span class=\"o\">+<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span> <span class=\"o\">+<\/span> <span class=\"s\">\"<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">))<\/span>\n\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"[%s] Joined #%s as @%s!<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span><span class=\"p\">,<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Name<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><code>BasicBot.JoinChannel()<\/code> is quite important as it passes along all the necessary information to Twitch, including the bot's password, to create the desired connection.<\/p>\n\n<p><code>bb.Credentials.Password<\/code> will be initialized by the <code>BasicBot.ReadCredentials()<\/code> function, which is coming up.<\/p>\n\n<h4>\n  \n  \n  ReadCredentials()\n<\/h4>\n\n<p>In order for <code>BasicBot.ReadCredentials()<\/code> to do its job, it will need to be able to parse a JSON file at a pre-specified path (stored in the <code>PrivatePath<\/code> field). So, we'll need to add four more packages to the import block.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"bufio\"<\/span>\n    <span class=\"s\">\"encoding\/json\"<\/span> <span class=\"c\">\/\/ 1<\/span>\n    <span class=\"s\">\"errors\"<\/span>\n    <span class=\"s\">\"fmt\"<\/span>\n    <span class=\"s\">\"io\"<\/span>            <span class=\"c\">\/\/ 2<\/span>\n    <span class=\"s\">\"io\/ioutil\"<\/span>     <span class=\"c\">\/\/ 3<\/span>\n    <span class=\"s\">\"net\"<\/span>\n    <span class=\"s\">\"net\/textproto\"<\/span>\n    <span class=\"s\">\"regexp\"<\/span>\n    <span class=\"s\">\"strings\"<\/span>       <span class=\"c\">\/\/ 4<\/span>\n    <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, the function itself:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Reads from the private credentials file and stores the data in the bot's Credentials field.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">ReadCredentials<\/span><span class=\"p\">()<\/span> <span class=\"kt\">error<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"c\">\/\/ reads from the file<\/span>\n    <span class=\"n\">credFile<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">ioutil<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadFile<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">PrivatePath<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">err<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Credentials<\/span> <span class=\"o\">=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">OAuthCred<\/span><span class=\"p\">{}<\/span>\n\n    <span class=\"c\">\/\/ parses the file contents<\/span>\n    <span class=\"n\">dec<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">json<\/span><span class=\"o\">.<\/span><span class=\"n\">NewDecoder<\/span><span class=\"p\">(<\/span><span class=\"n\">strings<\/span><span class=\"o\">.<\/span><span class=\"n\">NewReader<\/span><span class=\"p\">(<\/span><span class=\"kt\">string<\/span><span class=\"p\">(<\/span><span class=\"n\">credFile<\/span><span class=\"p\">)))<\/span>\n    <span class=\"k\">if<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">dec<\/span><span class=\"o\">.<\/span><span class=\"n\">Decode<\/span><span class=\"p\">(<\/span><span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Credentials<\/span><span class=\"p\">);<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"n\">io<\/span><span class=\"o\">.<\/span><span class=\"n\">EOF<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">err<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Note: An <code>*OAuthCred<\/code> is no longer being returned. So update the definition for <code>TwitchBot.ReadCredentials()<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">type<\/span> <span class=\"n\">TwitchBot<\/span> <span class=\"k\">interface<\/span> <span class=\"p\">{<\/span>\n    <span class=\"o\">...<\/span>\n    <span class=\"n\">ReadCredentials<\/span><span class=\"p\">()<\/span> <span class=\"kt\">error<\/span>\n    <span class=\"o\">...<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h4>\n  \n  \n  Say()\n<\/h4>\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Makes the bot send a message to the chat channel.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">Say<\/span><span class=\"p\">(<\/span><span class=\"n\">msg<\/span> <span class=\"kt\">string<\/span><span class=\"p\">)<\/span> <span class=\"kt\">error<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">if<\/span> <span class=\"s\">\"\"<\/span> <span class=\"o\">==<\/span> <span class=\"n\">msg<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">errors<\/span><span class=\"o\">.<\/span><span class=\"n\">New<\/span><span class=\"p\">(<\/span><span class=\"s\">\"BasicBot.Say: msg was empty.\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">conn<\/span><span class=\"o\">.<\/span><span class=\"n\">Write<\/span><span class=\"p\">([]<\/span><span class=\"kt\">byte<\/span><span class=\"p\">(<\/span><span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Sprintf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"PRIVMSG #%s %s<\/span><span class=\"se\">\\r\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span><span class=\"p\">,<\/span> <span class=\"n\">msg<\/span><span class=\"p\">)))<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"k\">return<\/span> <span class=\"n\">err<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h4>\n  \n  \n  Start()\n<\/h4>\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"c\">\/\/ Starts a loop where the bot will attempt to connect to the Twitch IRC server, then connect to the<\/span>\n<span class=\"c\">\/\/ pre-specified channel, and then handle the chat. It will attempt to reconnect until it is told to<\/span>\n<span class=\"c\">\/\/ shut down, or is forcefully shutdown.<\/span>\n<span class=\"k\">func<\/span> <span class=\"p\">(<\/span><span class=\"n\">bb<\/span> <span class=\"o\">*<\/span><span class=\"n\">BasicBot<\/span><span class=\"p\">)<\/span> <span class=\"n\">Start<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">ReadCredentials<\/span><span class=\"p\">()<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Println<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Println<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Aborting...\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"k\">return<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"k\">for<\/span> <span class=\"p\">{<\/span>\n        <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Connect<\/span><span class=\"p\">()<\/span>\n        <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">JoinChannel<\/span><span class=\"p\">()<\/span>\n        <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">HandleChat<\/span><span class=\"p\">()<\/span>\n        <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n\n            <span class=\"c\">\/\/ attempts to reconnect upon unexpected chat error<\/span>\n            <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Sleep<\/span><span class=\"p\">(<\/span><span class=\"m\">1000<\/span> <span class=\"o\">*<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Millisecond<\/span><span class=\"p\">)<\/span>\n            <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Println<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n            <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Println<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Starting bot again...\"<\/span><span class=\"p\">)<\/span>\n        <span class=\"p\">}<\/span> <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\n            <span class=\"k\">return<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><code>BasicBot.Start()<\/code> attempts to initialize the bot's credentials before kicking everything else off.<\/p>\n\n<p>Go ahead and build now, if there are no explosions you're done!<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go <span class=\"nb\">fmt<\/span> .\/... <span class=\"o\">&amp;&amp;<\/span> go build\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>BasicBot should now be fully functional.<\/p>\n\n<h2>\n  \n  \n  Step 3\/4\n<\/h2>\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> step-3\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Cool, now that we've got a fully functioning bot, let's run it. Unfortunately that can't be done within the <code>twitchbot<\/code> package since it's just a library. You can however grab my test repo!<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go get github.com\/foresthoffman\/twitchbotex\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Navigate your terminal into the twitchbotex directory...<\/p>\n\n<p>Linux\/MacOS<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span><span class=\"nb\">cd<\/span> <span class=\"nv\">$GOPATH<\/span>\/src\/github.com\/foresthoffman\/twitchbotex\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Windows<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight powershell\"><code><span class=\"err\">&gt;<\/span><span class=\"w\"> <\/span><span class=\"n\">cd<\/span><span class=\"w\"> <\/span><span class=\"o\">%<\/span><span class=\"nx\">GOPATH<\/span><span class=\"o\">%<\/span><span class=\"nx\">\\src\\github.com\\foresthoffman\\twitchbotex<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>...and open the <code>main.go<\/code> file. You're going to want to replace the <code>Channel<\/code> and <code>Name<\/code> values with your own channel usernames. Remember that the <code>Channel<\/code> value <strong>must<\/strong> be lowercase.<\/p>\n\n<p>Then, from your terminal, you're going to want to add a <code>private\/<\/code> directory in the <code>twitchbotex<\/code> package directory.<\/p>\n\n<p><code>private\/<\/code> is where your <code>oauth.json<\/code> file and the bot's password will be contained. To retrieve this password, you need to connect to Twitch using their <a href=\"http:\/\/twitchapps.com\/tmi\/\" rel=\"noopener noreferrer\">OAuth Password Generator<\/a>. The token will have the following pattern: \"oauth:secretsecretsecretsecretsecret\".<\/p>\n\n<p>Once you have the OAuth token for the bot account, you can place it in the <code>oauth.json<\/code> file containing the following JSON (using your token):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"password\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"oauth:secretsecretsecretsecretsecret\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This will allow the bot to read in the credentials before attempting to connect to Twitch's servers.<\/p>\n\n<p>With all that done, you should now be able to build and run the bot:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go <span class=\"nb\">fmt<\/span> .\/... <span class=\"o\">&amp;&amp;<\/span> go build <span class=\"o\">&amp;&amp;<\/span> .\/twitchbotex\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Entering <code>!tbdown<\/code> into your chat, with the bot running, will gently tell the bot to shutdown, per this block from <code>BasicBot.HandleChat()<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"o\">...<\/span>\n<span class=\"c\">\/\/ channel-owner specific commands<\/span>\n<span class=\"k\">if<\/span> <span class=\"n\">userName<\/span> <span class=\"o\">==<\/span> <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Channel<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">switch<\/span> <span class=\"n\">cmd<\/span> <span class=\"p\">{<\/span>\n    <span class=\"k\">case<\/span> <span class=\"s\">\"tbdown\"<\/span><span class=\"o\">:<\/span>\n        <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span>\n            <span class=\"s\">\"[%s] Shutdown command received. Shutting down now...<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span>\n            <span class=\"n\">timeStamp<\/span><span class=\"p\">(),<\/span>\n        <span class=\"p\">)<\/span>\n\n        <span class=\"n\">bb<\/span><span class=\"o\">.<\/span><span class=\"n\">Disconnect<\/span><span class=\"p\">()<\/span>\n        <span class=\"k\">return<\/span> <span class=\"no\">nil<\/span>\n    <span class=\"k\">default<\/span><span class=\"o\">:<\/span>\n        <span class=\"c\">\/\/ do nothing<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">}<\/span>\n<span class=\"o\">...<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Pretty Logs\n<\/h2>\n\n<p>If you'd like to add a dash of color to the bot's log messages, you can switch back to the <code>twitchbot<\/code> package really quick and run:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> step-4\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then rebuild and run <code>twitchbotex<\/code>, and the logs will be a little easier on the eyes.<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforesthoffman.com%2Fwp-content%2Fuploads%2F2017%2F12%2Fslice1.jpg\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fforesthoffman.com%2Fwp-content%2Fuploads%2F2017%2F12%2Fslice1.jpg\" title=\"Standard Output Color Comparison\" alt=\"An image comparison of the console without and with colored text.\" width=\"800\" height=\"400\"><\/a><\/p>\n\n<p>Thank you for reading! If you have any questions you can hit me up on Twitter (<a href=\"https:\/\/twitter.com\/forestjhoffman\" rel=\"noopener noreferrer\">@forestjhoffman<\/a>) or email (<a href=\"mailto:forestjhoffman@gmail.com\">forestjhoffman@gmail.com<\/a>).<\/p>\n\n<h2>\n  \n  \n  Image Attribution\n<\/h2>\n\n<ul>\n<li>Photo by <a href=\"https:\/\/unsplash.com\/@casparrubin?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Caspar Camille Rubin<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/twitch?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>\n<\/li>\n<li>Glitch, Copyright of Twitch.tv<\/li>\n<\/ul>\n\n","category":["showdev","webdev","go","api"]},{"title":"Building a Twitch.tv Chat Bot with Go - Part 1","pubDate":"Wed, 20 Dec 2017 04:12:58 +0000","link":"https:\/\/dev.to\/foresthoffman\/building-a-twitchtv-chat-bot-with-go---part-1-i3k","guid":"https:\/\/dev.to\/foresthoffman\/building-a-twitchtv-chat-bot-with-go---part-1-i3k","description":"<p>EDIT (5\/13\/2019): Since the time of writing the package has been renamed from \"twitchbot\" to \"bot\". However, the tagged releases used for this tutorial still use the old name.<\/p>\n\n<p>For those that are unfamiliar, <a href=\"https:\/\/twitch.tv\" rel=\"noopener noreferrer\">Twitch.tv<\/a> is a live-streaming platform for all things creative and related to games. Not necessarily just video games. Content creators play board games, roll-play games (e.g. Dungeons &amp; Dragons), and everything in between. On this platform, streamers are able to interact with a live chat room just for their channel (with a delay).<\/p>\n\n<p>To facilitate the streamer and chat interaction, Twitch uses a variant of <a href=\"https:\/\/tools.ietf.org\/html\/rfc1459.html\" rel=\"noopener noreferrer\">IRC<\/a>, and provides documentation for their implementation on their <a href=\"https:\/\/dev.twitch.tv\/docs\/irc\" rel=\"noopener noreferrer\">Twitch Developers page<\/a>. This allows developers to create chat bots that can moderate chat rooms, interact with chatters, and automate certain tasks for streamers. There are several well known ones, with many features, but the chat bot built in this walkthrough won't be as complex.<\/p>\n\n<h2>\n  \n  \n  Important Notes\n<\/h2>\n\n<p>The code for this project can be found on <a href=\"https:\/\/github.com\/foresthoffman\/bot\" rel=\"noopener noreferrer\">GitHub<\/a> and is under <a href=\"https:\/\/github.com\/foresthoffman\/bot\/blob\/master\/LICENSE\" rel=\"noopener noreferrer\">the MIT license<\/a>. Anyone who is developing a chat bot for Twitch chat should read and abide by their <a href=\"https:\/\/www.twitch.tv\/p\/legal\/developer-agreement\/\" rel=\"noopener noreferrer\">Developer Agreement<\/a> and their <a href=\"https:\/\/www.twitch.tv\/p\/legal\/terms-of-service\/\" rel=\"noopener noreferrer\">Terms of Service<\/a>. You should also be aware of the <a href=\"https:\/\/dev.twitch.tv\/docs\/irc#irc-command-and-message-limits\" rel=\"noopener noreferrer\">IRC message limits<\/a> which could result in a timeout (temporary ban) or even a lengthy IP ban, if they are broken. Therefore you should <strong>follow the rules<\/strong>, and make an alternate account just for your bot (in case of mistakes).<\/p>\n\n<p>On that note, <strong>I recommend having two accounts in total<\/strong>, one for testing the bot and one acting as the bot. That way you can test the bot in your own chat, rather than someone else's chat (which is generally a bad idea).<\/p>\n\n<h2>\n  \n  \n  Getting Started\n<\/h2>\n\n<p>In this walkthrough we'll go step by step and fill out all the features that will be needed for the chat bot to function in a live chat room.<\/p>\n\n<p>Here are the requirements for this project:<\/p>\n\n<ul>\n<li><a href=\"https:\/\/golang.org\/doc\/install\" rel=\"noopener noreferrer\">Have Go installed<\/a><\/li>\n<li>\n<a href=\"https:\/\/www.twitch.tv\/signup\" rel=\"noopener noreferrer\">Have two Twitch.tv Accounts<\/a> (one for you, and one for your bot)<\/li>\n<\/ul>\n\n<p>Once you have the requirements sorted, we can get started!<\/p>\n\n<p>Go ahead and run the following in your terminal, if you haven't already:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go get github.com\/foresthoffman\/bot\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then you can navigate to the package's source:<\/p>\n\n<p>Linux\/MacOS<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span><span class=\"nb\">cd<\/span> <span class=\"nv\">$GOPATH<\/span>\/src\/github.com\/foresthoffman\/bot\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Windows<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"o\">&gt;<\/span> <span class=\"nb\">cd<\/span> %GOPATH%\/src\/github.com\/foresthoffman\/bot\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>After which you can turn back time by running the following Git command:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git checkout <span class=\"nt\">-b<\/span> walkthrough step-0\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This will create a brand new branch called \"walkthrough\" and bring it to Step 0, where we'll begin writing code.<\/p>\n\n<p>If you get stuck and something won't compile, you can always jump forward or backward a step with (where \"#\" represents the index of the step you want to jump to):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> step-#\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If you want to jump to the end, you can always run:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>git reset <span class=\"nt\">--hard<\/span> master\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h2>\n  \n  \n  Step 0\n<\/h2>\n\n<p>Okay, so open up <code>twitchbot.go<\/code> in your favorite editor, and let's look at what we're working with.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">twitchbot<\/span>\n\n<span class=\"c\">\/\/ TODO:<\/span>\n<span class=\"c\">\/\/ 1. Connect to a Twitch.tv Chat channel.<\/span>\n<span class=\"c\">\/\/  a. Pass along necessary information for the connection.<\/span>\n<span class=\"c\">\/\/   i.   The IRC (chat) server.<\/span>\n<span class=\"c\">\/\/   ii.  The port on the server.<\/span>\n<span class=\"c\">\/\/   iii. The channel we want the bot to join.<\/span>\n<span class=\"c\">\/\/   iv.  The bot's name.<\/span>\n<span class=\"c\">\/\/   v.   A secure key to allow the bot to connect indirectly (not through the website).<\/span>\n<span class=\"c\">\/\/   vi.  A maximum speed at which the bot can respond.<\/span>\n<span class=\"c\">\/\/ 2. Listen for messages in the chat.<\/span>\n<span class=\"c\">\/\/ 3. Do things based on what is happening in the chat.<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>So, ignoring the licensing bit, we have a layout of what we need for the bot to function, in addition to some nice-to-haves. Let's focus on the most important one, which is #1.<\/p>\n\n<ol>\n<li>Connect to a Twitch.tv Chat channel.<\/li>\n<\/ol>\n\n<p>In order to accomplish #1 we need some data:<\/p>\n\n<ul>\n<li>The IRC (chat) server.<\/li>\n<li>The port on the server.<\/li>\n<li>The channel we want the bot to join.<\/li>\n<li>The bot's name.<\/li>\n<li>A secure key to allow the bot to connect indirectly (not through the website).<\/li>\n<li>A maximum speed at which the bot can respond.<\/li>\n<\/ul>\n\n<p>Using placeholder usernames and OAuth codes (you'll use your own when we get to it), we have the following data:<\/p>\n\n<ul>\n<li>irc.chat.twitch.tv<\/li>\n<li>6667<\/li>\n<li>TwitchBotTest<\/li>\n<li>TwitchBot<\/li>\n<li>oauth:secretsecretsecretsecretsecret<\/li>\n<li>once every 20\/30 of a millisecond<\/li>\n<\/ul>\n\n<p>So, we'll need the bot to keep track of 5 strings and one time interval. So, let's create a struct to handle that data, in <code>twitchbot.go<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">twitchbot<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">BasicBot<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Channel<\/span> <span class=\"kt\">string<\/span>\n    <span class=\"n\">MsgRate<\/span> <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Duration<\/span>\n    <span class=\"n\">Name<\/span>    <span class=\"kt\">string<\/span>\n    <span class=\"n\">Port<\/span>    <span class=\"kt\">string<\/span>\n    <span class=\"n\">OAuth<\/span>   <span class=\"kt\">string<\/span>\n    <span class=\"n\">Server<\/span>  <span class=\"kt\">string<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Now we have some nicely named fields to hold our data. Here are some things to keep in mind though: <code>Channel<\/code> must be lowercase or else the bot will successfully connect to the server, but will be blind to any messages in chat; and by having an <code>OAuth<\/code> field, we have to hard code our bot's password, which is not considered best practice.<\/p>\n\n<p>Rather than having an <code>OAuth<\/code> field, let's accept a path to a limited-access directory from which the bot's OAuth token can be read at runtime. That way it can be easily changed out, without having to recompile our source code or restart the bot.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">twitchbot<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"time\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">type<\/span> <span class=\"n\">BasicBot<\/span> <span class=\"k\">struct<\/span> <span class=\"p\">{<\/span>\n    <span class=\"n\">Channel<\/span>     <span class=\"kt\">string<\/span>\n    <span class=\"n\">MsgRate<\/span>     <span class=\"n\">time<\/span><span class=\"o\">.<\/span><span class=\"n\">Duration<\/span>\n    <span class=\"n\">Name<\/span>        <span class=\"kt\">string<\/span>\n    <span class=\"n\">Port<\/span>        <span class=\"kt\">string<\/span>\n    <span class=\"n\">PrivatePath<\/span> <span class=\"kt\">string<\/span> <span class=\"c\">\/\/ replaced the OAuth field<\/span>\n    <span class=\"n\">Server<\/span>      <span class=\"kt\">string<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I've left out the field comments and todo-list to save space, but that's it for Step 0. You can build &amp; fmt your code now!<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>go <span class=\"nb\">fmt<\/span> .\/... <span class=\"o\">&amp;&amp;<\/span> go build\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Thank you for reading! If you have any questions you can hit me up on Twitter (<a href=\"https:\/\/twitter.com\/forestjhoffman\" rel=\"noopener noreferrer\">@forestjhoffman<\/a>) or email (<a href=\"mailto:forestjhoffman@gmail.com\">forestjhoffman@gmail.com<\/a>).<\/p>\n\n<h2>\n  \n  \n  Image Attribution\n<\/h2>\n\n<ul>\n<li>Photo by <a href=\"https:\/\/unsplash.com\/@casparrubin?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Caspar Camille Rubin<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/twitch?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>\n<\/li>\n<li>Glitch, Copyright of Twitch.tv<\/li>\n<\/ul>\n\n","category":["showdev","webdev","go","api"]},{"title":"Detaching Unix Child Processes with Go","pubDate":"Tue, 03 Oct 2017 13:48:38 +0000","link":"https:\/\/dev.to\/foresthoffman\/detaching-unix-child-processes-with-go-80n","guid":"https:\/\/dev.to\/foresthoffman\/detaching-unix-child-processes-with-go-80n","description":"<p>EDIT (5\/13\/2019): <code>midproc<\/code> and <code>midprocrunner<\/code> have been renamed to <code>reap<\/code> and <code>reaper<\/code>, respectively.<\/p>\n\n<p>I ran into a situation with a project where I needed to build two separate programs to work together. The first being a server, and the second being an on-going client-facing process. The server is designed to kick off any number of instances of the on-going process. Before having written any code to have the server run the on-going processes as children, I had intended the child processes to be independent of the server. That is, if the server went down, the child processes would still function. They would store requests in a queue and periodically attempt to reconnect to the server. Once the connection was re-established, everything would go back to normal. It turned out that Unix processes don't quite work like I had expected.<\/p>\n\n<p>In reality, when a process spawns another process, it is called a \"child\"; the original is called the \"parent\". As long as the parent process has not exited, the child will continue to run. If for some reason the parent goes down, so does the child. That is, unless the child is considered \"detached\". A child process is detached when it no longer has a parent (I believe that technically, parent-less children are owned by an <code>init<\/code> process). Thus, you can have a parent process spawn a detached child and immediately exit, which will leave the child process all alone.<\/p>\n\n<p>However, there is a caveat with detached child processes. If a detached child process is given an exit signal (e.g. <a href=\"http:\/\/man7.org\/linux\/man-pages\/man2\/kill.2.html\" rel=\"noopener noreferrer\">SIGKILL<\/a>) before its original parent has exited, it will become a \"zombie\". Zombie processes do not consume resources, but they are still referenced in the process ID table. In order to dispose of the zombies, the original parent must exit.<\/p>\n\n<p>Now, if you're trying to build a SaaS product where up-time is paramount, shutting down the server to periodically cleanup zombie processes is not a viable solution.<\/p>\n\n<p>It was at this point when researching the situation that I decided that my goal was unachievable. I expressed my frustration to a co-worker, who gave some encouraging feedback. Still stumped, I took a few days to work on other things that needed my immediate attention. When I came back to the problem, I had a \"d'oh!\" moment.<\/p>\n\n<p>I realized that I was misunderstanding how processes interacted. In the example I mentioned previously, when a detached-child-process's top-level parent exits, the child is assigned to the <code>init<\/code> process. The same occurs when you have a nested child process. For example:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>- Grandparent Process\n  - Parent Process\n    - Child Process\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Here is where I was getting confused. I had assumed that when the parent process exits, the detached child process would be assigned to the grandparent process. I was wrong. No matter how nested a detached child process is, when its parent exits, it's assigned to the <code>init<\/code> process. That makes it a top-level process. Then, the grandparent and child processes would be side-by-side, running independently.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>- Grandparent Process\n- Child Process\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>When I finally understood this, I went about creating the solution. After which I released the <a href=\"https:\/\/github.com\/foresthoffman\/reap\" rel=\"noopener noreferrer\"><code>reap<\/code><\/a> and <a href=\"https:\/\/github.com\/foresthoffman\/reaper\" rel=\"noopener noreferrer\"><code>reaper<\/code><\/a> Go packages.<\/p>\n\n<p>The <code>reap<\/code> package can be used to create intermediate process runners. Or, in the context of the previous example, a temporary nested-parent process.<\/p>\n\n<p>The <code>reaper<\/code> package is an intermediate process runner that implements the <code>reap<\/code> package. It's a simple implementation that doesn't get in the way, and gets the job done.<\/p>\n\n<p>Here's an example of the usage of the <code>reaper<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">package<\/span> <span class=\"n\">main<\/span>\n\n<span class=\"k\">import<\/span> <span class=\"p\">(<\/span>\n    <span class=\"s\">\"bytes\"<\/span>\n    <span class=\"s\">\"fmt\"<\/span>\n    <span class=\"s\">\"os\/exec\"<\/span>\n    <span class=\"s\">\"strconv\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n\n    <span class=\"c\">\/\/ prepare a buffer, to which the PID will be written<\/span>\n    <span class=\"k\">var<\/span> <span class=\"n\">stdout<\/span> <span class=\"n\">bytes<\/span><span class=\"o\">.<\/span><span class=\"n\">Buffer<\/span>\n\n    <span class=\"c\">\/\/ prepare the command<\/span>\n    <span class=\"n\">sleepCmd<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">exec<\/span><span class=\"o\">.<\/span><span class=\"n\">Command<\/span><span class=\"p\">(<\/span><span class=\"s\">\"reaper\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"-cmd='sleep'\"<\/span><span class=\"p\">,<\/span> <span class=\"s\">\"-args='30'\"<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">sleepCmd<\/span><span class=\"o\">.<\/span><span class=\"n\">Stdout<\/span> <span class=\"o\">=<\/span> <span class=\"o\">&amp;<\/span><span class=\"n\">stdout<\/span>\n\n    <span class=\"c\">\/\/ run the command<\/span>\n    <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">sleepCmd<\/span><span class=\"o\">.<\/span><span class=\"n\">Run<\/span><span class=\"p\">()<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n\n    <span class=\"c\">\/\/ convert the PID string to a valid integer<\/span>\n    <span class=\"n\">pidInt<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">strconv<\/span><span class=\"o\">.<\/span><span class=\"n\">ParseInt<\/span><span class=\"p\">(<\/span><span class=\"n\">stdout<\/span><span class=\"o\">.<\/span><span class=\"n\">String<\/span><span class=\"p\">(),<\/span> <span class=\"m\">10<\/span><span class=\"p\">,<\/span> <span class=\"m\">64<\/span><span class=\"p\">)<\/span>\n    <span class=\"k\">if<\/span> <span class=\"no\">nil<\/span> <span class=\"o\">!=<\/span> <span class=\"n\">err<\/span> <span class=\"p\">{<\/span>\n        <span class=\"nb\">panic<\/span><span class=\"p\">(<\/span><span class=\"n\">err<\/span><span class=\"p\">)<\/span>\n    <span class=\"p\">}<\/span>\n    <span class=\"n\">pid<\/span> <span class=\"o\">:=<\/span> <span class=\"kt\">int<\/span><span class=\"p\">(<\/span><span class=\"n\">pidInt<\/span><span class=\"p\">)<\/span>\n    <span class=\"n\">fmt<\/span><span class=\"o\">.<\/span><span class=\"n\">Printf<\/span><span class=\"p\">(<\/span><span class=\"s\">\"Created a detached process with an ID of %d!<\/span><span class=\"se\">\\n<\/span><span class=\"s\">\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">pid<\/span><span class=\"p\">)<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Compiling and running the above program, with <code>reaper<\/code> installed, will result in a detached-process for the <code>sleep<\/code> command. Based on the time provided, <code>30<\/code> seconds by default, the process will stay around after its parent exits. It's also nice enough to clean up after itself when it's done. This is a simple example, but the runner could be used to detach any individual process.<\/p>\n\n<p>For my needs, this works splendidly. If you're interested in using this, and would like to see the ability to detach multiple commands at once (e.g. <code>sleep 30 &amp;&amp; echo \"something\" | awk '{ print $0 }'<\/code>) <a href=\"https:\/\/github.com\/foresthoffman\/reaper\/issues\" rel=\"noopener noreferrer\">let me know<\/a>!<\/p>\n\n<p><em>That's all, enjoy!<\/em><\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Cover image by <a href=\"https:\/\/unsplash.com\/@madebyjens?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Jens Lelie<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/fork-in-the-road?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>! :D<\/p>\n\n","category":["go","unix","webdev","showdev"]},{"title":"Hosting WordPress over HTTPS with Docker","pubDate":"Fri, 29 Sep 2017 15:51:11 +0000","link":"https:\/\/dev.to\/foresthoffman\/hosting-wordpress-over-https-with-docker-5gc","guid":"https:\/\/dev.to\/foresthoffman\/hosting-wordpress-over-https-with-docker-5gc","description":"<blockquote>\n<p>UPDATE 10\/13\/2020: Hello! Forest from the future here! I no longer use this setup. And, I do not intend to keep it up to date. Since I no longer maintain this for myself, and seeing as it is a very custom solution, I do not have the time to create a repository for it.<\/p>\n\n<p>This solution is at least 3 years old. If you are doomed to use a legacy version of WordPress and this solution works for you, I'm happy to have helped. On the other hand, if you are using a newer version of WordPress, Nginx, Docker, or Certbot, there may be some nuggets of useful information in here, but beyond that I wish you the best of luck!<\/p>\n\n<p>Cheers! \ud83d\ude04<\/p>\n<\/blockquote>\n\n<p><a href=\"https:\/\/twitter.com\/ForestJHoffman\/status\/912031600405123073\" rel=\"noopener noreferrer\">Just the other day<\/a>, I moved my portfolio to a separate server and started serving it over HTTPS. I was super stoked when it was all done! I wanted to talk a bit about what steps I took, since I found some annoying gotchas along the way. This isn't a step-by-step tutorial, rather I'm sharing the configurations that finally got it working for me.<\/p>\n\n<p>I use <code>example.com<\/code> as a placeholder here. If you're reading this and trying to get your own setup working, you'd replace <code>example.com<\/code> with your own domain. Also, though I doubt it needs to be said, don't use <code>root<\/code> as your MySQL password. <a><\/a><\/p>\n\n<h2>\n  \n  \n  Table of Contents\n<\/h2>\n\n<ul>\n<li>The Server Stack<\/li>\n<li>Docker Compose Configuration Overview<\/li>\n<li>\nNginx Proxy Container: docker-compose.yml, Dockerfile, crontab, startup.sh, nginx.conf\n<\/li>\n<li>\nThe WordPress Containers: docker-compose.yml\n<\/li>\n<li>\nNginx Container: docker-compose.yml, nginx.conf, nginx\/wordpress.conf\n<\/li>\n<li>\nWordPress Container: docker-compose.yml, wordpress\/wp-config.php\n<\/li>\n<li>\nMariadb Container: docker-compose.yml\n<\/li>\n<li>Installing SSL Certificates with Certbot<\/li>\n<li>Further Reading<\/li>\n<\/ul>\n\n<p><a><\/a><\/p>\n\n<h2>\n  \n  \n  The Server Stack\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p>I use <a href=\"https:\/\/www.docker.com\/\" rel=\"noopener noreferrer\">Docker<\/a> to isolate my servers, so that they can all have their own dedicated environments. <em>That, and if I mess something up on a Docker container, I can just rebuild it.<\/em> Also, I use <a href=\"https:\/\/docs.docker.com\/compose\/\" rel=\"noopener noreferrer\">Docker Compose<\/a> to handle grouping together my containers.<\/p>\n\n<p>I chose <a href=\"https:\/\/nginx.org\/en\/\" rel=\"noopener noreferrer\">Nginx<\/a> as my proxy and my WordPress web server. Note that the proxy isn't necessary here, but I needed it for a few other things. I wanted to keep things consistent, so that's why I opted to use Nginx across the board.<\/p>\n\n<p>If you're trying to achieve a similar setup, but sans-proxy, you can just as well provide your WordPress container's Nginx server with the SSL certificates and add in the proper server blocks for SSL-support. I mention those below in the Nginx Proxy Container section.<\/p>\n\n<p>For my SSL certificates, I'm using <a href=\"https:\/\/letsencrypt.org\/\" rel=\"noopener noreferrer\">Let's Encrypt<\/a> via <a href=\"https:\/\/github.com\/certbot\/certbot\" rel=\"noopener noreferrer\">Certbot<\/a>. It's a lovely little tool that automates the whole certificate registration process.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  Docker Compose Configuration Overview\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p>The following sections list out all the necessary configurations for the following containers:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight plaintext\"><code>- proxy:\n  - nginx\n- wordpress:\n  - nginx\n  - wordpress\n  - mariadb\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><a><\/a><\/p>\n\n<h2>\n  \n  \n  Nginx Proxy Container\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p><span id=\"nginx-proxy-compose\"><strong>docker-compose.yml<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>version: <span class=\"s1\">'3.3'<\/span>\n\nservices:\n  nginx-proxy:\n    container_name: nginx-proxy\n    build: <span class=\"nb\">.<\/span>\n    image: nginx-proxy:0.0.2\n    ports:\n      - <span class=\"s2\">\"80:80\"<\/span>\n      - <span class=\"s2\">\"443:443\"<\/span>\n    volumes:\n      - <span class=\"s2\">\".\/etc\/letsencrypt\/:\/etc\/letsencrypt\/\"<\/span>\n      - <span class=\"s2\">\".\/etc\/ssl\/:\/etc\/ssl\/\"<\/span>\n    <span class=\"nb\">command<\/span>: bash <span class=\"nt\">-c<\/span> <span class=\"s2\">\"startup.sh\"<\/span>\n    restart: always\n\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The container uses an adjacent Dockerfile for building the custom nginx-proxy image. The container exposes port <code>80<\/code> and port <code>443<\/code> to the host machine. When the container is brought up, it mounts the two local directories to the <code>\/etc\/letsencrypt\/<\/code> and <code>\/etc\/ssl\/<\/code> directories on the container, respectively. Then, the default startup command (<code>nginx -g 'daemon off;'<\/code>) is overridden to run the <code>startup.sh<\/code> file. Finally, the container is configured to restart whenever it goes now, via the <code>restart: always<\/code> line.<\/p>\n\n<p><span id=\"nginx-proxy-dockerfile\"><strong>Dockerfile<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>FROM nginx:1.13.5-alpine\n\nLABEL <span class=\"nv\">maintainer<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"Forest Hoffman&lt;forestjhoffman@gmail.com&gt;\"<\/span>\n\n<span class=\"c\">##<\/span>\n<span class=\"c\"># setting necessary server configurations<\/span>\n<span class=\"c\">##<\/span>\n\n<span class=\"c\"># add curl and other necessary packages for openssl and certbot<\/span>\nRUN apk add <span class=\"nt\">--update<\/span> <span class=\"se\">\\<\/span>\n                curl <span class=\"se\">\\<\/span>\n                bash <span class=\"se\">\\<\/span>\n                openssl <span class=\"se\">\\<\/span>\n                certbot <span class=\"se\">\\<\/span>\n                python <span class=\"se\">\\<\/span>\n                py-pip <span class=\"se\">\\<\/span>\n        <span class=\"o\">&amp;&amp;<\/span> pip <span class=\"nb\">install<\/span> <span class=\"nt\">--upgrade<\/span> pip <span class=\"se\">\\<\/span>\n        <span class=\"o\">&amp;&amp;<\/span> pip <span class=\"nb\">install<\/span> <span class=\"s1\">'certbot-nginx'<\/span> <span class=\"se\">\\<\/span>\n        <span class=\"o\">&amp;&amp;<\/span> pip <span class=\"nb\">install<\/span> <span class=\"s1\">'pyopenssl'<\/span>\n\nRUN addgroup staff\nRUN addgroup www-data\nRUN adduser <span class=\"nt\">-h<\/span> \/home\/dev\/ <span class=\"nt\">-D<\/span> <span class=\"nt\">-g<\/span> <span class=\"s2\">\"\"<\/span> <span class=\"nt\">-G<\/span> staff dev\nRUN adduser <span class=\"nt\">-S<\/span> <span class=\"nt\">-H<\/span> <span class=\"nt\">-g<\/span> <span class=\"s2\">\"\"<\/span> <span class=\"nt\">-G<\/span> www-data www-data\nRUN <span class=\"nb\">echo <\/span>dev:trolls | chpasswd\n\n<span class=\"c\">##<\/span>\n<span class=\"c\"># copying nginx configuration file<\/span>\n<span class=\"c\">##<\/span>\n\nCOPY nginx.conf \/etc\/nginx\/nginx.conf\nRUN <span class=\"nb\">chown <\/span>root:staff \/etc\/nginx\/nginx.conf\n\n<span class=\"c\"># adds certbot cert renewal job to cron<\/span>\nCOPY crontab \/tmp\/crontab-certbot\nRUN <span class=\"o\">(<\/span>crontab <span class=\"nt\">-l<\/span><span class=\"p\">;<\/span> <span class=\"nb\">cat<\/span> \/tmp\/crontab-certbot<span class=\"o\">)<\/span> | crontab -\n\n<span class=\"c\"># copies over the startup file which handles initializing the nginx<\/span>\n<span class=\"c\"># server and the SSL certification process (if necessary)<\/span>\nCOPY startup.sh \/bin\/startup.sh\nRUN <span class=\"nb\">chown <\/span>root:root \/bin\/startup.sh\nRUN <span class=\"nb\">chmod <\/span>ug+rx \/bin\/startup.sh\nRUN <span class=\"nb\">chmod <\/span>go-w \/bin\/startup.sh\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The Dockerfile will include all the necessary requirements for running the Certbot within the container. It will also add a line to the container's <a href=\"http:\/\/www.unixgeeks.org\/security\/newbie\/unix\/cron-1.html\" rel=\"noopener noreferrer\">cron<\/a> which will run the Certbot in renewal mode, early in the morning (according to the Docker container's system time).<\/p>\n\n<p><span id=\"nginx-proxy-crontab\"><strong>crontab<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\"># Renew Let's Encrypt SSL Certificates that have &lt; 30 days to go,<\/span>\n<span class=\"c\"># in the morning (UTC time).<\/span>\n0 11 <span class=\"k\">*<\/span> <span class=\"k\">*<\/span> <span class=\"k\">*<\/span> \/usr\/bin\/certbot renew <span class=\"nt\">--quiet<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p><span id=\"nginx-proxy-startup\"><strong>startup.sh<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">#!\/bin\/bash<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># startup.sh<\/span>\n<span class=\"c\">#<\/span>\n\n<span class=\"c\"># Startup the nginx server. The server has to be active for the Let's Encrypt Certbot to<\/span>\n<span class=\"c\"># register and install the certificates.<\/span>\nnginx <span class=\"nt\">-g<\/span> <span class=\"s2\">\"daemon on;\"<\/span>\n\n<span class=\"c\"># Checks that the SSL certificates are installed. If they are, renews any that are old, and<\/span>\n<span class=\"c\"># installs them if not.<\/span>\n<span class=\"k\">if<\/span> <span class=\"o\">[[<\/span> <span class=\"nt\">-d<\/span> <span class=\"s2\">\"\/etc\/letsencrypt\/live\/example.com\"<\/span> <span class=\"o\">]]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n        <\/span>certbot renew <span class=\"nt\">--quiet<\/span>\n<span class=\"k\">else\n        if<\/span> <span class=\"o\">!<\/span> <span class=\"o\">[[<\/span> <span class=\"nt\">-d<\/span> <span class=\"s2\">\"\/etc\/letsencrypt\/live\/example.com\"<\/span> <span class=\"o\">]]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n                <\/span>certbot <span class=\"nt\">--nginx<\/span> <span class=\"nt\">-m<\/span> your-email@ex.com <span class=\"nt\">--agree-tos<\/span> <span class=\"nt\">--no-eff-email<\/span> <span class=\"nt\">--redirect<\/span> <span class=\"nt\">--expand<\/span> <span class=\"nt\">-d<\/span> example.com,www.example.com\n        <span class=\"k\">fi\n        if<\/span> <span class=\"o\">!<\/span> <span class=\"o\">[[<\/span> <span class=\"nt\">-f<\/span> <span class=\"s2\">\"\/etc\/ssl\/certs\/dhparam.pem\"<\/span> <span class=\"o\">]]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n                <\/span>openssl dhparam <span class=\"nt\">-out<\/span> \/etc\/ssl\/certs\/dhparam.pem 2048\n        <span class=\"k\">fi\nfi<\/span>\n\n<span class=\"c\"># Shuts down the daemonized nginx server and fires up one in the foreground.<\/span>\nnginx <span class=\"nt\">-s<\/span> stop <span class=\"o\">&amp;&amp;<\/span> nginx <span class=\"nt\">-g<\/span> <span class=\"s1\">'daemon off;'<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The <code>startup.sh<\/code> script is intended to be run whenever the container is brought up (i.e. with <code>docker-compose up<\/code>). It will only attempt to register certifications if the directory for the certifications are not already in the <code>\/etc\/letsencrypt\/live<\/code> directory on the container. The paths in <code>startup.sh<\/code> must coincide with those in the Nginx configuration file. Speaking of which...<\/p>\n\n<p><span id=\"nginx-proxy-config\"><strong>nginx.conf<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>user www-data<span class=\"p\">;<\/span>\nworker_processes 1<span class=\"p\">;<\/span>\npid \/run\/nginx.pid<span class=\"p\">;<\/span>\n\nevents <span class=\"o\">{<\/span>\n    worker_connections 1024<span class=\"p\">;<\/span>\n    <span class=\"c\"># multi_accept on;<\/span>\n<span class=\"o\">}<\/span>\n\nhttp <span class=\"o\">{<\/span>\n    <span class=\"c\">#sendfile on;<\/span>\n\n    upstream docker-wordpress <span class=\"o\">{<\/span>\n            server nginx-wordpress:80<span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># listen for HTTPS requests to example domain<\/span>\n    server <span class=\"o\">{<\/span>\n            ssl_dhparam \/etc\/ssl\/certs\/dhparam.pem<span class=\"p\">;<\/span>\n            server_name example.com www.example.com<span class=\"p\">;<\/span>\n\n            listen 443 ssl<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\n            ssl_certificate \/etc\/letsencrypt\/live\/example.com\/fullchain.pem<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\n            ssl_certificate_key \/etc\/letsencrypt\/live\/example.com\/privkey.pem<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\n            include \/etc\/letsencrypt\/options-ssl-nginx.conf<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\n\n            location \/ <span class=\"o\">{<\/span>\n                    proxy_pass       http:\/\/docker-wordpress<span class=\"p\">;<\/span>\n                    proxy_redirect   off<span class=\"p\">;<\/span>\n                    proxy_set_header Host <span class=\"nv\">$host<\/span><span class=\"p\">;<\/span>\n                    proxy_set_header X-Real-IP <span class=\"nv\">$remote_addr<\/span><span class=\"p\">;<\/span>\n                    proxy_set_header X-Forwarded-For <span class=\"nv\">$proxy_add_x_forwarded_for<\/span><span class=\"p\">;<\/span>\n                    proxy_set_header X-Forwarded-Host <span class=\"nv\">$server_name<\/span><span class=\"p\">;<\/span>\n                    proxy_set_header X-Forwarded-Proto https<span class=\"p\">;<\/span>\n            <span class=\"o\">}<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># handle ACME challenge from Certbot, and send HTTP requests to HTTPS<\/span>\n    server <span class=\"o\">{<\/span>\n            listen 80<span class=\"p\">;<\/span>\n            server_name example.com www.example.com<span class=\"p\">;<\/span>\n\n            <span class=\"c\"># listen for ACME challenge from Certbot<\/span>\n            location ^~ \/.well-known\/acme-challenge\/ <span class=\"o\">{<\/span>\n                    <span class=\"c\"># No HTTP authentication<\/span>\n                    allow all<span class=\"p\">;<\/span>\n\n                    default_type <span class=\"s2\">\"text\/plain\"<\/span><span class=\"p\">;<\/span>\n            <span class=\"o\">}<\/span>\n\n            location <span class=\"o\">=<\/span> \/.well-known\/acme-challenge\/ <span class=\"o\">{<\/span>\n                    <span class=\"k\">return <\/span>404<span class=\"p\">;<\/span>\n            <span class=\"o\">}<\/span>\n\n            <span class=\"c\"># Redirect other HTTP traffic to HTTPS<\/span>\n            location \/ <span class=\"o\">{<\/span>\n                    access_log off<span class=\"p\">;<\/span>\n                    <span class=\"k\">return <\/span>301 https:\/\/<span class=\"nv\">$host$request_uri<\/span><span class=\"p\">;<\/span>\n            <span class=\"o\">}<\/span>\n    <span class=\"o\">}<\/span>\n<span class=\"o\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This configures the proxy to listen for requests hitting port <code>443<\/code> (e.g. requests using <code>https:\/\/*<\/code>). If the requests are for the <code>example.com<\/code> or <code>www.example.com<\/code> domain, then the request is passed on to the <code>http:\/\/docker-wordpress<\/code> server. The <code>http:\/\/docker-wordpress<\/code> \"upstream\" server is defined as port <code>80<\/code> of the connected <code>nginx-wordpress<\/code> container. Connecting the containers together is accomplished by using the <code>networks<\/code> key in the <code>docker-compose.yml<\/code> files.<\/p>\n\n<p>[Edited 12\/05\/17]: I updated the Nginx configuration file above to include a server block for allowing the proxy to pass Certbot's ACME challenge. This is required for certificate renewal.<\/p>\n\n<p>Any requests to port <code>80<\/code> are not using SSL (just normal <code>http:\/\/*<\/code>), so those requests are redirected to port <code>443<\/code> by forcing the use of HTTPS in the url.<\/p>\n\n<p>This block...<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>listen 443 ssl<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\nssl_certificate \/etc\/letsencrypt\/live\/example.com\/fullchain.pem<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\nssl_certificate_key \/etc\/letsencrypt\/live\/example.com\/privkey.pem<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\ninclude \/etc\/letsencrypt\/options-ssl-nginx.conf<span class=\"p\">;<\/span> <span class=\"c\"># managed by Certbot<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Indicates where the SSL certificates for the desired domains will reside on the proxy container.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  The WordPress Containers\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p>Note that I've broken up the container configurations in the sub-sections below, but they are actually all from the same <code>docker-compose.yml<\/code> file.<\/p>\n\n<p>Here's what it looks like all together.<\/p>\n\n<p><span id=\"wordpress-compose\"><strong>docker-compose.yml<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>version: <span class=\"s1\">'3.3'<\/span>\n\nservices:\n  nginx:\n    container_name: nginx-wordpress\n    image: nginx:latest\n    ports:\n      - <span class=\"s1\">'80'<\/span>\n    volumes:\n      - .\/nginx:\/etc\/nginx\/conf.d\n      - .\/nginx.conf:\/etc\/nginx\/nginx.conf\n      - .\/logs\/nginx:\/var\/log\/nginx\n      - .\/wordpress:\/var\/www\/html\n    depends_on:\n      - wordpress\n    restart: always\n    networks:\n      - nginxproxy_default\n  mysql:\n    container_name: mysql\n    image: mariadb\n    ports:\n      - <span class=\"s1\">'3306'<\/span>\n    volumes:\n      - .\/mysql:\/var\/lib\/mysql\n    environment:\n      - <span class=\"nv\">MYSQL_ROOT_PASSWORD<\/span><span class=\"o\">=<\/span>root\n    restart: always\n    networks:\n      - nginxproxy_default\n  wordpress:\n    container_name: wp\n    image: wordpress-prod:4.8.1-php5.6-fpm\n    ports:\n      - <span class=\"s1\">'9000'<\/span>\n    volumes:\n      - .\/wordpress:\/var\/www\/html\n    environment:\n      - <span class=\"nv\">WORDPRESS_DB_NAME<\/span><span class=\"o\">=<\/span>wordpress\n      - <span class=\"nv\">WORDPRESS_TABLE_PREFIX<\/span><span class=\"o\">=<\/span>wpprefix_\n      - <span class=\"nv\">WORDPRESS_DB_HOST<\/span><span class=\"o\">=<\/span>mysql\n      - <span class=\"nv\">WORDPRESS_DB_PASSWORD<\/span><span class=\"o\">=<\/span>root\n    depends_on:\n      - mysql\n    restart: always\n    networks:\n      - nginxproxy_default\nnetworks:\n  nginxproxy_default:\n    external: <span class=\"nb\">true<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The key thing to see here is the top-level <code>networks<\/code> key, which allows each of the containers to connect to the proxy's default network. The network is not generated by any of the containers in this compose file, so the <code>nginxproxy_default<\/code> network is defined as an external network via the <code>external: true<\/code> line.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  Nginx for WordPress Container\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p><span id=\"wordpress-nginx-compose\"><strong>docker-compose.yml<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">###<\/span>\n  nginx:\n    container_name: nginx-wordpress\n    image: nginx:latest\n    ports:\n      - <span class=\"s1\">'80'<\/span>\n    volumes:\n      - .\/nginx:\/etc\/nginx\/conf.d\n      - .\/nginx.conf:\/etc\/nginx\/nginx.conf\n      - .\/logs\/nginx:\/var\/log\/nginx\n      - .\/wordpress:\/var\/www\/html\n    depends_on:\n      - wordpress\n    restart: always\n    networks:\n      - nginxproxy_default\n<span class=\"c\">###<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The nginx server in front of the WordPress container exposes port <code>80<\/code>, and mounts four directories. The first two are for nginx configuration, the next is for preserving logs, and the last one is for preserving the WordPress core file structure.<\/p>\n\n<p><span id=\"nginx-config\"><strong>nginx.conf<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>user www-data<span class=\"p\">;<\/span>\nworker_processes auto<span class=\"p\">;<\/span>\n\nevents <span class=\"o\">{<\/span>\n    worker_connections 768<span class=\"p\">;<\/span>\n    <span class=\"c\"># multi_accept on;<\/span>\n<span class=\"o\">}<\/span>\n\nhttp <span class=\"o\">{<\/span>\n\n    <span class=\"c\">##<\/span>\n    <span class=\"c\"># Basic Settings<\/span>\n    <span class=\"c\">##<\/span>\n\n    sendfile on<span class=\"p\">;<\/span>\n    tcp_nopush on<span class=\"p\">;<\/span>\n    tcp_nodelay on<span class=\"p\">;<\/span>\n    keepalive_timeout 65<span class=\"p\">;<\/span>\n    types_hash_max_size 2048<span class=\"p\">;<\/span>\n    <span class=\"c\"># server_tokens off;<\/span>\n\n    server_names_hash_bucket_size 64<span class=\"p\">;<\/span>\n    <span class=\"c\"># server_name_in_redirect off;<\/span>\n\n    include \/etc\/nginx\/mime.types<span class=\"p\">;<\/span>\n    default_type application\/octet-stream<span class=\"p\">;<\/span>\n\n    <span class=\"c\">##<\/span>\n    <span class=\"c\"># SSL Settings<\/span>\n    <span class=\"c\">##<\/span>\n\n    ssl_protocols TLSv1 TLSv1.1 TLSv1.2<span class=\"p\">;<\/span> <span class=\"c\"># Dropping SSLv3, ref: POODLE<\/span>\n    ssl_prefer_server_ciphers on<span class=\"p\">;<\/span>\n\n    <span class=\"c\">##<\/span>\n    <span class=\"c\"># Logging Settings<\/span>\n    <span class=\"c\">##<\/span>\n\n    access_log \/var\/log\/nginx\/access.log<span class=\"p\">;<\/span>\n    error_log \/var\/log\/nginx\/error.log<span class=\"p\">;<\/span>\n\n    <span class=\"c\">##<\/span>\n    <span class=\"c\"># Gzip Settings<\/span>\n    <span class=\"c\">##<\/span>\n\n    <span class=\"nb\">gzip <\/span>on<span class=\"p\">;<\/span>\n    gzip_disable <span class=\"s2\">\"msie6\"<\/span><span class=\"p\">;<\/span>\n\n    <span class=\"c\"># gzip_vary on;<\/span>\n    <span class=\"c\"># gzip_proxied any;<\/span>\n    <span class=\"c\"># gzip_comp_level 6;<\/span>\n    <span class=\"c\"># gzip_buffers 16 8k;<\/span>\n    <span class=\"c\"># gzip_http_version 1.1;<\/span>\n    <span class=\"c\"># gzip_types text\/plain text\/css application\/json application\/javascript text\/xml application\/xml application\/xml+rss text\/javascript;<\/span>\n\n    <span class=\"c\">##<\/span>\n    <span class=\"c\"># Virtual Host Configs<\/span>\n    <span class=\"c\">##<\/span>\n\n    include \/etc\/nginx\/conf.d\/<span class=\"k\">*<\/span>.conf<span class=\"p\">;<\/span>\n    include \/etc\/nginx\/sites-enabled\/<span class=\"k\">*<\/span><span class=\"p\">;<\/span>\n<span class=\"o\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This is just a basic configuration file that includes any additional configuration files mounted to the <code>\/etc\/nginx\/conf.d<\/code> and <code>\/etc\/nginx\/sites-enabled\/<\/code> directories. This file also specifies the <code>www-data<\/code> user as the user that Nginx should use when attempting to serve files. This means that the server's file structure (mounted to <code>\/var\/www\/html<\/code>) should be accessible by the <code>www-data<\/code> user\/group.<\/p>\n\n<p><span id=\"wordpress-nginx-config\"><strong>nginx\/wordpress.conf<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>server <span class=\"o\">{<\/span>\n    listen 80<span class=\"p\">;<\/span>\n\n    root \/var\/www\/html<span class=\"p\">;<\/span>\n    index index.php<span class=\"p\">;<\/span>\n\n    access_log \/var\/log\/nginx\/wp-access.log<span class=\"p\">;<\/span>\n    error_log \/var\/log\/nginx\/wp-error.log<span class=\"p\">;<\/span>\n\n    location \/ <span class=\"o\">{<\/span>\n        try_files <span class=\"nv\">$uri<\/span> <span class=\"nv\">$uri<\/span>\/ \/index.php?<span class=\"nv\">$args<\/span><span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    location ~ <span class=\"se\">\\.<\/span><span class=\"o\">(<\/span>png|jpg|jpeg|gif|svg<span class=\"o\">)<\/span><span class=\"nv\">$ <\/span><span class=\"o\">{<\/span>\n        try_files <span class=\"nv\">$uri<\/span> <span class=\"nv\">$uri<\/span>\/<span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># Deny access to config.<\/span>\n    location <span class=\"o\">=<\/span> \/wp-config.php <span class=\"o\">{<\/span>\n        deny all<span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># Deny access to htaccess.<\/span>\n    location ~ \/<span class=\"se\">\\.<\/span> <span class=\"o\">{<\/span> \n        deny all<span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># Directly allow access to \/wp-admin\/admin-ajax.php. This is necessary for<\/span>\n    <span class=\"c\"># WordPress to function on the admin side.<\/span>\n    location ~<span class=\"k\">*<\/span> ^\/wp-admin\/admin-ajax.php<span class=\"nv\">$ <\/span><span class=\"o\">{<\/span>\n        try_files <span class=\"nv\">$uri<\/span> <span class=\"o\">=<\/span>404<span class=\"p\">;<\/span>\n        fastcgi_intercept_errors on<span class=\"p\">;<\/span>\n        fastcgi_pass wordpress:9000<span class=\"p\">;<\/span>\n        fastcgi_index index.php<span class=\"p\">;<\/span>\n        fastcgi_param SCRIPT_FILENAME <span class=\"nv\">$document_root$fastcgi_script_name<\/span><span class=\"p\">;<\/span>\n        include fastcgi_params<span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n\n    <span class=\"c\"># Allow access to all other PHP files.<\/span>\n    location ~ <span class=\"se\">\\.<\/span>php<span class=\"nv\">$ <\/span><span class=\"o\">{<\/span>\n        try_files <span class=\"nv\">$uri<\/span> <span class=\"o\">=<\/span>404<span class=\"p\">;<\/span>\n        fastcgi_split_path_info ^<span class=\"o\">(<\/span>.+<span class=\"se\">\\.<\/span>php<span class=\"o\">)(<\/span>\/.+<span class=\"o\">)<\/span><span class=\"nv\">$;<\/span>\n        fastcgi_pass wordpress:9000<span class=\"p\">;<\/span>\n        fastcgi_index index.php<span class=\"p\">;<\/span>\n        include fastcgi_params<span class=\"p\">;<\/span>\n        fastcgi_param SCRIPT_FILENAME <span class=\"nv\">$document_root$fastcgi_script_name<\/span><span class=\"p\">;<\/span>\n        fastcgi_param PATH_INFO <span class=\"nv\">$fastcgi_path_info<\/span><span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n<span class=\"o\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This Nginx server specifically caters to the WordPress container. It listens for requests to port <code>80<\/code> and sends any requests for PHP files (include admin pages) to the WordPress container on port <code>9000<\/code>. This nginx server doesn't need to listen on port <code>443<\/code> because the proxy is handling the SSL authentication. Any incoming requests for files on the WordPress server will be routed through the proxy.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  WordPress Container\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p><span id=\"wordpress-wordpress-compose\"><strong>docker-compose.yml<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">###<\/span>\n  wordpress:\n    container_name: wp\n    image: wordpress-prod:4.8.1-php5.6-fpm\n    ports:\n      - <span class=\"s1\">'9000'<\/span>\n    volumes:\n      - .\/wordpress:\/var\/www\/html\n    environment:\n      - <span class=\"nv\">WORDPRESS_DB_NAME<\/span><span class=\"o\">=<\/span>wordpress\n      - <span class=\"nv\">WORDPRESS_TABLE_PREFIX<\/span><span class=\"o\">=<\/span>wpprefix_\n      - <span class=\"nv\">WORDPRESS_DB_HOST<\/span><span class=\"o\">=<\/span>mysql\n      - <span class=\"nv\">WORDPRESS_DB_PASSWORD<\/span><span class=\"o\">=<\/span>root\n    depends_on:\n      - mysql\n    restart: always\n    networks:\n      - nginxproxy_default\n<span class=\"c\">###<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The WordPress container exposes port <code>9000<\/code>. It has one volume mounted for preserving the WordPress file structure. It also has four environment variables that correspond to the connected database. Note that the <code>WORDPRESS_DB_HOST<\/code> name is the same as the <code>container_name<\/code> of the connected mysql container.<\/p>\n\n<p><span id=\"wordpress-wordpress-wp-config\"><strong>wordpress\/wp-config.php<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight php\"><code><span class=\"cp\">&lt;?php<\/span>\n<span class=\"c1\">###<\/span>\n<span class=\"c1\">\/\/ If we're behind a proxy server and using HTTPS, we need to alert Wordpress of that fact<\/span>\n<span class=\"c1\">\/\/ see also http:\/\/codex.wordpress.org\/Administration_Over_SSL#Using_a_Reverse_Proxy<\/span>\n<span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"k\">isset<\/span><span class=\"p\">(<\/span><span class=\"nv\">$_SERVER<\/span><span class=\"p\">[<\/span><span class=\"s1\">'HTTP_X_FORWARDED_PROTO'<\/span><span class=\"p\">])<\/span> <span class=\"o\">&amp;&amp;<\/span> <span class=\"nv\">$_SERVER<\/span><span class=\"p\">[<\/span><span class=\"s1\">'HTTP_X_FORWARDED_PROTO'<\/span><span class=\"p\">]<\/span> <span class=\"o\">===<\/span> <span class=\"s1\">'https'<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\n    <span class=\"nv\">$_SERVER<\/span><span class=\"p\">[<\/span><span class=\"s1\">'HTTPS'<\/span><span class=\"p\">]<\/span> <span class=\"o\">=<\/span> <span class=\"s1\">'on'<\/span><span class=\"p\">;<\/span>\n\n    <span class=\"c1\">\/\/ Force SSL in admin.<\/span>\n    <span class=\"nb\">define<\/span><span class=\"p\">(<\/span><span class=\"s1\">'FORCE_SSL_ADMIN'<\/span><span class=\"p\">,<\/span> <span class=\"kc\">true<\/span><span class=\"p\">);<\/span>\n<span class=\"p\">}<\/span>\n\n<span class=\"cm\">\/* That's all, stop editing! Happy blogging. *\/<\/span>\n<span class=\"c1\">###<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Ignoring the default configurations, these few lines can be placed before the \"stop editing!\" comment. These lines force WordPress to make internal requests using HTTPS, including on the admin side.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  Mariadb (MySQL) for WordPress Container\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p><span id=\"wordpress-mariadb-compose\"><strong>docker-compose.yml<\/strong><\/span><br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">###<\/span>\n  mysql:\n    container_name: mysql\n    image: mariadb\n    ports:\n      - <span class=\"s1\">'3306'<\/span>\n    volumes:\n      - .\/mysql:\/var\/lib\/mysql\n    environment:\n      - <span class=\"nv\">MYSQL_ROOT_PASSWORD<\/span><span class=\"o\">=<\/span>root\n    restart: always\n    networks:\n      - nginxproxy_default\n<span class=\"c\">###<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The difference between the Mariadb\/MySQL container and the others is the <code>mysql<\/code> volume being mounted, and the root-password environment variable. All the other configurations in this block are very similar to the previous containers.<br>\n<a><\/a><\/p>\n<h2>\n  \n  \n  Installing SSL Certificates with Certbot\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<p>In order to run the Certbot on the Nginx Proxy, the server has to be up and running first. Assuming that the certifications are not already in their local <code>.\/etc\/letsencrypt<\/code> directory, attempting to run the proxy now would fail. I discovered that in order for the proxy to run, several things need to happen.<\/p>\n\n<p>1) The <code>nginxproxy_default<\/code> network needs to be created. If the proxy has never been run before, then the network can't exist. Create the network by running the following in a terminal:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>docker network create nginxproxy_default\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>2) The WordPress containers need to be running before the proxy can start. First navigate to wherever the <code>docker-compose.yml<\/code> file is for the WordPress containers. Then run the following in a terminal:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>docker-compose up <span class=\"nt\">-d<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Note: The <code>-d<\/code> flag starts the containers in headless mode, meaning that you won't see any errors unless you use the command <code>docker-compose logs<\/code> or check the status of the containers via <code>docker ps -a<\/code>.<\/p>\n\n<p>3) The paths to the certificates need to be commented out before the proxy container can run the Certbot. In the proxy's <code>nginx.conf<\/code> I had to comment out the server block containing the <code>listen 443 ssl;<\/code> line and the server block handling redirection from HTTP to HTTPS. In their place, I temporarily entered this server block:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>server <span class=\"o\">{<\/span>\n    listen 80<span class=\"p\">;<\/span>\n    server_name example.com www.example.com<span class=\"p\">;<\/span>\n    location \/ <span class=\"o\">{<\/span>\n         proxy_pass       http:\/\/docker-wordpress<span class=\"p\">;<\/span>\n         proxy_redirect   off<span class=\"p\">;<\/span>\n         proxy_set_header Host <span class=\"nv\">$host<\/span><span class=\"p\">;<\/span>\n         proxy_set_header X-Real-IP <span class=\"nv\">$remote_addr<\/span><span class=\"p\">;<\/span>\n         proxy_set_header X-Forwarded-For <span class=\"nv\">$proxy_add_x_forwarded_for<\/span><span class=\"p\">;<\/span>\n         proxy_set_header X-Forwarded-Host <span class=\"nv\">$server_name<\/span><span class=\"p\">;<\/span>\n    <span class=\"o\">}<\/span>\n<span class=\"o\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Then, the Nginx Proxy can be built using the <code>docker-compose up --build<\/code> command. Docker-compose will then user the non-default startup command, which will run the <code>startup.sh<\/code> script in the container. I would recommend not using headless mode when building the proxy, that way you can be sure that the Certbot was able to register the certificates.<\/p>\n\n<p>Once the Certbot has successfully created the certificates, the temporary sever block added in step 3 above can be removed and the two other server blocks uncommented. After changing the configuration, the proxy will need to be reloaded with an <code>docker-compose down &amp;&amp; docker-compose up -d<\/code>.<\/p>\n\n<p>Doing a quick check with <code>docker ps -a<\/code> should display something along these lines:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code>CONTAINER ID  IMAGE                            COMMAND                 CREATED     STATUS        PORTS                                     NAMES\n62b3a673f1cb  nginx-proxy:0.0.2                <span class=\"s2\">\"nginx -g 'daemon ...\"<\/span>  3 days ago  Up 2 minutes  0.0.0.0:80-&gt;80\/tcp, 0.0.0.0:443-&gt;443\/tcp  nginx-proxy\nea4acq9cc868  nginx:latest                     <span class=\"s2\">\"nginx -g 'daemon ...\"<\/span>  4 days ago  Up 3 minutes  0.0.0.0:8081-&gt;80\/tcp                      nginx-wordpress\nc7badc76c834  wordpress-prod:4.8.1-php5.6-fpm  <span class=\"s2\">\"docker-entrypoint...\"<\/span>  4 days ago  Up 3 minutes  0.0.0.0:8082-&gt;9000\/tcp                    wp\n2ba321f4afdd  mariadb                          <span class=\"s2\">\"docker-entrypoint...\"<\/span>  4 days ago  Up 3 minutes  0.0.0.0:8083-&gt;3306\/tcp                    mysql\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If everything looks well, navigating to <code>https:\/\/example.com\/wp-admin<\/code> should display the good 'ole WordPress setup screen.<br>\n<a><\/a><\/p>\n\n<h2>\n  \n  \n  Further Reading\n<\/h2>\n\n<p>^ ToC<\/p>\n\n<ul>\n<li><a href=\"https:\/\/docs.docker.com\/compose\/compose-file\/\" rel=\"noopener noreferrer\">Docker Compose Documentation<\/a><\/li>\n<li><a href=\"https:\/\/docs.docker.com\/compose\/wordpress\/\" rel=\"noopener noreferrer\">Docker Quickstart with WordPress<\/a><\/li>\n<li><a href=\"https:\/\/hub.docker.com\/_\/nginx\/\" rel=\"noopener noreferrer\">Nginx Image on DockerHub<\/a><\/li>\n<li><a href=\"https:\/\/www.digitalocean.com\/community\/tutorials\/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-16-04\" rel=\"noopener noreferrer\">Enabling SSL on Nginx<\/a><\/li>\n<li><a href=\"https:\/\/www.digitalocean.com\/community\/tutorials\/docker-explained-how-to-containerize-and-use-nginx-as-a-proxy\" rel=\"noopener noreferrer\">Dockerizing Nginx as a Proxy<\/a><\/li>\n<\/ul>\n\n","category":["wordpress","docker","nginx","https"]},{"title":"Localizing Go to JavaScript","pubDate":"Tue, 12 Sep 2017 01:36:24 +0000","link":"https:\/\/dev.to\/foresthoffman\/localizing-go-to-javascript","guid":"https:\/\/dev.to\/foresthoffman\/localizing-go-to-javascript","description":"<p>While working a Go backend for a side-project, I implemented a custom templating system among other things. For my project, I needed to be able to pass nonce values down to my JavaScript. I realized that keeping the data in the front-end up-to-date with the backend would require a lot of leg work. In order to save time and effort, I built the <a href=\"https:\/\/github.com\/foresthoffman\/localize\" rel=\"noopener noreferrer\"><code>localize<\/code><\/a> package.<\/p>\n\n<p>This package takes a pre-defined Go data structure and recursively translates it to JavaScript primitives. The JavaScript that is spit back out can be used in just about any fashion, but it is designed to work best with the <code>html\/template<\/code> package. Since the <code>html\/template<\/code> package provides support for calling functions assigned to data passed to the <code>template.Template.Execute()<\/code> function, templates can fire off the localization process themselves. Once you have a template setup to utilize the localize package, it's a fire and forget situation. The best kind, in my opinion.<\/p>\n\n<p>Here's a simple example of the syntax:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight go\"><code><span class=\"k\">import<\/span><span class=\"p\">(<\/span>\n    <span class=\"s\">\"github.com\/foresthoffman\/localize\"<\/span>\n<span class=\"p\">)<\/span>\n\n<span class=\"k\">func<\/span> <span class=\"n\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n    <span class=\"c\">\/\/ Generates a new localization map with the provided data.<\/span>\n    <span class=\"n\">dataMap<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">localize<\/span><span class=\"o\">.<\/span><span class=\"n\">NewMap<\/span><span class=\"p\">(<\/span>\n        <span class=\"c\">\/\/ This will tell the localizer to assign the data to<\/span>\n        <span class=\"c\">\/\/ the \"_localData\" global JavaScript variable.<\/span>\n        <span class=\"s\">\"_localData\"<\/span><span class=\"p\">,<\/span>\n        <span class=\"n\">localize<\/span><span class=\"o\">.<\/span><span class=\"n\">Data<\/span><span class=\"p\">{<\/span>\n            <span class=\"s\">\"motd\"<\/span><span class=\"o\">:<\/span> <span class=\"s\">\"Hello world, welcome to a new day!\"<\/span><span class=\"p\">,<\/span>\n\n            <span class=\"c\">\/\/ \"nonce\" will hold an object with an element with<\/span>\n            <span class=\"c\">\/\/ the key, \"login\", and the value,<\/span>\n            <span class=\"c\">\/\/ \"LaKJIIjIOUhjbKHdBJHGkhg\"<\/span>\n            <span class=\"s\">\"nonce\"<\/span><span class=\"o\">:<\/span> <span class=\"k\">map<\/span><span class=\"p\">[<\/span><span class=\"kt\">string<\/span><span class=\"p\">]<\/span><span class=\"kt\">string<\/span><span class=\"p\">{<\/span>\n                <span class=\"s\">\"login\"<\/span><span class=\"o\">:<\/span> <span class=\"s\">\"LaKJIIjIOUhjbKHdBJHGkhg\"<\/span><span class=\"p\">,<\/span>\n            <span class=\"p\">},<\/span>\n        <span class=\"p\">},<\/span>\n    <span class=\"p\">)<\/span>\n\n    <span class=\"c\">\/\/ ...proper error handling, data manipulation, etc.<\/span>\n<span class=\"p\">}<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>EDIT (08\/11\/19): <a href=\"https:\/\/github.com\/foresthoffman\/localize\/blob\/master\/test\/template.go\" rel=\"noopener noreferrer\"><code>test\/template.go<\/code><\/a> provides a very handy example of using <code>localize<\/code> with an actual HTTP server.<\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Photo by <a href=\"https:\/\/unsplash.com\/@cmzw?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash\" rel=\"noopener noreferrer\">MW<\/a> on <a href=\"https:\/\/unsplash.com\/photos\/a-black-and-white-photo-of-a-spiral-design-K5T3UMuc114?utm_content=creditCopyText&amp;utm_medium=referral&amp;utm_source=unsplash\" rel=\"noopener noreferrer\">Unsplash<\/a><\/p>\n\n<p>That's it! :)<\/p>\n\n","category":["go","javascript","webdev","showdev"]},{"title":"Go formatting with rgblog","pubDate":"Wed, 09 Aug 2017 22:42:34 +0000","link":"https:\/\/dev.to\/foresthoffman\/go-formatting-with-rgblog","guid":"https:\/\/dev.to\/foresthoffman\/go-formatting-with-rgblog","description":"<p>One of my projects involves a lot of console output, and to make it readable, I added some color. Cyan for important messages, red for errors, and yellow for normal (everything is fine) logs.<\/p>\n\n<p>The short of it is: I wanted to do the same somewhere else, so I published it.<\/p>\n\n<p>If you're a Go developer and want some quick and easy functions to color things, check it out.<\/p>\n\n<p>Here's the <a href=\"https:\/\/github.com\/foresthoffman\/rgblog\" rel=\"noopener noreferrer\">GitHub repo<\/a> if you want to see the README. Or, you can just <code>go get github.com\/foresthoffman\/rgblog<\/code> and use it right away.<\/p>\n\n<p>If you'd like to see what the colors would look like, without downloading the package, you can run the following in your shell:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nb\">echo<\/span> <span class=\"nt\">-e<\/span> <span class=\"s2\">\"\n<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0;37mNormal Colors<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nBlack<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;30mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nRed<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;31mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nGreen<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;32mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nYellow<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;33mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nBlue<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;34mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nMagenta<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;35mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nCyan<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;36mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nWhite<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[0;37mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\n\n<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[1;37mBold\/Bright Colors<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nBlack<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;30mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nRed<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;31mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nGreen<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;32mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nYellow<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;33mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nBlue<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;34mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nMagenta<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;35mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nCyan<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;36mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\nWhite<\/span><span class=\"se\">\\t\\0<\/span><span class=\"s2\">33[1;37mThe quick brown fox jumped over the lazy dog<\/span><span class=\"se\">\\0<\/span><span class=\"s2\">33[0m\n\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>In mine it looks like this:<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/http%3A%2F%2Fforesthoffman.com%2Fwp-content%2Fuploads%2F2017%2F08%2Fansi-color-table.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/http%3A%2F%2Fforesthoffman.com%2Fwp-content%2Fuploads%2F2017%2F08%2Fansi-color-table.png\" width=\"800\" height=\"400\"><\/a><\/p>\n\n<p>Cheers!<\/p>\n\n<h2>\n  \n  \n  Credits\n<\/h2>\n\n<p>Cover image by <a href=\"https:\/\/unsplash.com\/@steve_j?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Steve Johnson<\/a> on <a href=\"https:\/\/unsplash.com\/s\/photos\/rainbow-letter?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText\" rel=\"noopener noreferrer\">Unsplash<\/a>! :D<\/p>\n\n","category":["go","showdev"]},{"title":"Running WordPress PHPUnit Tests With Docker","pubDate":"Tue, 11 Apr 2017 04:31:57 +0000","link":"https:\/\/dev.to\/foresthoffman\/running-wordpress-phpunit-tests-with-docker","guid":"https:\/\/dev.to\/foresthoffman\/running-wordpress-phpunit-tests-with-docker","description":"<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/http%3A%2F%2Fforest.stfhs.net%2Fforest%2Fwp-content%2Fuploads%2F2017%2F04%2Fdocker-wordpress-phpunit-820x461.3.jpg\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/http%3A%2F%2Fforest.stfhs.net%2Fforest%2Fwp-content%2Fuploads%2F2017%2F04%2Fdocker-wordpress-phpunit-820x461.3.jpg\" width=\"800\" height=\"400\"><\/a><\/p>\n\n<p>I recently moved my web development workspace from MAMP to LAMP with Docker. The transition was difficult, due to issues with Ruby and RVM on my host machine. Now that it is working, all is well!<\/p>\n\n<p>When I started tinkering with Docker, my goal was just to see if I could replicate my WordPress development environment. After I did that, I improved it a bit. Now, i've suitably compartmentalized each project. Each WordPress project has its own WordPress installation, MySQL server, error logs, plugin and theme directories.<\/p>\n\n<p>The only thing missing was being able to run unit tests!<\/p>\n\n<p>I vaguely remember having a frustrating time getting the Core WordPress tests to run with PHPUnit, the last time I set it up. This time was equally frustrating. I found an unfortunate lack of up-to-date and complete resources to deal with this process, so I made this article to fill the gap! I hope this helps someone avoid the same frustrations I experienced. If not, eh, at least I have the process documented for my future self.<\/p>\n\n<p>The error that I kept receiving upon running <code>$ phpunit<\/code> was some HTML content, and a load of errors. The essence of it being, <code>&lt;h1&gt;Error establishing a database connection&lt;\/h1&gt;<\/code>.<\/p>\n\n<p>Rather than including a testing database in each WordPress project, I created a MySQL container to be shared by all of my projects. I also configured the container to restart whenever the Docker daemon starts, so that it will always be available.<\/p>\n\n<h3>\n  \n  \n  Setting up WordPress Core Tests\n<\/h3>\n\n<p>In the same directory as my other WordPress projects (which contain individual <code>Dockerfile<\/code> and <code>docker-compose.yml<\/code> files), I created a <code>wordpress-develop<\/code> directory. To this new directory, I cloned the WordPress Core Tests repository.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>svn co https:\/\/develop.svn.wordpress.org\/trunk\/ wordpress-develop\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>I then prepared the <code>wp-tests-config.php<\/code> file within <code>wordpress-develop\/<\/code>. I found where the various global variables were being set at the bottom of the file, and updated them as follows:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight php\"><code><span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'DB_NAME'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'wordpress'<\/span> <span class=\"p\">);<\/span>\n<span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'DB_USER'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'root'<\/span> <span class=\"p\">);<\/span>\n<span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'DB_PASSWORD'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'root'<\/span> <span class=\"p\">);<\/span>\n<span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'DB_HOST'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'127.0.0.1:914'<\/span> <span class=\"p\">);<\/span>\n\n<span class=\"c1\"># ...<\/span>\n\n<span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'WP_TESTS_DOMAIN'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'localhost'<\/span> <span class=\"p\">);<\/span>\n<span class=\"nb\">define<\/span><span class=\"p\">(<\/span> <span class=\"s1\">'WP_TESTS_TITLE'<\/span><span class=\"p\">,<\/span> <span class=\"s1\">'Testing'<\/span> <span class=\"p\">);<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>This sets the database name to wordpress, the username and password to root, and domain to localhost. The issue that I ran into earlier was that the database host didn't recognize the value <code>'localhost:914'<\/code> as valid. This resulted in failed attemps to connect to the database, when I ran <code>phpunit<\/code> from within <code>wordpress-develop\/<\/code>. By exchanging <code>localhost<\/code> for a raw IP, <code>127.0.0.1<\/code> PHPUnit was able to connect. I actually found this fix recommended on a related <a href=\"http:\/\/wordpress.stackexchange.com\/questions\/203883\/database-connection-problem-with-phpunit-and-wordpress\" rel=\"nofollow noopener noreferrer\">Stack Exchange<\/a> comment.<\/p>\n\n<p>I chose the 914 port, because these <a href=\"https:\/\/en.wikipedia.org\/wiki\/List_of_TCP_and_UDP_port_numbers\" rel=\"nofollow noopener noreferrer\">UNIX port tables<\/a> indicate that the 914-986 range is officially unassigned. I needed a port that wouldn't get in the way, and 914 wasn't being used on my machine. I ran <code>$ curl http:\/\/127.0.0.1:914<\/code> just to check.<\/p>\n\n<h3>\n  \n  \n  Creating the MySQL daemon\n<\/h3>\n\n<p>Using a combination of <code>docker run<\/code> and the <code>restart<\/code> option, it is possible to create a MySQL container that will startup whenever the Docker daemon does. That includes when the host machine restarts.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>docker run <span class=\"nt\">--name<\/span> wptest-mysql <span class=\"nt\">--env<\/span> <span class=\"nv\">MYSQL_ROOT_PASSWORD<\/span><span class=\"o\">=<\/span>root <span class=\"nt\">--env<\/span> <span class=\"nv\">MYSQL_DATABASE<\/span><span class=\"o\">=<\/span>wordpress <span class=\"nt\">--env<\/span> <span class=\"nv\">MYSQL_USER<\/span><span class=\"o\">=<\/span>root <span class=\"nt\">--env<\/span> <span class=\"nv\">MYSQL_PASSWORD<\/span><span class=\"o\">=<\/span>root <span class=\"nt\">-p<\/span> 0.0.0.0:914:3306 <span class=\"nt\">--restart<\/span><span class=\"o\">=<\/span>always <span class=\"nt\">-d<\/span> mysql:5.7\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>The above command includes all of the data that we provided in the <code>wp-tests-config.php<\/code> file, and a bit more. The <code>name<\/code> option determines the name of the container; for easy identification I chose <code>wptest-mysql<\/code>, since <code>wordpress-develop-mysql<\/code> was a bit long. The <code>env<\/code> option is repeated several times, providing the container with the necessary environmental variables. The <code>p<\/code> option maps a port on the host machine to the container; so, <code>0.0.0.0:914<\/code> maps port <code>914<\/code> on the host to the port <code>3306<\/code> on the container. The <code>restart<\/code> option with the <code>always<\/code> value indicates that Docker should attempt to restart the container no matter how it exited. The <code>d<\/code> option indicates the image off of which to base the container. I used <a href=\"https:\/\/hub.docker.com\/_\/mysql\/\" rel=\"nofollow noopener noreferrer\">MySQL version 5.7<\/a>.<\/p>\n\n<p>At this point, running <code>$ docker ps -a<\/code> should reveal the new <code>wptest-mysql<\/code> container running with local port 914.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>docker ps <span class=\"nt\">-a<\/span>\n24d9f30b5286        mysql:5.7           <span class=\"s2\">\"docker-entrypoint...\"<\/span>   46 hours ago        Up 46 hours         0.0.0.0:914-&gt;3306\/tcp   wptest-mysql\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If it appears that the container is running properly, navigate to your <code>wordpress-develop<\/code> directory and run <code>$ phpunit<\/code>. If everything is running as it should, you'll see the WordPress test install begin to run, followed by the tests.<\/p>\n\n<h3>\n  \n  \n  Running WordPress tests elsewhere\n<\/h3>\n\n<p>Running the WordPress Core Tests is fantastic, but I really needed to run my own tests. In order to do this I took the path to my WordPress tests and assigned it to the <code>WP_TESTS_DIR<\/code> environmental variable in my <code>~\/.bash_profile<\/code>. This environmental variable is relied on by the project-based <code>tests\/bootstrap.php<\/code> files, which require the necessary WordPress files from the <code>wordpress-develop<\/code> directory.<\/p>\n\n<p>Copy-pasted from my <code>~\/.bash_profile<\/code>:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nb\">export <\/span><span class=\"nv\">DOCKER<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$HOME<\/span><span class=\"s2\">\/Documents\/Programming\/docker\"<\/span>\n<span class=\"nb\">export <\/span><span class=\"nv\">WP_DEVELOP_DIR<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$DOCKER<\/span><span class=\"s2\">\/wordpress\/wordpress-develop\"<\/span>\n<span class=\"nb\">export <\/span><span class=\"nv\">WP_TESTS_DIR<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$WP_DEVELOP_DIR<\/span><span class=\"s2\">\/tests\/phpunit\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>If you have no need for the other environment variables, you can stick it all together like so (obviously, use the correct path on your machine):<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nb\">export <\/span><span class=\"nv\">WP_TESTS_DIR<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$HOME<\/span><span class=\"s2\">\/Documents\/Programming\/docker\/wordpress\/wordpress-develop\/tests\/phpunit\"<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Run <code>$ . ~\/.bash_profile<\/code> to activate the changes. After that, running <code>$ phpunit<\/code> next to a project's respective <code>phpunit.xml<\/code> file and <code>tests<\/code> directory should run the local tests, as expected.<\/p>\n\n<p>That's it, woot!<\/p>\n\n","category":["wordpress","php","docker","testing"]},{"title":"Abstraction for the sake of Abstraction","pubDate":"Fri, 20 Jan 2017 03:32:20 +0000","link":"https:\/\/dev.to\/foresthoffman\/abstraction-for-the-sake-of-abstraction","guid":"https:\/\/dev.to\/foresthoffman\/abstraction-for-the-sake-of-abstraction","description":"<p>I wanted to talk a bit about task runners and automation. Personally, I've utilized NPM and GruntJS to automate repetitive tasks on multiple projects. That includes file compression, JavaScript\/PHP linting, Sass compilation, script\/style minification, amd running PHP\/JS unit tests. I've even used it to keep mirrored SVN and Git repos in sync.<\/p>\n\n<p>Having just started a new project, I went back to my previous projects for a refresher. I didn't want to reinvent the wheel, so I was going to scavenge whatever I needed from the old projects. Looking back, I used the following NPM packages:<\/p>\n\n<ol>\n<li> grunt<\/li>\n<li> grunt-cli<\/li>\n<li> grunt-concurrent<\/li>\n<li> grunt-contrib-concat<\/li>\n<li> grunt-contrib-cssmin<\/li>\n<li> grunt-contrib-jshint<\/li>\n<li> grunt-contrib-sass<\/li>\n<li> grunt-contrib-uglify<\/li>\n<li> grunt-contrib-watch<\/li>\n<li>grunt-exec<\/li>\n<li>grunt-mocha-test<\/li>\n<li>load-grunt-tasks<\/li>\n<li>time-grunt<\/li>\n<li>sinon<\/li>\n<li>jsdom<\/li>\n<li>mocha<\/li>\n<li>chai<\/li>\n<\/ol>\n\n<p>That didn't seem too bad, but then I looked at the Gruntfile. It was 195 lines long! One of my other projects used less packages, but the Gruntfile was 425 lines long...Yeah.<\/p>\n\n<p>I came to the sudden realization that I had put a ton of effort into utilizing Grunt for these projects. The reason being that I was learning to use NPM and Grunt and the like. Although it was useful for learning automation and how to write unit tests, it was otherwise unnecessary. Nearly all the tasks that I needed to accomplish could be accomplished without Grunt. I realized that I was adding abstraction for the sake of abstraction.<\/p>\n\n<p>When that bubble popped, I remembered the NPM has built-in functionality for running commands using <code>npm run-script<\/code>. Yup, I totally forgot about that!<\/p>\n\n<p>So, I thought, \"how could this be done with NPM only?\"<\/p>\n\n<p>Here's what I've come up with, for the above project, with comparisons between using Grunt and the default NPM run-script feature.<\/p>\n\n<h3>\n  \n  \n  Sass Compilation and Minification\n<\/h3>\n\n<p>Grunt setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight javascript\"><code><span class=\"nf\">require<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">load-grunt-tasks<\/span><span class=\"dl\">'<\/span> <span class=\"p\">)(<\/span> <span class=\"nx\">grunt<\/span> <span class=\"p\">);<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">initConfig<\/span><span class=\"p\">({<\/span>\n    <span class=\"na\">paths<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">sass<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">dir<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">styles<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">files<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.sass.dir %&gt;\/**\/*.scss<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">},<\/span>\n        <span class=\"na\">css<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">dir<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">public\/styles<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">},<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"na\">sass<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">options<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">style<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">expanded<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">},<\/span>\n        <span class=\"na\">dist<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">files<\/span><span class=\"p\">:<\/span> <span class=\"p\">[{<\/span>\n                <span class=\"na\">expand<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n                <span class=\"na\">cwd<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.sass.dir %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"na\">src<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span><span class=\"dl\">'<\/span><span class=\"s1\">**\/*.scss<\/span><span class=\"dl\">'<\/span><span class=\"p\">],<\/span>\n                <span class=\"na\">dest<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.css.dir %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"na\">ext<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">.css<\/span><span class=\"dl\">'<\/span>\n            <span class=\"p\">}]<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">});<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">registerTask<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">scss<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">sass<\/span><span class=\"dl\">'<\/span> <span class=\"p\">);<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Grunt usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>grunt scss\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"scripts\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"sass\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"sass --scss -t compressed styles\/*.scss public\/styles\/style.min.css\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>npm run sass\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h3>\n  \n  \n  JavaScript Linting, Compression, and Minification\n<\/h3>\n\n<p>Grunt setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight javascript\"><code><span class=\"nf\">require<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">load-grunt-tasks<\/span><span class=\"dl\">'<\/span> <span class=\"p\">)(<\/span> <span class=\"nx\">grunt<\/span> <span class=\"p\">);<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">initConfig<\/span><span class=\"p\">({<\/span>\n    <span class=\"na\">paths<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">js<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">source<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">scripts\/*.js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">public_dir<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">public\/scripts\/<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">public_dest<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.public_dir %&gt;main.js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">public_ugly<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.public_dir %&gt;main.min.js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">files<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.source %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">Gruntfile.js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">test\/**\/*.js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">!test\/utils\/**\/*.js<\/span><span class=\"dl\">'<\/span>\n            <span class=\"p\">]<\/span>\n        <span class=\"p\">},<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"na\">concat<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">js<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">src<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.source %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">dest<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.public_dest %&gt;<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"na\">uglify<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">options<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">mangle<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n                <span class=\"na\">except<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span><span class=\"dl\">'<\/span><span class=\"s1\">jQuery<\/span><span class=\"dl\">'<\/span><span class=\"p\">]<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">},<\/span>\n        <span class=\"na\">target<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">files<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.public_ugly %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"p\">[<\/span><span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.public_dest %&gt;<\/span><span class=\"dl\">'<\/span><span class=\"p\">]<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"na\">jshint<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">options<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">curly<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">eqeqeq<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">browser<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">devel<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">undef<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">unused<\/span><span class=\"p\">:<\/span> <span class=\"kc\">false<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">mocha<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n            <span class=\"na\">globals<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">jQuery<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">module<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">require<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">window<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span><span class=\"p\">,<\/span>\n                <span class=\"dl\">'<\/span><span class=\"s1\">global<\/span><span class=\"dl\">'<\/span><span class=\"p\">:<\/span> <span class=\"kc\">true<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">},<\/span>\n        <span class=\"na\">dist<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.js.files %&gt;<\/span><span class=\"dl\">'<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">});<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">registerTask<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">js<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"p\">[<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">jshint<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">uglify<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">concat<\/span><span class=\"dl\">'<\/span> <span class=\"p\">]<\/span> <span class=\"p\">);<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Grunt usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>grunt js\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"scripts\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"js\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"jshint scripts\/*.js test\/*.test.js &amp;&amp; uglifyjs scripts\/*.js -cmo public\/scripts\/word_search.min.js\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>npm run js\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h3>\n  \n  \n  Mocha Unit Tests\n<\/h3>\n\n<p>Grunt setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight javascript\"><code><span class=\"nf\">require<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">load-grunt-tasks<\/span><span class=\"dl\">'<\/span> <span class=\"p\">)(<\/span> <span class=\"nx\">grunt<\/span> <span class=\"p\">);<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">initConfig<\/span><span class=\"p\">({<\/span>\n    <span class=\"na\">paths<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">test<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">files<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">test\/**\/*.test.js<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">},<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"na\">mochaTest<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">test<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">options<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n                <span class=\"na\">reporter<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">spec<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span>\n                <span class=\"na\">require<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">test\/utils\/jsdom-config.js<\/span><span class=\"dl\">'<\/span>\n            <span class=\"p\">},<\/span>\n            <span class=\"na\">src<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.test.files %&gt;<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">}<\/span>\n<span class=\"p\">});<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">registerTask<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">mocha<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">mochaTest<\/span><span class=\"dl\">'<\/span> <span class=\"p\">);<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Grunt usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>grunt mocha\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"scripts\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"test\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"mocha -R spec -r test\/utils\/jsdom-config.js test\/*.test.js\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM usage:<\/p>\n\n<p>Since <code>npm run-script<\/code> accepts <code>test<\/code> as a predefined task, the command is even shorter!<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>npm <span class=\"nb\">test<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h3>\n  \n  \n  Deployment\n<\/h3>\n\n<p>My source directory is right next to my distribution directory. The Grunt task for deployment requires the <code>grunt-exec<\/code> dependency, which allows running command line expressions. The deploy task runs all the linting, uglification, and sass compilation before copying the <code>public\/<\/code> directory to the distribution directory. For the sake of brevity, I'm not going to list all the tasks out again, just the key ones.<\/p>\n\n<p>Grunt setup:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight javascript\"><code><span class=\"nf\">require<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">load-grunt-tasks<\/span><span class=\"dl\">'<\/span> <span class=\"p\">)(<\/span> <span class=\"nx\">grunt<\/span> <span class=\"p\">);<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">initConfig<\/span><span class=\"p\">({<\/span>\n    <span class=\"na\">pkg<\/span><span class=\"p\">:<\/span> <span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nx\">file<\/span><span class=\"p\">.<\/span><span class=\"nf\">readJSON<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">package.json<\/span><span class=\"dl\">'<\/span> <span class=\"p\">),<\/span>\n    <span class=\"na\">paths<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">host<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">dir<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">..\/foresthoffman.github.io\/&lt;%= pkg.name %&gt;\/<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">},<\/span>\n        <span class=\"na\">source<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">dir<\/span><span class=\"p\">:<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">public\/<\/span><span class=\"dl\">'<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">},<\/span>\n    <span class=\"cm\">\/* Other task definitions *\/<\/span>\n    <span class=\"na\">exec<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n        <span class=\"na\">copy<\/span><span class=\"p\">:<\/span> <span class=\"p\">{<\/span>\n            <span class=\"na\">cmd<\/span><span class=\"p\">:<\/span> <span class=\"nf\">function <\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\n                <span class=\"kd\">var<\/span> <span class=\"nx\">host_dir_path<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nx\">template<\/span><span class=\"p\">.<\/span><span class=\"nf\">process<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.host.dir %&gt;<\/span><span class=\"dl\">'<\/span> <span class=\"p\">);<\/span>\n                <span class=\"kd\">var<\/span> <span class=\"nx\">source_dir_path<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nx\">template<\/span><span class=\"p\">.<\/span><span class=\"nf\">process<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">&lt;%= paths.source.dir %&gt;<\/span><span class=\"dl\">'<\/span> <span class=\"p\">);<\/span>\n\n                <span class=\"kd\">var<\/span> <span class=\"nx\">copy_command<\/span> <span class=\"o\">=<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">cp -r <\/span><span class=\"dl\">'<\/span> <span class=\"o\">+<\/span> <span class=\"nx\">source_dir_path<\/span> <span class=\"o\">+<\/span> <span class=\"dl\">'<\/span><span class=\"s1\"> <\/span><span class=\"dl\">'<\/span> <span class=\"o\">+<\/span> <span class=\"nx\">host_dir_path<\/span><span class=\"p\">;<\/span>\n\n                <span class=\"k\">return<\/span> <span class=\"nx\">copy_command<\/span><span class=\"p\">;<\/span>\n            <span class=\"p\">}<\/span>\n        <span class=\"p\">}<\/span>\n    <span class=\"p\">},<\/span>\n<span class=\"p\">});<\/span>\n\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">registerTask<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">build<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"p\">[<\/span>\n    <span class=\"dl\">'<\/span><span class=\"s1\">jshint<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">sass<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">cssmin<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">mochaTest<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">concat<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">uglify<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> \n    <span class=\"dl\">'<\/span><span class=\"s1\">exec:zip<\/span><span class=\"dl\">'<\/span>\n<span class=\"p\">]);<\/span>\n<span class=\"nx\">grunt<\/span><span class=\"p\">.<\/span><span class=\"nf\">registerTask<\/span><span class=\"p\">(<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">deploy<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"p\">[<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">build<\/span><span class=\"dl\">'<\/span><span class=\"p\">,<\/span> <span class=\"dl\">'<\/span><span class=\"s1\">exec:copy<\/span><span class=\"dl\">'<\/span> <span class=\"p\">]<\/span> <span class=\"p\">);<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Grunt usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>grunt deploy\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM setup:<\/p>\n\n<p>I made a simple bash script, so that I could completely drop Grunt for this task.<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">##<\/span>\n<span class=\"c\"># cp_public<\/span>\n<span class=\"c\">##<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nv\">$# <\/span><span class=\"o\">==<\/span> 2 <span class=\"o\">]<\/span> <span class=\"o\">||<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-d<\/span> <span class=\"nv\">$1<\/span> <span class=\"o\">]<\/span> <span class=\"o\">||<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-d<\/span> <span class=\"nv\">$2<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n    <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"cp_public \/path\/to\/source \/path\/to\/dist\"<\/span>\n    <span class=\"nb\">exit\n<\/span><span class=\"k\">fi<\/span>\n\n<span class=\"c\"># the name property line from the package.json file in the current directory<\/span>\n<span class=\"nv\">name_line<\/span><span class=\"o\">=<\/span><span class=\"si\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nt\">-Ei<\/span> <span class=\"s2\">\"^<\/span><span class=\"se\">\\s<\/span><span class=\"s2\">*<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">name<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\s\\\"<\/span><span class=\"s2\">.*<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,$\"<\/span> package.json | <span class=\"nb\">grep<\/span> <span class=\"nt\">-oEi<\/span> <span class=\"s2\">\"[^<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,]+\"<\/span><span class=\"si\">)<\/span>\n\n<span class=\"c\"># the package name by itself<\/span>\n<span class=\"nv\">name<\/span><span class=\"o\">=<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"nv\">$name_line<\/span> | <span class=\"nb\">cut<\/span> <span class=\"nt\">-d<\/span> <span class=\"s2\">\" \"<\/span> <span class=\"nt\">-f<\/span> 3<span class=\"si\">)<\/span>\n\n<span class=\"nv\">path_reg<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"se\">\\\/<\/span><span class=\"s2\">\"<\/span>\n\n<span class=\"nv\">source_path<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"k\">${<\/span><span class=\"nv\">1<\/span><span class=\"p\">%<\/span><span class=\"nv\">$path_reg<\/span><span class=\"k\">}<\/span><span class=\"si\">)<\/span><span class=\"s2\">\/\"<\/span>\n<span class=\"nv\">dist_path<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"k\">${<\/span><span class=\"nv\">2<\/span><span class=\"p\">%<\/span><span class=\"nv\">$path_reg<\/span><span class=\"k\">}<\/span><span class=\"si\">)<\/span><span class=\"s2\">\/<\/span><span class=\"nv\">$name<\/span><span class=\"s2\">\"<\/span>\n\n<span class=\"nb\">cp<\/span> <span class=\"nt\">-r<\/span> <span class=\"nv\">$source_path<\/span> <span class=\"nv\">$dist_path<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"scripts\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"deploy\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\".\/cp_public public\/ ..\/foresthoffman.github.io\/\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>NPM usage:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"nv\">$ <\/span>npm run deploy\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h3>\n  \n  \n  Watchers\n<\/h3>\n\n<p>Rather than using Grunt to run concurrent file watchers I opted to use the <a href=\"https:\/\/www.npmjs.com\/package\/npm-watch\" rel=\"noopener noreferrer\"><code>npm-watch<\/code><\/a> package. This allows me to indicate X number of scripts (from the scripts property of my <code>package.json<\/code> file) and the files that should trigger the scripts while the watcher is active. I only really needed to have the watcher handle js files changing, since the sass command has it's own <code>--watch<\/code> argument.<\/p>\n\n<p>Then the configuration for the watch statement looks like this:<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"watch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"js\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"scripts\/*.js\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"err\">,<\/span><span class=\"w\">\n<\/span><span class=\"nl\">\"script\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"js\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"uglifyjs scripts\/*.js -cmo public\/scripts\/main.min.js\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"watch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"npm-watch\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>Which can then be run via, <code>npm run watch<\/code>.<\/p>\n\n<p>While writing this I actually implemented the changes that I've mentioned here. I've deleted my <code>Gruntfile<\/code> and am left with the configurations in my <code>package.json<\/code> and my new bash script in <code>cp_public<\/code>. Everything is working as intended. Woot!<\/p>\n\n<p><code>package.json<\/code>...<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight json\"><code><span class=\"nl\">\"devDependencies\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"chai\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"^3.4.1\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"jsdom\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"^7.2.2\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"mocha\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"^2.3.4\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"npm-watch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"^0.1.7\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"sinon\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"^1.17.2\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><span class=\"err\">...<\/span><span class=\"w\">\n<\/span><span class=\"nl\">\"watch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"js\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"scripts\/*.js\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"err\">,<\/span><span class=\"w\">\n<\/span><span class=\"nl\">\"scripts\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"p\">{<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"deploy\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\".\/cp_public public\/ ..\/foresthoffman.github.io\/ || true\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"sass\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"sass --scss -t compressed styles\/*.scss public\/styles\/style.min.css\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"sassWatch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"sass --watch --scss -t compressed styles\/style.scss:public\/styles\/style.min.css\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"js\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"uglifyjs scripts\/*.js -cmo public\/scripts\/main.min.js\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"test\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"mocha -R spec -r test\/utils\/jsdom-config.js test\/*.test.js || true\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"testWatch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"mocha -w -R spec -r test\/utils\/jsdom-config.js test\/*.test.js || true\"<\/span><span class=\"p\">,<\/span><span class=\"w\">\n    <\/span><span class=\"nl\">\"watch\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"npm-watch\"<\/span><span class=\"w\">\n<\/span><span class=\"p\">}<\/span><span class=\"w\">\n<\/span><\/code><\/pre>\n\n<\/div>\n\n\n\n<p>and <code>cp_public<\/code>...<br>\n<\/p>\n\n<div class=\"highlight js-code-highlight\">\n<pre class=\"highlight shell\"><code><span class=\"c\">#!\/bin\/bash<\/span>\n\n<span class=\"c\">##<\/span>\n<span class=\"c\"># cp_public<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Copies the source directory to the target directory.<\/span>\n<span class=\"c\">#<\/span>\n<span class=\"c\"># Usage: cp_public \/path\/to\/source \/path\/to\/dist<\/span>\n<span class=\"c\"># <\/span>\n<span class=\"c\"># The package.json file, from which the package name is collected is relative to where this script<\/span>\n<span class=\"c\"># is executed. It uses the current directory. That is why this script should be placed next to the<\/span>\n<span class=\"c\"># package.json file in the project's hierarchy.<\/span>\n<span class=\"c\">##<\/span>\n\n<span class=\"k\">if<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nv\">$# <\/span><span class=\"o\">==<\/span> 2 <span class=\"o\">]<\/span> <span class=\"o\">||<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-d<\/span> <span class=\"nv\">$1<\/span> <span class=\"o\">]<\/span> <span class=\"o\">||<\/span> <span class=\"o\">[<\/span> <span class=\"o\">!<\/span> <span class=\"nt\">-d<\/span> <span class=\"nv\">$2<\/span> <span class=\"o\">]<\/span><span class=\"p\">;<\/span> <span class=\"k\">then\n    <\/span><span class=\"nb\">echo<\/span> <span class=\"s2\">\"cp_public \/path\/to\/source \/path\/to\/dist\"<\/span>\n    <span class=\"nb\">exit\n<\/span><span class=\"k\">fi<\/span>\n\n<span class=\"c\"># the name property line from the package.json file in the current directory<\/span>\n<span class=\"nv\">name_line<\/span><span class=\"o\">=<\/span><span class=\"si\">$(<\/span><span class=\"nb\">grep<\/span> <span class=\"nt\">-Ei<\/span> <span class=\"s2\">\"^<\/span><span class=\"se\">\\s<\/span><span class=\"s2\">*<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">name<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\s\\\"<\/span><span class=\"s2\">.*<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,$\"<\/span> package.json | <span class=\"nb\">grep<\/span> <span class=\"nt\">-oEi<\/span> <span class=\"s2\">\"[^<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,]+\"<\/span><span class=\"si\">)<\/span>\n\n<span class=\"c\"># the package name by itself<\/span>\n<span class=\"nv\">name<\/span><span class=\"o\">=<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"nv\">$name_line<\/span> | <span class=\"nb\">cut<\/span> <span class=\"nt\">-d<\/span> <span class=\"s2\">\" \"<\/span> <span class=\"nt\">-f<\/span> 3<span class=\"si\">)<\/span>\n\n<span class=\"nv\">path_reg<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"se\">\\\/<\/span><span class=\"s2\">\"<\/span>\n\n<span class=\"nv\">source_path<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"k\">${<\/span><span class=\"nv\">1<\/span><span class=\"p\">%<\/span><span class=\"nv\">$path_reg<\/span><span class=\"k\">}<\/span><span class=\"si\">)<\/span><span class=\"s2\">\/\"<\/span>\n<span class=\"nv\">dist_path<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"si\">$(<\/span><span class=\"nb\">echo<\/span> <span class=\"k\">${<\/span><span class=\"nv\">2<\/span><span class=\"p\">%<\/span><span class=\"nv\">$path_reg<\/span><span class=\"k\">}<\/span><span class=\"si\">)<\/span><span class=\"s2\">\/<\/span><span class=\"nv\">$name<\/span><span class=\"s2\">\"<\/span>\n\n<span class=\"nb\">cp<\/span> <span class=\"nt\">-r<\/span> <span class=\"nv\">$source_path<\/span> <span class=\"nv\">$dist_path<\/span>\n<\/code><\/pre>\n\n<\/div>\n\n\n\n<h3>\n  \n  \n  That's all folks!\n<\/h3>\n\n<p>I will admit that there aren't many good packages for concurrency without having to take on a lot more dependencies. In that way using Grunt (or another runner) is beneficial. Personally, I can no longer justify adding that complexity for what seems like a relatively minute benefit.<\/p>\n\n<p>I like tooling and automation. I think writing unit tests is a freaking fantastic and very rewarding endeavor. However, I feel that there is a point at which the tools we use to automate rudimentary tasks add too much complexity. I'm seeing more and more of this everyday, thanks to the speed at which the web development ecosystem is progressing.<\/p>\n\n<p>Anyone else feel the same? Anyone stuck in situation where they can't actually justify removing these kinds of dependencies? Anyone still a huge fan of runners, and will use them in future projects?<\/p>\n\n<p>Obligatory xkcd comic plug:<\/p>\n\n<p><a href=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fov21suzpstmesyo4lmbb.png\" class=\"article-body-image-wrapper\"><img src=\"https:\/\/media2.dev.to\/dynamic\/image\/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto\/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fov21suzpstmesyo4lmbb.png\" alt=\"XKCD 927\" width=\"500\" height=\"283\"><\/a><\/p>\n\n","category":["automation","testing","javascript"]}]}}