[{"content":"Sortable is a very nice JavaScript library for making HTML lists sortable via drag-and-drop, and it’s got a very clear and easy to use API. Unfortunately, I spent a very long time struggling to integrate it into my app, so now that I’ve figured it out I might as well collect my findings into the blog post I wish I could have referenced instead.\nIt was quite hard to get working for two reasons. First, I’m using Svelte (version 5), which is a UI framework that very much wants to solely manage the HTML. Second, I’m using Tauri (version 2.0), a framework for building cross-platform applications; this adds complexity beyond a web app.\nBefore getting started, I searched for resources that would tell me exactly how to integrate all three things together. Sadly, there were none. The closest I got, and what I started with, was this “Svelte 5 and SortableJS” blog post. Unfortunately, there were still several issues to iron out thanks to the Tauri variable. Ultimately, I ended up with an attachment-based solution for an even more ~modern~ Svelte flavor1.\nWriting the attachment The attachment itself is very simple. I created a function that takes Sortable options and returns an attachment that creates a Sortable with the provided options.\nimport Sortable from "sortablejs"; import type { Attachment } from "svelte\/attachments"; export const sortableList = (options: Sortable.Options): Attachment => { return (element: Element) => { const sortable = Sortable.create(element as HTMLElement, options); return () => { sortable.destroy(); }; }; }; Similarily to the “Svelte 5 and SortableJS” blog post, I provided a helper function for reordering, though I chose to return null if the order remained the same to avoid unnecessary updates.\nexport function reorder<T>( array: T[], evt: Sortable.SortableEvent, ): T[] | null { const newArray = [...$state.snapshot(array)] as T[]; const { oldIndex, newIndex } = evt; if (oldIndex === undefined || newIndex === undefined) { return null; } if (newIndex === oldIndex) { return null; } const target = newArray[oldIndex]; const increment = newIndex < oldIndex ? -1 : 1; for (let k = oldIndex; k !== newIndex; k += increment) { newArray[k] = newArray[k + increment]; } newArray[newIndex] = target; return newArray; } The exact parameters in your Sortable options will vary, but the most important bit is to actually reorder your items!\nimport { reorder } from "$lib\/sortable.svelte"; const sortableOptions = { onUpdate(event: any) { const newList = reorder(groups, event); if (newList == null) { return; } \/\/ Placeholder for updating the order of items. updateOrdering(newList); } } Then, we need to create the list and attach sortableList. You can generate list items dynamically with @each, but I found that it’s crucial to wrap the list in a #key block to recreate the list and Sortable every time the items are reordered. If you do not do this, the item ordering will get out of sync between Svelte and Sortable and your list will not display in the correct order.\n<script> import { sortableList } from "$lib\/sortable.svelte"; \/\/ Your list of things const items = [ { id: "item-id", text: "item-text", } ]; <\/script> {#key items} <ul {@attach sortableList(sortableOptions)}> {#each items as item (item.id)} <li>{item.text}<\/li> {\/each} <\/ul> {\/key} Making drag and drop work From the Svelte side, everything looks great now. In fact, it will work perfectly if you throw it in a REPL2, which is maddening because locally, in your Tauri app, you will not be able to drag any list items!\nThis is the first Tauri-introduced issue I encountered. The application window’s drag-and-drop functionality was preventing the drag-and-drop event from registering in the webview. To avoid this, I had to add dragDropEnabled: false to my tauri.conf.json:\n{ "app": { "windows": [ { "dragDropEnabled": false } ], } } Many thanks to this Reddit post for providing the hint I needed to find this fix!\nAI to the rescue At this point, I had a working sortable list with one big problem. While I could drag one item to reorder it, I could not consistently reorder items consecutively. The vast majority of the time, after moving one item I would have to click anywhere else on the page to be able to move another item. If I did not click elsewhere, clicking a sortable item would not start a drag event, and I would end up highlighting the text instead.\nAnd yes, “vast majority of the time” means that on occassion I would be able to reorder items consecutively. So at first glance, this looks like a race condition along the lines of the list being rendered before the sortable functionality is available. However, that hypothesis is somewhat invalidated by the fact that when you are unable to reorder items consecutively, you cannot move the second item no matter how long you wait after moving the first item.\nIn any event, I did the obvious thing which is to add a bunch of print statements and then try various things to ascertain the order of events. When that failed, I had a few more hypotheses, but with my limited JavaScript knowledge I didn’t know how to test them out.\nEnter AI. At work, I’m a pretty big fan of Claude Code because it doesn’t interrupt my stream of thought by injecting slop when I hit Tab to, literally, insert spaces. However, it’s a lot easier to use a tool when you don’t have to pay for it! Luckily, Sourcegraph makes a similar terminal-based agent that has a free, ad-supported mode: Amp. The ads are not bothersome at all and the agent worked quite well. With Amp, I was able to iterate quickly on my hypotheses.\nThe most promising hypothesis I had was that dragging the first item hijacked pointer events in a way that required a “reset” (i.e. by clicking elsewhere). I tried a few related fixes, such as injecting pointerup events after destroying the Sortable, to no avail. Along the way, Amp suggested various code changes, one of which led to me printing event.originalEvent from the event received by onUpdate.\nI noticed that the original event is actually a DragEvent. I’m no frontend developer, but this seems a little different from a pointer-based event. I responded to Amp with this:\nDispatching pointerup doesn’t help. Even though it’s logged, I still need to click again to reactivate the sortable. I also noticed that event.originalEvent is a dragEvent with preventDefault: true.\nWith that, it produced an interesting code block:\nconst sortable = Sortable.create(el, { ...options, forceFallback: true, \/\/ Force pointer events instead of drag events }); From Sortable docs, forceFallback changes how drag-and-drop works to be compatible with non-HTML5 browsers, even on HTML5 browsers. Setting this option completely fixes the issue!\nUnfortunately, what I can’t tell you is why this works, and after bashing my head on this for far too long I lack the inclination to do further research. Amp reported that “HTML5 Drag and Drop has known quirks, especially in Electron-like environments like Tauri”, but when pressed to cite sources it deflected with “I was making educated guesses based on the symptoms you showed, not citing specific sources. I don’t have concrete sources to back up those claims.” Perhaps someone else can write an explanation as to why this is the fix \ud83d\ude05\nNonetheless, this is a solution I may not have been able to come up with without AI. There is basically no chance I would’ve read the Sortable docs carefully enough to notice the forceFallback parameter, especially because it wasn’t an option I was looking for.\nOn the other hand, AI alone couldn’t solve this either. Amp was only able to suggest a solution because I provided a prompt based on a hunch. Amusingly, once this is published, both humans and AI will be able to successfully use Sortable with Svelte and Tauri!\nTo be honest, my AI assistant told me to do that while debugging. It didn’t fix the immediate issue, so you probably don’t need to use @attach instead of $effect, but I’m not going back to rewrite it. ↩︎\nYes, that’s how I first tried to debug. ↩︎\n","permalink":"https:\/\/lynshi.github.io\/posts\/sortable_svelte_and_tauri\/","summary":"Sortable is a very nice JavaScript library for making HTML lists sortable via drag-and-drop, and it’s got a very clear and easy to use API. Unfortunately, I spent a very long time struggling to integrate it into my app, so now that I’ve figured it out I might as well collect my findings into the blog post I wish I could have referenced instead.\nIt was quite hard to get working for two reasons.","title":"Sortable + Svelte + Tauri: Getting a sortable list right in a cross-platform app"},{"content":"Last time we introduced the Gossip Glomers challenge from Fly.io and discussed our approach to Challenge #2: Unique ID Generation.\nThis time, we’ll talk about the first three parts of Challenge #3: Broadcast. Parts D and E are saved for a separate post as they’re a bit more involved.\nThe overall theme of Challenge 3 is to build a broacast system to propagate messages to all nodes1. We iteratively build up our system, from a single-node cluster that simply stores and returns received messages, to a multi-node cluster that shares received messages, to a fault-tolerant multi-node cluster that can operate even during network partitions by Part C (D and E are about efficiency).\nAs before, my code is on GitHub at lynshi\/gossip-glomers under internal\/broadcast.\n3a: Single-Node Broadcast We start off implementing only one node to ensure that we receive and save broadcasted messages correctly. There are three types of messages our node needs to handle:\nbroadcast: Store the value received. read: Return all values received by the node. topology: Receive information about neighbors. To store values, we’ll use an array of integers. Since we may have multiple handlers running concurrently, we’ll use a channel to synchronize access. This channel will have a buffer size of 1 and we’ll initialize it with an empty integer array. To add to the array, the broadcast handler will read the array to consume the single copy, append to it, and write it back to the channel. Similarly, the read handler will read the array from the channel and return a copy in the response.\nIn order to make it easy to have a distinct implementation for each part, we’ll wrap the data structures used in a struct. For example, for Part A we’ll define SingleNodeNode.\ntype SingleNodeNode struct { mn *maelstrom.Node messages chan []int } func NewSingleNodeNode(ctx context.Context, n *maelstrom.Node) { messages = make(chan []int, 1) messages <- make([]int, 0, 1) n := SingleNodeNode{ mn: mn, messages: messages, } go func() { <-ctx.Done() close(messages) }() n.addBroadcastHandle() n.addReadHandle() n.addTopologyHandle() return &n } func (n *SingleNodeNode) broadcastSingleNodeBuilder() maelstrom.HandlerFunc { broadcast := func(req maelstrom.Message) error { \/\/ ... message, _ := getMessage(body) msgs := <-n.messages msgs = append(msgs, int(message)) n.messages <- msgs \/\/ ... } return broadcast } func (n *SingleNodeNode) readBuilder() maelstrom.HandlerFunc { read := func(req maelstrom.Message) error { msgs := <-n.messages \/\/ Now that we have a local copy, we can immediately return it to the channel so that other \/\/ goroutines are unblocked. n.messages <- msgs resp := make(map[string]any) resp["type"] = "read_ok" resp["messages"] = msgs return n.mn.Reply(req, resp) } return read } My code for Part A is at internal\/broadcast\/3a.go.\nA note on topology The topology message type is odd. The problem statement says that we can ignore the provided neighbors and build our own topology from Maelstrom’s list of all nodes, as all nodes can communicate with each other. At first, I was confused about this as there doesn’t seem to be a point of this message then, and based on a glance at community.fly.io I wasn’t the only one2. However, someone explained that “the topology is just a way to logically arrange nodes” and that Maelstrom allows you to select a topology, so my conclusion is that the topology can be interpreted as a recommendation for inter-node communication3 and also provides consistency with Maelstrom’s problem formulation, which causes the Maelstrom controller to send a topology message when starting up the nodes. Ultimately, I chose to ignore this message for all sections of this challenge as I constructed my own topology later on.\n3b: Multi-Node Broadcast In Part B, we introduce multiple nodes, and upon receiving a broadcast message a node must distribute that message to all other nodes within a few seconds. Because all messages are unique, I decided to store the messages in a map[int]interface{} instead so that saved messages are automatically deduplicated 4. Similarly to previous section, we initialize a MultiNodeNode by adding an empty map to the messages channel.\ntype MultiNodeNode struct { mn *maelstrom.Node messages chan map[int]interface{} } func NewMultiNodeNode(ctx context.Context, mn *maelstrom.Node) *MultiNodeNode { messages := make(chan map[int]interface{}, 1) messages <- make(map[int]interface{}) n := &MultiNodeNode{ mn: mn, messages: messages, } \/\/ ... return &n } Since we have to forward messages received from the controller to other nodes as soon as possible, upon receipt of a broadcast message that did not originate from another node, the node initiates a goroutine to send the message to every other node. We use the Maelstrom-provided method Send, which is a fire-and-forget method that sends a message to the specified destination, as there aren’t network failures in this scenario.\nbroadcast := func(req maelstrom.Message) error { \/\/ ... \/\/ Only forward if the message did not come from another node. if strings.HasPrefix(req.Src, "n") { return nil } go func() { for _, neighbor := range n.mn.NodeIDs() { req := make(map[string]any) req["type"] = "broadcast" req["message"] = message go n.mn.Send(neighbor, req) } }() resp := make(map[string]any) resp["type"] = "broadcast_ok" return n.mn.Reply(req, resp) } The read handler is very similar to before, except we now must convert the map into an array.\nread := func(req maelstrom.Message) error { messages := <-n.messages n.messages <- messages resp := make(map[string]any) resp["type"] = "read_ok" resp_messages := make([]int, 0, len(messages)) for v, _ := range messages { resp_messages = append(resp_messages, v) } resp["messages"] = resp_messages return n.mn.Reply(req, resp) } The full code for Part B is at internal\/broadcast\/3b.go.\n3c: Fault Tolerant Broadcast Part C introduces network partitions to temporarily prevent inter-node communication. To accommodate, we use RPC instead of Send, as RPC checks for a successful response. RPC takes a callback handler, which we use to set a local success variable to true to prevent further retries5. Otherwise, the network call is retried.\nTo reduce latency, we’ll have every node forward received broadcast messages even if they weren’t the first node to get it. This means we can detour around partitions when possible at the cost of message duplication. Of course, if a node finds that it has already received the message, we skip forwarding as it must have already done so earlier; this avoids infinite forwarding cycles. This resulted in tiny latencies (milliseconds): :stable-latencies {0 0, 0.5 0, 0.95 0, 0.99 3, 1 3}. Without this optimization \u2014 that is, if only the first node forwards \u2014 the latency was :stable-latencies {0 0, 0.5 1022, 0.95 10504, 0.99 11563, 1 12205}.\nfunc (n *FaultTolerantNode) forward_to_all(message int) { for _, neighbor := range n.mn.NodeIDs() { if neighbor == n.mn.ID() { continue } req := make(map[string]any) req["type"] = "broadcast" req["message"] = message go n.forward(neighbor, req) } } func (n *FaultTolerantNode) forward(neighbor string, body map[string]any) { for { success := false err := n.mn.RPC(neighbor, body, func(resp maelstrom.Message) error { success = true return nil }) if err == nil && success { return } \/\/ Let's not bother with fancy backoffs since we know the partition \/\/ heals eventually. time.Sleep(500 * time.Millisecond) } } broadcast := func(req maelstrom.Message) error { \/\/ ... messages := <-n.messages _, val_exists := messages[message] messages[message] = nil n.messages <- messages if !val_exists { go n.forward_to_all(message) } \/\/ ... } For the complete implementation of Part C, see internal\/broadcast\/3c.go.\nNote that nodes never crash so we don’t have to worry about persisting data to disk. ↩︎\nmaelstrom challenge: request to implement topology and then ignore it is very confusing. ↩︎\nThis is relevant for later challenges, where efficiency requirements mean you can’t have a node talk to every other node. ↩︎\nGo doesn’t have sets, so a map[int]interface{} is a workaround for creating a set of integers as the value in each key-value pair is ignored and usually set to nil. ↩︎\nI didn’t look at the documentation, but from brief experimentation I hypothesize that Send and RPC only return errors if there was an issue sending the message (which is probably rare). However, Send doesn’t guarantee receipt, while RPC will call the callback handler once the message is received, even if the recipient node doesn’t actually reply with anything. ↩︎\n","permalink":"https:\/\/lynshi.github.io\/posts\/gossip-glomers-3ac\/","summary":"Last time we introduced the Gossip Glomers challenge from Fly.io and discussed our approach to Challenge #2: Unique ID Generation.\nThis time, we’ll talk about the first three parts of Challenge #3: Broadcast. Parts D and E are saved for a separate post as they’re a bit more involved.\nThe overall theme of Challenge 3 is to build a broacast system to propagate messages to all nodes1. We iteratively build up our system, from a single-node cluster that simply stores and returns received messages, to a multi-node cluster that shares received messages, to a fault-tolerant multi-node cluster that can operate even during network partitions by Part C (D and E are about efficiency).","title":"Gossip Glomers 3 (a-c): Single-Node, Multi-Node, and Fault Tolerant Broadcast"},{"content":"One of the challenges for practicing implementing distributed systems is that it is not easy to simulate the various situations a distributed system might find itself in. Moreover, I previously could not even come up with an easy way to deploy a toy setup; the only thing I could think of is to use minikube and build a Kubernetes-based environment, but frankly at that point it is too much investment for me1.\nFortunately, I recently came across a series of distributed systems challenges created by fly.io and Kyle Kingsbury (author of Jepsen): Gossip Glomers. The challenges use Maelstrom, a framework for running and testing toy implementations of distributed systems, so that you only have to implement the individual nodes and not worry about anything else. Even better, Maelstrom provides a Go library containing the boilerplate for creating Maelstrom Nodes, leaving you to focus on the fun stuff2!\nWith this being so accessible, I guess I’ll be working through the Gossip Glomers as I get time. When I complete a challenge, I’ll also write a post about my thought process3!\nRepository Notes My code can be found on GitHub at lynshi\/gossip-glomers. Implementations for each challenge can be found under internal\/. Finally, I’ve set up GitHub Actions to run tests for each challenge.\nChallenge 2: Unique ID Generation Challenge 2 is Unique ID Generation. The system must generate globally unique IDs and be totally available, so network partitions must not lead to violation of the uniqueness constraint.\nIn order to be resilient to partitions, we cannot rely on node-to-node communication to synchronize the list of used IDs. One way of ensuring that nodes do not generate conflicting IDs is to split the range of possible IDs across nodes so that none of the ranges overlap.\nWhile the obvious approach is to split the pool of possible IDs in 34, an easier to implement (and subjectively more elegant) method is to have a unique prefix for each node. The prefix can be derived from the node ID (nodes are numbered n1, n2, …). Then, when asked for an ID, each node will return a locally-unique ID prefixed with its node ID. For example, the first ID generated by node 1 might look like 1-0.\nUpon rereading the prompt, I realized that IDs may be of any type, so the format <node ID>-<locally incrementing index> suffices to solve the problem. However, I completedly missed this as I was working on it, so I came up with a scheme to create integer IDs.\nSince the test sends 1000 requests per second for 30 seconds, the system is expected to generate 30,000 unique IDs. This means that the set of ID numbers generated by each node must include the range [0, 30000); this fits in a 15-bit number (2^15 = 32768). So, to generate a globally unique ID from a locally unique one, we left-shift the node ID by 15 bits to obtain the prefix, then add the locally unique ID. Overall, our ID generator generates 17-bit IDs5.\nImplementation Naively, the way to create locally unique IDs is to have a counter that starts at 0. Every time a generate message is received, we return the current counter value and increment the counter.\nThere is a small issue \u2014 presumably, a node can receive concurrent requests. There needs to be a synchronization primitive for the counter to ensure that each counter value is only read once. Normally we would use a lock to protect access, but we are using Go so let’s do better!\nWe’ll use a channel to provide unique IDs. The channel is initialized with a capacity of 1 so that we generate IDs on-demand, but you could set a higher capacity to have more IDs ready at a time, or even prepopulate with all possible IDs up front6. We use uint32 for the counter to avoid type conversions as the final ID requires 17 bits.\ncounter := make(chan uint32, 1) Next, we’ll start a goroutine to add a new, unique ID whenever the channel is emptied.\ni := uint32(0) go func() { for { select { case <-ctx.Done(): close(counter) return case counter <- i: i++ } } }() With a channel providing unique IDs now available, the unique_ids handler only needs to read a value from the channel to respond to a request. Recall the need to prefix the read ID with the node ID, which we do by shifting the node ID left 15 bits.\nnode_id, err := strconv.Atoi(n.ID()[1:]) if err != nil { return errors.Wrapf(err, "could not get integer from %s", n.ID()) } id, ok := <-counter if !ok { return errors.New("counter channel closed") } node_id = node_id << 15 \/\/ ... resp["id"] = uint32(node_id) + id For those curious, errors.Wrapf comes from github.com\/pkg\/errors, which provides convenient utilities for wrapping errors nicely to bubble up trace information.\nThat’s all there is to it! The full code can be found at internal\/unique_ids\/unique_ids.go.\nWe haven’t even gotten to how to create and run proper tests yet! ↩︎\nPlus the added benefit of getting to use Go \ud83e\udd23 ↩︎\nThe first challenge is just a “Hello, World!” for getting you set up, so I’ll skip it. ↩︎\nIn this challenge, there are only three nodes. ↩︎\nThe first two bits will be used for the unique prefix (00, 01, and 10). ↩︎\nBe careful not to create a situation where the channel is unbuffered and you continuously add IDs without an upper limit on the ID value! ↩︎\n","permalink":"https:\/\/lynshi.github.io\/posts\/gossip-glomers-intro-and-unique-id-generation\/","summary":"One of the challenges for practicing implementing distributed systems is that it is not easy to simulate the various situations a distributed system might find itself in. Moreover, I previously could not even come up with an easy way to deploy a toy setup; the only thing I could think of is to use minikube and build a Kubernetes-based environment, but frankly at that point it is too much investment for me1.","title":"Gossip Glomers: Intro and Unique ID Generation"},{"content":"In the past two years, we’ve moved twice. The first time we moved, we packed a bag of clothes intended for recycling or donation. Then, we got lazy and simply brought that bag with us while moving. The second time we moved, we brought the bag, still packed, without a second thought.\n7 months after settling into our new home, we decided enough was enough. We were going to get rid of all of the clothing and shoes we weren’t wearing anymore!\nThe problem \u2014 and a big reason why we procrastinated for so long \u2014 is that finding a place that isn’t going to just resell1 (or worse, ship overseas2) your clothing is not so easy! King County’s Threadcycle program has a list of partners that will accept used clothing, but every partner on the list resells and\/or ships abroad. Heck, some of the organizations are even for-profit (e.g. Value Village, which “marks [clothes] up for resale at its own thrift stores”, and USAgain)!\nThe next obvious step is to search on Reddit but:\nThe Reddit API protests made Reddit hard to use at the time and also raised questions about the durability of information there (besides Reddit itself going down, people are removing their old comments and posts). You still have to sift through inapplicable organizations (lots of pointers to Goodwill) and vet each individually (both for quality of the organization and acceptable items). Luckily for you, I already slogged through a bunch of Reddit posts and came out with a few options. Here, I’ll discuss some promising ones I found as well as the organizations we ended up giving our clothes to, both as donations and for recycling (don’t donate stuff that’s not suitable for wearing!).\nIf none work for you, the best way to keep looking is probably to find shelters and domestic violence centers, or you can try a Buy Nothing group!\nPromising organizations that will accept most clothing donations While there are actually a decent amount of organizations that accept clothing donations to give away directly, many of them have item restrictions. That’s fine if you only have allowed items or are ok sorting your stuff and making multiple trips to donate things, but it’s certainly not convenient. Thus, I’m only going to list places that will accept almost anything that is in good shape.\nAdult clothing These places take adult clothing but we didn’t select them for various reasons.\nWest Seattle Clothesline The West Seattle Food Bank operates The Clothesline, which is a free clothing bank. They accept “new or like-new clothing, bedding, hygiene products, tents, and suitcases\/backpacks to be distributed to the community”. The only drawback? They’re in West Seattle, which can be quite the hike if you weren’t already planning to be in the area!\nThe Bridge Care Center Located in Ballard, The Bridge Care Center provides resources for those experiencing homelessness or economic hardship. For example, they distribute adult clothing, shoes, and hygiene items. Unfortunately, at the time of writing they are about to move and thus have restrictions on acceptable items. However, you can check their donations page to see if those restrictions have now been lifted.\nElizabeth Gregory Home The Elizabeth Gregory Home is geared towards supporting women who are “experiencing homelessness, trauma, or economic insecurity”. As a result, they only accept women’s clothing. While they don’t explicitly say that they give out clothing for free, they offer lots of free services and items so it would be strange if clothing donations ended up being resold.\nChildren’s clothing Since we only had adult clothing, we didn’t donate to any of these places but they appear to be reasonable options.\nWellspring Family Services Wellspring Family Services provides services for families facing homelessness. They operate a free family store for parents to shop for children aged 0-17. Consequently, they only accept donations for children 0-17 years old.\nThe Works The Works provides free children’s and teen clothing for Shoreline School District families. However, donations are only accepted during the school year.\nWhere we donated Jubilee Women’s Center Jubilee Women’s Center supports women experiencing poverty. They operate a boutique where women can shop for clothing. They accept women’s clothing and don’t list any specific restrictions (though obviously only donate clean items in good condition).\nFacing Homelessness Facing Homeless operates The Window of Kindness. Volunteers hand out “socks, gloves, snacks, sleeping bags, and other supplies” and chat with those that drop in. They find that most of the people they serve are adult men so “dressier” women’s clothing should be donated elsewhere, but otherwise they accept most things you can think of with a few restrictions on what needs to be new. A few other things to note:\nTheir “Donate Supplies” page says to email them to schedule a drop-off, but the main Donate page has a Calendly link to book a slot directly. There’s no “store” to walk into and drop things off so you might have to wait in line to get someone to come out to take your stuff. Where we recycled The Recology Store Recology is a company contracted for solid-waste collection in Seattle. For some reason, they also operate stores in King County where they sell products that promote a zero-waste lifestyle.\nWhile I don’t know how I feel about buying products from a waste management company, the nice thing about the Recology Stores is that they accept hard-to-recycle items and only charge a fee for non-customers3. Besides clothing, I was happy to find that they accept Styrofoam \u2014 we’ve been driving all the way down to Styro Recycle in Kent to recycle that4!\nUnfortunately, I couldn’t find any specific information about Recology’s recycling process for textiles, but I would hope that a literal waste management company is processing them correctly!\nI suppose there’s nothing wrong in principle with clothing being sold in thrift stores where the proceeds go to charity, but that’s still worse than letting it be given away for free. ↩︎\nUsed clothing sent abroad, ostensibly to support local sellers and provide affordable clothing, can negatively impact local clothing industries or end up in landfills anyway. ↩︎\nYou can check Seattle’s Solid Waste Collection Contracts to see if you’re a Recology or Waste Management customer. That said, they didn’t ask for my address, so I don’t know if they are super strict about which service area you fall into as long as you’re in Seattle. ↩︎\nWe might keep doing that anyway since (in my humble opinion) Washington’s best bubble tea is in Kent. ↩︎\n","permalink":"https:\/\/lynshi.github.io\/posts\/getting-rid-of-old-clothes-in-seattle\/","summary":"In the past two years, we’ve moved twice. The first time we moved, we packed a bag of clothes intended for recycling or donation. Then, we got lazy and simply brought that bag with us while moving. The second time we moved, we brought the bag, still packed, without a second thought.\n7 months after settling into our new home, we decided enough was enough. We were going to get rid of all of the clothing and shoes we weren’t wearing anymore!","title":"Getting Rid of Old Clothes in Seattle"},{"content":"I\u2019ll soon be attending Markus Kuppe\u2019s workshop on TLA+ and one of the pre-read materials is Dijkstra\u2019s EWD998 - Shmuel Safra’s version of termination detection. I haven\u2019t read a serious, academic paper since college like 3 years ago1, so it was quite an adventure getting back into the swing of things. As I was reading, I spent a lot of time going back and forth to make things make sense, because the hallmark of a real paper is that you can\u2019t just consume it in one go if you actually want to understand the material.\nOne of the nice things about a college course\/lecture is that you get the teacher\u2019s notes edition of the paper\/proof. You get the \u201cwhy\u201d behind statements in the paper, not just how. Whereas if I\u2019m reading a paper by myself, I often have to pause and ask \u201cWell, I see that you\u2019ve gone from Point A to Point B, and I believe your logic was sound, but why did we do that? And how on earth did you come up with Point A in the first place?!\u201d\nSo, after I spent a good chunk of my time unwrapping this paper, I figured I could throw something onto the Internet to make someone else\u2019s reading a little easier. Read on for what is essentially EWD998 restated with longer explanations where Dijkstra’s brevity forced me to stop and think :)\nBe warned that the mathematical notation is still around (though I try to also explain it in English). It is a succinct yet clear way to express the ideas, which is rather neat!2\nThe premise of the paper is that there\u2019s a circle of $N$ machines indexed from 0. Each machine is either active or passive. Active machines may send messages to other machines. These messages take time to travel between machines; however, no messages are ever lost. An active machine can become passive “spontaneously”. Meanwhile, a passive machine only becomes active upon receipt of a message.\nWe want to figure out when the system is stable; that is, all machines are passive and there are no messages in transit. We\u2019re going to devise a solution where the first machine, $m_0$3, can detect when the system has reached a stable state (i.e. termination). This termination detection algorithm is called “the probe”.\nThe machines, regardless of their active\/passive status, can always communicate such that:\nMachine $m_0$ can kick off the probe by sending a signal (i.e. a token) to $m_{N - 1}$. $m_{i + 1}$ can always propogate the probe by sending the token to $m_{i}$, even if $m_{i + 1}$ is passive. Eventually, this token makes its way back to $m_0$. Based on information on the token and the state of $m_0$, we’ll be able to conclude whether the stable state has been reached4.\nThe strategy for solving this problem is to iteratively construct an invariant $P$ that is always true in the system and rules to maintain the invariant. This invariant will help us determine the conditions that are satisfied if and only if the system is stable when the token returns to $m_0$.\nConcretely, we’ll\nConstruct an invariant. Devise a rule for system operation to maintain it. Find an edge case that causes the invariant to break. Each edge case can be addressed by adding a new condition to the invariant, so we loop back to 1. When we add a new condition, we have to be careful that the condition still allows us to detect termination.\nBased on Dijkstra’s closing5, this strategy seem to have been discussed for some time, which explains why EWD998 reads so matter-of-factly6. But here I’m telling you up front what to expect, so you don’t have to tear your hair out wondering what magic possessed Safra such that the logic moves so effortlessly from step to step!\nLet $t$ be the index of the machine holding the token and $B$ be the number of messages on their way. We would like to determine whether the system has reached stable state when the token returns to $m_0$ (i.e. $t = 0$), and our definitions mean that termination can be stated as7 $$\\forall i \\in [0, N): m_i \\text{ is passive} \\land B = 0$$\nIn other words, the system is stable if all machines are passive and no messages are on their way (to wake up a machine).\nIt’s clear we need to know $B$ to detect termination. If we knew how many messages had been sent and received by each machine, we can easily compute $B$. Thus, let’s construct our first invariant, $P_0$, to help us keep track of $B$.\n$$P_0: \\quad B = \\Sigma_{i = 0}^{N - 1}c_i$$\nIntuitively, we want $c_i$ to be the net messages sent by machine $i$, and we enforce this by adding a rule to the system.\nRule 0: Each machine maintains its own counter $c_i$, incrementing it by 1 when it sends a message and decrementing it by 1 when it receives a message8. It follows that we should initialize all $c_i$ to 0 when the machines first start.\n$P_0$ doesn’t depend on $t$, so $m_0$ receiving the token after it has been sent around the ring doesn’t help us determine anything. Consequently, let’s say that the token has a value $q$ and add a condition $P_1$.\n$$P_1: \\quad \\forall i \\in (t, N): m_i \\text{ is passive} \\land \\Sigma_{i = t + 1}^{N - 1}c_i = q$$\nIf $P_1$ is true, that means that all machines that have already handled the token are passive, and the sum of their counters $c_i$ is equal to the value of the token. The token being integer-valued and connected to the $c_i$ is critical to the termination detection algorithm as it encodes information about $B$, which is hard to know without an overall view of the system, into something that can be passed around9.\nNow, our invariant is $P_0 \\land P_1$. If the invariant holds, when the token returns at $t = 0$,\n$$ \\begin{aligned} & P_0 \\land P_1 \\land t = 0 \\\\ \\implies & B = \\Sigma_{i = 0}^{N - 1}c_i \\land \\forall i \\in (0, N): m_i \\text{ is passive} \\land \\Sigma_{i = 1}^{N - 1}c_i = q \\\\ \\implies & B = c_0 + q \\land \\forall i \\in (0, N): m_i \\text{ is passive} \\end{aligned} $$\nThat is, all machines other than $m_0$ are passive (plus the little equation, $c_0 + q = B$, that follows from some arithmetic).\nRecall that all machines are passive and $B = 0$ when the system is stable. Thus, given the invariant, the system is stable when $t = 0$ if $m_0$ is passive (it’s the only machine we aren’t certain about the state of) and $c_0 + q = 0$.\nHow do we maintain $P_1$? Observe that the probe starts with $t = N - 1, q = 0$. The boundary conditions mean that at this point, there are no terms in the $\\forall$ and $\\Sigma$ operations, so while we aren’t making a statement on which machines are passive, we are stating that $q$ must be 0 since there’s nothing being summed. That leads us to Rule 1.\nRule 1: When the probe starts, $m_0$ sends the token with a value of 0 (i.e. $q = 0$) to $m_{N - 1}$.\nWhen a machine transmits the token, due to the invariant it must be making a statement that it is passive. It also must update the value of the token. Thus, we introduce Rule 2.\nRule 2: A machine $m_{i + 1}$ only transmits the token to the next machine $m_i$ after $m_{i + 1}$ becomes passive, and the token’s value is updated to $q = q + c_{i + 1}$.\nWhen is $P_1$ false? Most directly, $P_1$ is false if, when the token is at index $t$, some machine $m_i$, $t < i < N$, is active or $\\Sigma_{i = t + 1}^{N - 1}c_i \\neq q$. Let $m_i$ be the first machine to cause a violation of $P_1$.\nSuppose $m_i$ violates $P_1$ by becoming active. Recall that a machine only forwards the token when it becomes passive, so if $m_i$ is now active, it must have received a message.\nMeanwhile, suppose $\\Sigma_{j = t + 1}^{N - 1}c_j \\neq q$. Since $m_i$ is the first machine to cause a violation of $P_1$, $\\Sigma_{j = i + 1}^{N - 1}c_j = q_i$ (let $q_k$ be the value of the token when $m_k$ received it) must have been true for the whole period during which $m_i$ held the token. By Rule 2, $\\Sigma_{j = i}^{N - 1} = q_{i - 1}$ must also be true when $m_i$ transmits the token, and remain true until the violation is caused by $m_i$. A violation can be caused by $m_i$ if $c_i$ changes. As $m_i$ is passive, $c_i$ can only change via receipt of a message.\nWe conclude that the only way for $m_i$ to cause a violation of $P_1$ is by receiving a message. In real world terms, this means that $m_i$ was woken up by a message that was either still in-transit or yet to be sent when it became passive. Further, in the instance right before the violation, there must be a message on its way to $m_i$. This is expressed by $B \\geq 1$.\nSince $P_0 \\land P_1$ is true the moment before $P_1$ is violated, we have\n$$ \\begin{aligned} 1 \\leq B = & \\Sigma_{i = 0}^{N - 1}c_i \\\\ B = & \\Sigma_{i = 0}^{t}c_i + \\Sigma_{i = t + 1}^{N - 1}c_i \\\\ B = & \\Sigma_{i = 0}^{t}c_i + q \\\\ \\therefore 0 < 1 &\\leq \\Sigma_{i = 0}^{t}c_i + q \\\\ \\\\ P_2: \\quad \\quad \\quad & \\Sigma_{i = 0}^{t}c_i + q > 0 \\end{aligned} $$\nObserve that $P_2$ is true even if $m_i$, $t < i < N$, receives a message because no machine with $i > t$ is involved in the statement. So we can say either $P_1$ or $P_2$ is true, and our invariant becomes $P_0 \\land (P_1 \\lor P_2)$.\nWhen the probe concludes at $t = 0$, $P_2$ evaluates as $c_0 + q > 0$. Recall that when the system is stable, we have $c_0 + q = 0$, so $P_2$ must be false when that occurs. Thus, the invariant returns to just $P_0 \\land P_1$ when $c_0 + q = 0$, and we already know how to conclude termination when $P_0 \\land P_1$. As a result, introducing $P_2$ does not make it any harder to conclude termination.\nIt’s getting gnarly now! Under $P_2$, $m_i$ where $0 \\leq i \\leq t$ is free to send as many messages as desired - doing so only increases $c_i$, so the invariant is maintained. However, if $c_i$ decreases due to $m_i$ receiving a message, we might falsify $P_2$! So, let’s add a statement to the invariant to capture the receipt of a message by $m_i$.\n$$ P_3: \\quad \\exists i \\in [0, t]: \\text{machine } i \\text{ is black} $$\nWe also add a corresponding Rule 3.\nRule 3: When a machine receives a message, it changes its color to black.\nFortunately, this still doesn’t make it any harder to detect termination. Our invariant is now $P_0 \\land (P_1 \\lor P_2 \\lor P_3)$, and we already know that $P_2$ is false at termination, so we just need to make sure that $P_3$ is also false at termination.\nAt $t = 0$, when we can try to determine termination, $m_0$ must be black to satisfy $P_3$. If $P_3$ is false, we can apply our previously derived techniques for detecting termination; thus, if $m_0$ is white when the token returns, we have a shot at detecting termination.\nWhat, you thought we were going to stop at $P_3$? Nope! If a black machine propagates the token, we are in danger of violating $P_3$ since that machine falls out of the set of machines referenced in $P_3$. This doesn’t necessarily mean the full invariant is invalidated as other parts of it can be true, so let’s construct a scenario where $P_1$ and $P_2$ are also false when a black machine propagates the token.\nLet $m_t$, $t > 0$ be the last black machine. Suppose that machines $m_i$, $i < t$ have $c_i = 0$ and are white. While $m_t$ holds the token, it sends a message to $m_x$, $x > t$ - this breaks $P_1$!\nThen, $m_t$ passes the token to $m_{t - 1}$. Suddenly, $P_2$ is falsified as $\\sum_{i = 0}^{t - 1}c_i = 0$. Meanwhile, no machines $0..t - 1$ received a message, so all machines within the set described in $P_3$ are white and $P_3$ is false too!\nClearly, we have to introduce $P_4$ to rectify the situation. Let’s color the token!\n$$P_4: \\quad \\text{the token is black}$$\nRule 4: If a black machine is to transmit the token, it colors the token black before sending it.\nIf the token is white when it makes it back to $m_0$, then $P_4$ is false, and we have a chance at detecting termination.\nAt this point, we are done modifying the invariant! The color of the token never changes back to white from black, so it can never be made untrue at some later step.\nOverall, we constructed the invariant as follows:\nConstructed $P_1$. $P_1$ can be falsified by the receipt of a message by $m_i$, $t < i < N$, so we constructed $P_2$ which is true at least as early as the instant before $P_1$ becomes false. $P_2$ can be falsified by receipt of a message by $m_i$, $0 \\leq i \\leq t$, so we constructed $P_3$, which becomes true the instant $m_i$ receives a message. $P_3$ can be falsified by the last black machine transmitting the token, so we constructed $P_4$, which becomes true whenever any black machine transmits the token. Once true, $P_4$ never becomes false. Thus, by construction our invariant holds for all states in the system.\nLet us summarize what we know so far. The invariant $P_0 \\land (P_1 \\lor P_2 \\lor P_3 \\lor P_4)$ is always true. $P_0$ is true by design, and we know how to detect termination when $P_0 \\land P_1$. By the invariant, $P_1$ must be true if $P_2$, $P_3$, and $P_4$ are all false. Thus, we have a chance at detecting termination when the token returns at $t = 0$ if:\n$c_0 + q = 0 \\implies \\neg P_2$ (i.e. $P_2$ is false) $m_0 \\text{ is white} \\implies \\neg P_3$ $\\text{The token is white} \\implies \\neg P_4$ When only $P_1$ is true when the token returns, all machines $1..N-1$ are passive, so the system is totally passive if $m_0$ is also passive.\nThus, points 1-3 and $m_0$ being passive at $t = 0$ are the conditions for concluding that the system has reached stable state.\nWhat happens if any of the conditions are false? Well, we can’t make a conclusive determination, so we run the probe again - what, you thought we could only probe once?\nRule 5: $m_0$ initiates another probe after an unsuccessful probe.\nRunning the probe again is insufficient though! If the token can’t change back to white, the outcome of the next probe will be no different ($P_4$ will always be true). Further, even if the token color can change, if machine colors don’t change, then either $m_0$ stays black or the token is turned back to black as it makes its way around the ring. Consequently, we need to be able to change colors for both the token and all machines.\nWhile changing colors back to white, we must be careful to ensure the invariant $P_0 \\land (P_1 \\lor P_2 \\lor P_3 \\lor P_4)$ is maintained! Otherwise, all of our good work will have gone to waste.\nNote that the invariant is true when the probe is first started. $P_0$ is always true thanks to Rule 0, and we established earlier that $P_1$ is trivially true when $m_0$ sends the token to $m_{N - 1}$. So, we can whiten $m_0$ and the token when initiating the probe again.\nRule 6: At initiation of the probe, $m_0$ whitens itself and the token.\nAdditionally, note that $P_3$ only makes a statement about machines whose index is $t$ or less. As a result, it’s always safe to whiten a machine whose index exceeds $t$.\nRule 7: After transmitting the token to $m_i$, $m_{i + 1}$ whitens itself.\nIt follows that when no messages are in-flight ($B = 0$), no machines turn black anymore, all black machines whiten themselves eventually, and the token eventually stays white. When $B=0$ and all machines are passive, every $c_i$ becomes constant because no machine is able to send messages and there are no messages in transit that might decrease some $c_i$. When a probe is started during such a state, $P_1$ stays true throughout the probe. When the probe ends, $c_0 + q = B = 0$. Therefore, all conditions for detecting termination eventually become true after the system becomes passive!10\nExhausted? That makes two of us!\nI started writing this post while I was reading EWD998, but I’ve already completed the workshop and have yet to publish it \ud83d\ude05\nSo, I’ll take this opportunity to note that the workshop was terrific! Many thanks and kudos to Markus for putting together such a great experience. We implemented the termination detection algorithm - you can see my spec here - and I learned a lot about TLA+! Hopefully I’ll get a chance to write a spec for some of the systems I’ve designed to catch all the edge cases I undoubtedly have missed.\nAckchyually, I read the Raft paper when I first started working (technically still almost 3 years ago though!) and Paxos vs Raft: Have we reached consensus on distributed consensus? sometime last year. But they didn\u2019t have any mathematical notation, so my brain didn\u2019t really get fried and therefore it doesn’t count. ↩︎\nAfter reading this article, Leslie Lamport wrote to say that “math isn’t a hinderance to understanding but rather a requirement.” I try to be a bit gentler, but I do agree! Words can be ambiguous, but symbols are not. ↩︎\nDijkstra uses the term nr.i to refer to machine i, but that doesn’t look good and is even a little confusing in LaTeX. ↩︎\nThis concludes the easy part! If I were a student, I’d be worried about plagiarism because I’m almost quoting the paper verbatim so far. Thankfully, we don’t have to be so rigorous here. ↩︎\n“Neither the algorithm nor its variations are the point of the note, which is about the derivation strategy, which worked again.” ↩︎\nEWD998 references EWD840, which solves a similar problem with the assumption of instantaneous message delivery. EWD840 feels a bit more accessible - maybe because I read (*cough* skimmed) it after EWD998, but it also seems to contain a bit more color on why certain steps are taken. ↩︎\nDijkstra uses $\\underline{A}$, $\\underline{S}$, and $\\underline{E}$ for $\\forall$ (for all), $\\Sigma$ (sum), and $\\exists$ (exists), respectively. I’ll use the math symbols since they’re more familiar to me. All ranges are over integers, so $[0, 5) = \\{ 0, 1, 2, 3, 4 \\}$. ↩︎\nDijkstra remarks that because the value of $B$ changes in a distributed fashion (i.e. based on the individual sending\/receiving actions of each machine), our only option to compute $B$ is to try to compute it in a distributed fashion too. Sounds reasonable to me! ↩︎\nThis explains why Dijkstra’s previous solution in EWD840 only uses a colored token. In that scenario, messages are delivered instantaneously, so we don’t have to worry about keeping track of how many messages are in-flight. This also suggests that an integer-valued token (as well as formulation of the problem) was Shmuel Safra’s key contribution. ↩︎\n“Eventually” is doing a lot of work for us here! As Markus clarified, this derivation only ensures that the rules for system operation and termination detection are guaranteed to be correct. No statements are made about non-correctness properties (e.g. efficiency - whether there are opportunities to reduce the lag between termination and detection). Similarly, the spec constructed in the workshop only satisfies safety and liveness; that is, it shows that the algorithm eventually detects termination if the system terminates, and termination is correctly detected. It turns out that TLA+ can also be used to study non-correctness properties too! ↩︎\n","permalink":"https:\/\/lynshi.github.io\/posts\/understanding-ewd998\/","summary":"I\u2019ll soon be attending Markus Kuppe\u2019s workshop on TLA+ and one of the pre-read materials is Dijkstra\u2019s EWD998 - Shmuel Safra’s version of termination detection. I haven\u2019t read a serious, academic paper since college like 3 years ago1, so it was quite an adventure getting back into the swing of things. As I was reading, I spent a lot of time going back and forth to make things make sense, because the hallmark of a real paper is that you can\u2019t just consume it in one go if you actually want to understand the material.","title":"Understanding EWD998: Shmuel Safra's version of termination detection"},{"content":" Originally published on Medium.\nWhen you use Azure CDN to deliver content, you may need to purge your endpoint so that changes you make are sent to your users, as files are cached in the Azure CDN until their time-to-live (TTL) expires. If you don\u2019t set a TTL for your files, Azure automatically sets a TTL of 7 days. Even if you set a lower TTL, your updates may not coincide with the cache expiration.\nFor example, my personal website uses Azure CDN and is updated on every push I make to GitHub. It\u2019s tough to set a good caching rule for this because my changes are unpredictable. When I\u2019m working on my site, I might push several times a day, but I can also go weeks without modifying my website.\nWhile it is possible to purge an endpoint through the Azure portal, I wanted to automate this process. Fortunately, this is possible with GitHub Actions.\nCreating a Service Principal An Azure service principal is \u201can identity created for use with applications, hosted services, and automated tools to access Azure resources\u201d. It is recommended, for better security, to use service principals with automated tools, since access is restricted by roles assigned to the service principal. In this section, I\u2019ll discuss how to create a service principal and give it the CDN Endpoint Contributor role. To get started, you\u2019ll need the Azure CLI.\nTo create a new service principal, enter the following command in your terminal, making sure to copy the correct<subscription_id> and <resource_group_name> from your Azure portal.\naz ad sp create-for-rbac -n "$nameOfServicePrincipal" --role "CDN Endpoint Contributor" --sdk-auth --scopes "\/subscriptions\/$subscriptionId\/resourceGroups\/$resourceGroup" The output will look similar to the following; be sure to copy and save it somewhere safe, as you will not be able to retrieve the clientSecret in the future.\n{ "clientId": "<GUID>", "clientSecret": "<GUID>", "subscriptionId": "<GUID>", "tenantId": "<GUID>" } In the command, we are creating a service principal and assigning it a role of \u201cCDN Endpoint Contributor\u201d. This allows the service principal to manage CDN endpoints. The --sdk-auth flag causes the output to be correctly formatted for use in the GitHub Action; without it, you will get the following error when you copy the output to GitHub.\nError: Not all values are present in the creds object. Ensure clientId, clientSecret, tenantId and subscriptionId are supplied. Adding GitHub\u00a0Secrets Add the following secrets (encrypted environment variables) to your GitHub repository:\nAZURE_CREDENTIALS: paste the output from the az ad sp create-for-rbac command. AZURE_CDN_ENDPOINT: the name of your Azure CDN endpoint. AZURE_CDN_PROFILE_NAME: the name of your Azure CDN profile. AZURE_RESOURCE_GROUP: the name of the resource group containing the Azure CDN profile. Writing the GitHub Actions Workflow\u00a0File Now that you have a service principal and GitHub secrets, all that remains is to write the GitHub Action. There are two essential steps in the workflow: 1) logging in to Azure, and 2) purging the CDN. To start, go to your repository on GitHub and select the \u201cActions\u201d tab. Click \u201cNew workflow\u201d, then choose \u201cSet up a workflow yourself\u201d or the appropriate starter for your needs.\nThe following is the bare minimum required for this workflow.\nThe Azure service principal login step logs in to Azure CLI with the credentials you added to your GitHub secret. The Purge CDN step purges the CDN, so your updates should occur between these two steps. In the Purge CDN step, --content-paths "\/*" purges all content. --no-wait means the script does not wait for the purge operation to finish before continuing; this flag is optional. You can read more about the az cdn endpoint purge command here.\nThat\u2019s all there is to it, happy coding!\n","permalink":"https:\/\/lynshi.github.io\/posts\/@shilyndon\/purging-azure-cdn-with-github-actions-1c18e2adaf18\/","summary":"Originally published on Medium.\nWhen you use Azure CDN to deliver content, you may need to purge your endpoint so that changes you make are sent to your users, as files are cached in the Azure CDN until their time-to-live (TTL) expires. If you don\u2019t set a TTL for your files, Azure automatically sets a TTL of 7 days. Even if you set a lower TTL, your updates may not coincide with the cache expiration.","title":"Purging Azure CDN With GitHub Actions"},{"content":" Originally published on Medium.\nI host my own static website on Azure Storage, and I got tired of running a script to upload my content every time I made a change to my website. After some Googling, I discovered GitHub Actions, made public about a month ago, which lets users automate how they build, test, and deploy their projects. Interestingly, though I found great guides for automating static website deployment to Azure using GitLab and building and deploying Gatsby sites with GitHub Actions, there wasn\u2019t one guide with everything I needed, and I had to cobble pieces together from different sources. This is an attempt to rectify that.\nThis post assumes your static website is already up and running on Azure Storage. If not, the Azure documentation on static website hosting was enough for me to get started.\nSetup To begin, go to the Azure Storage account with your static website content, and find the setting \u201cAccess keys\u201d. Copy either one of the connection strings; this is needed to upload content to Azure via GitHub actions.\nNow, open your personal website repository on GitHub. Go to the settings tab and select \u201cSecrets\u201d in the left panel. Click \u201cAdd a new secret\u201d, name it \u201cAZURE_STORAGE_CONNECTION_STRING\u201d (you can use a different name as long as you update the GitHub Actions\u00a0.yml accordingly), and paste the connection string value obtained from Azure into the \u201cValue\u201d area.\nWorkflow Script To build your Action, go to the \u201cActions\u201d tab in your repository. Select \u201cNew workflow\u201d and use the Node.js starter.\nIn the page that loads, paste the following code into the editor.\nThe above\u00a0.yml is tailored to my personal website, which was built with React and Gatsby. Hence, you may not need all the steps listed.\nThe Install dependencies step is only necessary if you used Node packages. The Build with Gatsby step can be replaced with your build command, if you have one. The Azure upload step is the only required step. The first line in the script deletes all your old files in Azure, while the second uploads your local files. Since Gatsby generates my website files in the public folder, I upload that (-s parameter); you may need a different value there. After uploading your files, you may need to purge your CDN endpoint to see the updates. (Update: I now use GitHub actions to automate purging.)\nThat\u2019s it! Now, your content will automatically be uploaded to Azure Storage on every push.\n","permalink":"https:\/\/lynshi.github.io\/posts\/@shilyndon\/automating-deployment-of-a-gatsby-static-website-on-azure-storage-with-github-actions-c81a63b32a9a\/","summary":"Originally published on Medium.\nI host my own static website on Azure Storage, and I got tired of running a script to upload my content every time I made a change to my website. After some Googling, I discovered GitHub Actions, made public about a month ago, which lets users automate how they build, test, and deploy their projects. Interestingly, though I found great guides for automating static website deployment to Azure using GitLab and building and deploying Gatsby sites with GitHub Actions, there wasn\u2019t one guide with everything I needed, and I had to cobble pieces together from different sources.","title":"Automating Deployment of a (Gatsby) Static Website on Azure Storage with GitHub Actions"}]