{"@attributes":{"version":"2.0"},"channel":{"title":"DHO is Mostly Confused","link":"https:\/\/9vx.org\/index.xml","description":"Recent content on DHO is Mostly Confused","generator":"Hugo -- gohugo.io","language":"en-us","copyright":"This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. (http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/)","lastBuildDate":"Sun, 22 Apr 2018 00:40:00 +0000","item":[{"title":"Wait-free Monitors for Concurrent Batching Workloads","link":"https:\/\/9vx.org\/post\/wait-free-monitors\/","pubDate":"Sun, 22 Apr 2018 00:40:00 +0000","guid":"https:\/\/9vx.org\/post\/wait-free-monitors\/","description":"

Monitors are hard to use.<\/span>\n Per Wikipedia’s entry on the topic<\/a>, “a monitor is a synchronization construct that allows threads to have both mutual exclusion and the ability to wait (block) for a certain condition to become true.” The use case for such a construct is often to signal other threads that new work may be completed. Unfortunately, they’re really easy to use incorrectly, largely because many APIs for them are terrible.<\/p>\n\n

In POSIX, monitors are provided through the API for condition variables. A new condition variable is created with pthread_cond_init(3)<\/code>; conditions are waited on and signalled using pthread_cond_wait(3)<\/code> \/ pthread_cond_timedwait(3)<\/code> and pthread_cond_signal(3)<\/code> \/ pthread_cond_broadcast(3)<\/code>, respectively. To use the pthread_cond_wait(3)<\/code> interface, you also need a mutex (because what you really have is a monitor); this mutex must be held before calling pthread_cond_wait(3)<\/code>, which will unlock the mutex before waiting on the condition, and return with the mutex still held. If this doesn’t sound subtle enough, consider the really crappy bit: the pthread_cond_signal(3)<\/code> API does not accept a mutex as an argument at all! What you get from this is a ton of confusion about lost signals when people call pthread_cond_signal(3)<\/code> without holding the mutex around signalling the condition.<\/p>\n\n

And what happens in this scenario? You lose messages: notifications on a condition are not persistent. This API assumes that the only time the mutex will be unlocked is when a thread is waiting on a condition, and that signals will only occur with that mutex held. The problem is that, although the API had an opportunity to enforce this, it did not do so.<\/p>\n\n

Apologists for the API say that one should “just know” how to use it correctly: these requirements are common knowledge. But misuse appears even in popular software, written by experienced individuals. One such example can be found in the source code of the widely popular Varnish Cache<\/a>: the condvar is waited on with the lock held<\/a>, but notifications are sent without the lock held<\/a>. In fact, even though the condvar is waited on with the lock held, that lock is also released for portions of the loop that waits on the condition, so it’s still possible for the mutex to be acquired when the condition isn’t being waited upon. And it’s done this way because that mutex is conflated between the condition and shared mutable state in the structure it lives in. This misuse of condvars has existed in Varnish for over a decade.<\/p>\n\n

Let’s take a step back<\/span>\n and consider a couple use-cases for monitors.<\/p>\n\n

First, let’s consider a log producer where many worker threads are producing log messages, and a consumer thread is persisting those to some kind of storage in batches. When no work is being done, we don’t want the consumer thread to spin waiting for logs to write. So our producers acquire a lock and signal a condition that the consumer waits on. This is problematic in high-volume services, because there’s significant contention on the monitor’s mutex. Additionally, if we have misused the API, it’s possible that a low-volume service with a burst of traffic will have logs delayed until the next log item occurs – which may be a while, since the service doesn’t have much volume.<\/p>\n\n

Consider a sharded worker pool where one or more producers may generate work items, and a pool of workers must each do something different with these work items. In this scenario, every worker thread must see every message. You might be tempted to use pthread_cond_broadcast(3)<\/code>, but this turns out to be an anti-pattern. First of all, each waiter on the condvar must have acquired the mutex to return from pthread_cond_wait(3)<\/code>, so this means that your worker jobs actually execute serially. And if you’ve decided to solve this by unlocking the mutex while you perform your work, another producer may decide to broadcast while no workers are waiting. In reality, you have two options if you want to maintain correctness while also parallelizing your work: either you use N condition variables and N mutexes to signal N workers, or you need to wait on a semaphore (that is also protected by the monitor’s lock!) to ensure all your workers are waiting before you broadcast. Neither of these solutions are particularly elegant.<\/p>\n\n

My final gripe with monitors<\/span>\n is that they require lock-based synchronization. Lock-based synchronization is nearly the canonical way to protect shared mutable state from concurrent access, but this approach has several drawbacks.<\/p>\n\n

Locks are not composable. When we talk about composability with regard to an API, we really mean a couple of things. First, a composable interface is self-contained<\/em>: it does not have dependencies on external state for its correctness. Mutexes are not self-contained: their correct use depends on consuming the mutex API at every appropriate place in code, and the API is also sensitive to ordering. Ordering issues don’t sound bad when all you have to consider are lock and unlock operations, but sometimes mutable state must be protected by multiple mutexes. At this point, the ordering of which mutex is locked first is important, as is the ordering of unlock operations – again, at every point that these mutexes have to operate. Apologists will say that it’s easier to use locks than anything else; I have yet to see or work on a large, multi-threaded codebase that hasn’t run into issues directly related to lack of composability.<\/p>\n\n

Another issue with locks is effectively how they work. If you ask folks what a lock does, they’ll typically say something like, “A lock protects concurrent access to shared memory within its critical section.” But this isn’t really what locks do. Locks are a construct that serialize execution through their critical section such that only a single process may execute that code at a time. This has the side-effect of achieving the goal of protecting concurrent access to shared mutable state, but unless you consider how<\/em> it achieves that, you’re missing numerous downsides. While a lock is held, no other concurrent task may acquire the lock. This means that resources dedicated to running code are effectively stalled waiting for the critical section to finish. Maybe this doesn’t seem so bad; critical sections are typically pretty short. The problem is that modern schedulers only allow tasks to run for so long. Once you’ve used your timeshare, it’s going to want to run something else. If you’ve lost your timeshare in the middle of a critical section, literally every other process waiting on that lock is also<\/em> not runnable, and your system makes no progress.<\/p>\n\n

These issues are fundamental to the monitor API provided by pthreads. (Other monitor implementations may address the composability problem a bit better, but the issue with progress guarantees remains by virtue of mutual exclusion.) The pthread monitor interface introduces additional dependencies between a mutex and data, imposes additional requirements on both lock ordering and critical section size, and does nothing to solve the problem of system progress (which is ostensibly one reason you’d want to use threads at all).<\/p>\n\n

Can we do better?<\/span>\n At my previous job, I had a few situations similar to the log producer situation I mentioned above: multiple threads producing work items, and a single thread acting upon that work in batches. I wanted to use some lock-free datastructures from Concurrency Kit<\/a> to create a queue of work that the consumer could easily batch, but there were some problems. With monitors, I’d be blocking work producers while the batch work was being completed, and in some cases that could have taken a non-trivial amount of time. In other cases, the consumer was able to operate quickly, but the production was high-volume, and I didn’t want to have producers blocking each other in their fast path. So I made a wait-free monitor for MPSC workloads.<\/p>\n\n

I had wanted to write about this monitor while I worked there, but never got around to it. Now that I don’t work there anymore, I don’t have the code. And I didn’t remember what clever tricks I had used. So I decided to re\u00efnvent it. My new version is portable to any POSIX system with support for Concurrency Kit’s<\/a> atomic primitives; I seem to recall that what I’d done in the original code was Linux-specific. I may implement system-specific versions later; POSIX’s lack of an interface like eventfd(2)<\/code> in Linux, and its lack of ability to report time slept in poll(2)<\/code> on EINTR<\/code> make the POSIX implementation a bit more heavyweight than it really needs to be. The remainder of this article will be a discussion of my new implementation.<\/p>\n\n

The notification system can exist in one of three states.<\/p>\n\n

enum monitor_state {\n    STATE_WORKING,\n    STATE_PENDING,\n    STATE_BLOCKED,\n};\n<\/code><\/pre>\n\n
STATE_WORKING<\/code> is the initial state of the system, and is the state of the monitor when it is created. This state indicates that no waiters are currently listening to the monitor. The second state of the monitor is STATE_PENDING<\/code>, which indicates that work is pending. The final state is STATE_BLOCKED<\/code>, which indicates that the consumer wanted to do some work, but none was available.<\/p>\n\n
The structure itself is relatively simple: it contains some file descriptors for a pipe (to block when no work is pending), the state of the monitor, and an unfortunate flag as to whether the pipe needs draining (this is a side-effect of POSIX lacking anything eventfd(2)<\/code>-like). It also contains a generation counter that is used to avoid having multiple producers write to the pipe to unblock a consumer. This turns out to be rather important, for reasons discussed below.<\/p>\n\n
struct monitor {\n    union {\n        struct {\n            int r;\n            int w;\n        } pipe;\n        int fds[2];\n    };\n    uint8_t state;\n    uint8_t drain;\n    uint64_t gen;\n};\n<\/code><\/pre>\n\n
Creating a monitor is as you would expect:<\/p>\n\n
struct monitor *\nmonitor_create(void)\n{\n    struct monitor *m = malloc(sizeof *m);\n\n    if (m == NULL) {\n        return NULL;\n    }\n\n    m->state = STATE_WORKING;\n    if (pipe(m->fds) == -1) {\n        perror("pipe");\n        free(m);\n        return NULL;\n    }\n\n    return m;\n}\n<\/code><\/pre>\n\n
Destroying a monitor is a bit of a hassle because I wanted to actually handle errors with close(2)<\/code>. I had hoped this would make things idempotent such that you could retry destroying the thing when close(2)<\/code> failed, but I’m not sure what the caller would do here anyway. I might just ignore these errors anyway; I’m not sure whether it’s really feasible to get EIO<\/code>, and I’m not sure I care about any realistic ways to get EBADF<\/code>.<\/p>\n\n
bool\nmonitor_destroy(struct monitor *m)\n{\n    bool fail = false;\n\n    int r;\n    if (m->pipe.r != -1) {\n        do {\n            r = close(m->pipe.r);\n        } while (r == -1 && errno == EINTR);\n\n        if (r == -1) {\n            return false;\n        }\n    }\n\n    m->pipe.r = -1;\n\n    if (m->pipe.w != -1) {\n        do {\n            r = close(m->pipe.w);\n        } while (r == -1 && errno == EINTR);\n\n        if (r == -1) {\n            return false;\n        }\n    }\n\n    m->pipe.w = -1;\n\n    free(m);\n    return true;\n}\n<\/code><\/pre>\n\n
The actual cool stuff happens when we start thinking about how a waiter should behave when there’s no mutex involved. First, some function boilerplate:<\/p>\n\n
bool\nmonitor_wait(struct monitor *n, int timeout_ms)\n{\n    enum monitor_state cur;\n    bool rv = true;\n\n    assert(n != NULL);\n<\/code><\/pre>\n\n
When we enter this function, the first question we’re trying to answer is whether any work was produced while the worker wasn’t listening. Since we start out in STATE_WORKING<\/code>, this means that if we enter the function in that state, no work is pending for us to do, and so we should block. Alternatively, if we enter in STATE_PENDING<\/code>, that means work does<\/em> exist, and we shouldn’t bother blocking. If we guarantee that this function is the only function that can move the state to STATE_BLOCKED<\/code> and that it always exits in STATE_WORKING<\/code>, we can achieve this by doing a simple atomic compare-and-swap to move our state from STATE_WORKING<\/code> to STATE_BLOCKED<\/code>. If the CAS operation fails, we know that work is pending.<\/p>\n\n
    if (!ck_pr_cas_8(&m->state, STATE_WORKING, STATE_BLOCKED)) {\n        ck_pr_store_8(&m->state, STATE_WORKING);\n        ck_pr_fence_store();\n        goto drain;\n    }\n<\/code><\/pre>\n\n
Alternatively, when the CAS operation succeeds, that means no work was available for us to complete. In this case, we need to wait for a notification; we’ll receive this over the read side of our pipe(2)<\/code>. Because we may not want to wait forever, we allow the user to specify a timeout that we pass to poll(2)<\/code>.<\/p>\n\n
    struct pollfd pfd[1] = {{\n        .fd = m->pipe.r,\n        .events = POLLIN|POLLPRI,\n    }};\n\n    errno = 0;\n    int p = poll(pfd, 1, timeout);\n    if (p == -1 && errno == EINTR) {\n        errno = EAGAIN;\n        rv = false;\n        goto drain;\n    } else if (p == -1) {\n        perror("poll");\n        rv = false;\n        goto drain;\n    } else if (p == 0 || (pfd[0].revents & (POLLHUP|POLLERR)) != 0) {\n        rv = false;\n        goto drain;\n    }\n<\/code><\/pre>\n\n
You’ve noticed that we’re doing goto drain;<\/code> instead of ever returning anything so far. Some errors returned by syscalls we call can’t easily be dealt with inside this code, so it’s possible that we exit without any work to do anyway. However, this does mean that it’s possible that someone had to write to the pipe to wake us, and there’s data stuck there. We detect this condition and try to drain the pipe when it has been written to, even when we didn’t need to block for work. This does mean that there are cases where we might do syscalls when we didn’t need to, but this is only really likely to occur when there’s a timeout. If poll(2)<\/code> is told to block forever, this is only likely to happen on EINTR<\/code>, or for (probably catastrophic) errors. If everything goes well here, we clear our draining flag, which will only be set again if we block.<\/p>\n\n
drain:\n    if (ck_pr_load_8(&m->drain) == 1) {\n        ssize_t r; \n        uint8_t buf;\n        do {\n            p = poll(pfd, 1, 0);\n            if (p == 0) {\n                break;\n            } else if (p == -1 && errno == EINTR) {\n                continue;\n            } else if (p == -1) {\n                perror("drain poll");\n                rv = false;\n                goto ret;\n            } else if ((pfd[0].revents & (POLLHUP|POLLERR)) != 0) {\n                rv = false;\n                goto ret;\n            }\n\n            do {\n                r = read(m->pipe.r, &buf, sizeof buf)\n            } while (r == -1 && errno == EINTR);\n\n            if (r == -1) {\n                perror("read");\n                goto ret;\n            }\n        } while (p == 1);\n    }\n\n    ck_pr_store_8(&m->drain, 0);\n<\/code><\/pre>\n\n
Whenever we leave this function, we have to leave in STATE_WORKING<\/code>. However, we do return to the caller whether we left because work was available (in which case rv<\/code> is true<\/code>). At some point, I’d like to add an error API so the caller can figure out exactly what<\/em> failed when this function returns false<\/code> – so that’ll probably happen before this makes it to GitHub. In any case, transitioning to STATE_WORKING<\/code> whenever we return is really the big trick to making this wait-free.<\/p>\n\n
ret:\n    ck_pr_store_8(&m->state, STATE_WORKING);\n    ck_pr_fence_store();\n\n    return rv;\n}\n<\/code><\/pre>\n\n
Waking up a waiter is pretty simple. First, our function prologue…<\/p>\n\n
bool\nmonitor_wake(struct monitor *m)\n{\n    enum monitor_state cur;\n\n    assert(m != NULL);\n<\/code><\/pre>\n\n
Whenever we enter this function, it is always OK to set the state to STATE_PENDING<\/code>. We set this state using an atomic fetch-and-set operation, which allows us to move the state forward while also knowing what the previous state was. If the consumer is working, the state will either have already been set to STATE_PENDING<\/code> by another producer, or it will still be in STATE_WORKING<\/code>. In either case, we can be assured that when the consumer is ready, it will notice that we’ve produced work.<\/p>\n\n
    cur = ck_pr_fas_8(&m->state, STATE_PENDING);\n    if (cur == STATE_PENDING || cur == STATE_WORKING) {\n        ck_pr_fence_store();\n        return true;\n    }\n<\/code><\/pre>\n\n
When the consumer is blocked, however, the state we read will be STATE_BLOCKED<\/code>, and we will have set that to STATE_PENDING<\/code>. This turns out to be just fine. If the consumer is blocked indefinitely, it doesn’t care about what the value is until its woken up again via its pipe. If it’s blocked on a timeout, one of a few benign scenarios can happen:<\/p>\n\n
    \n
  1. The timeout occurs, and the consumer re-enters without any new work having been produced. The state will remain STATE_WORKING<\/code>, and the consumer will block again.<\/p><\/li>\n\n
  2. The timeout occurs, and a producer sets the state to STATE_PENDING<\/code> above after the consumer sets the state to STATE_WORKING<\/code>. The consumer re-enters and notices the new work. Nothing is written to the pipe, and nothing is read from the pipe.<\/p><\/li>\n\n
  3. The timeout occurs, and several producers set the state to STATE_PENDING<\/code> above before the consumer sets the state to STATE_WORKING<\/code>. The producers race on updating the generation count below; the winner will write to the pipe. The consumer sets the state to STATE_WORKING<\/code> and returns. No new work is produced. When it re-enters, it will set its state to STATE_BLOCKING<\/code> and immediately read the value written by the producer, picking up that work.<\/p><\/li>\n\n
  4. The same as above, but new work is produced before the consumer re-enters. The consumer enters and sees the state as STATE_PENDING<\/code>. It also notices that the drain flag is set, and drains the pipe. It will have picked up the work that was produced by the subsequent producers as well as the one that wrote to the pipe. Even if there is a race on the drain flag being set, the consumer will eventually notice the drain and clean it up.<\/p><\/li>\n<\/ol>\n\n
    None of these cases are terrible: in all cases, the consumer is eventually notified of the work. When an error occurs in a syscall, the producer is free to try again.<\/p>\n\n
        uint64_t gen = ck_pr_load_64(&m->gen);\n    if (!ck_pr_cas_8(&m->gen, gen, gen + 1)) {\n        ck_pr_fence_store();\n        return true;\n    }\n\n    ssize_t r;\n    do {\n        uint8_t buf = 0;\n        r = write(m->pipe.w, &buf, sizeof buf);\n    } while (r == -1 && errno == EINTR);\n\n    if (r == -1) {\n        return false;\n    }\n\n    ck_pr_store_8(&m->drain, 1);\n    ck_pr_fence_store();\n\n    return true;\n}\n<\/code><\/pre>\n\n
    The system described above is wait-free when work is available, which is really the only situation in which non-blocking behavior is helpful for a monitor, since the goal is to block when no work can be done. This is ideal for concurrent batching workloads of nearly any volume. The composability of the interface means that, even for low volume systems, there’s no chance that forgetting about implementation subtleties will result in missed notifications. And the wait-free nature means that with high-volume workloads, your producer’s fast-path only takes the hit of a single atomic operation on average, and you will almost never incur the penalty of a syscall.<\/p>\n\n
    This is a stepping stone for additional work.<\/span>\n The system presented here is really only appropriate for single-consumer workloads. In particular, multiple consumers sharing the same monitor would notice the monitor in STATE_BLOCKED<\/code> upon entry to the monitor_wait<\/code> function. This would cause them to erroneously think work was available, and result in spinning. Before I put this code on Github, I’d really like to solve the multiple-consumer problem. In particular, I’d like to have interfaces for signalling any consumer, signalling a specific consumer, and broadcasting to all consumers. I believe that this is all possible to do wait-free, assuming that the multiple-consumer versions simply slap an interface around managing N different monitors.<\/p>\n\n
    As a final note, I haven’t tested this code yet at all; it’s entirely possible that it’s broken. I’m not writing any C in my new job, which is a bit of a bummer. This also means that I don’t really have any use-cases for this code at the moment, making it a bit more difficult to test. I’ll of course spend a bit more time on this before publishing anything to Github.<\/p>\n"},{"title":"A Malloc Idiom in Go","link":"https:\/\/9vx.org\/post\/a-malloc-idiom-in-go\/","pubDate":"Thu, 22 Feb 2018 19:45:00 +0000","guid":"https:\/\/9vx.org\/post\/a-malloc-idiom-in-go\/","description":"
    I’m finally writing Go.<\/span>\n Although I was very active in contributing to the project for the first two-and-a-bit years, I’ve been pretty absent since 2012. Initially, I contributed because I am a fan of Plan 9 from Bell Labs<\/a> and of FreeBSD<\/a>. I loved the idea of having a usable, CSP-inspired language, but the initial release of Go only ran on Linux and OS X. I only had a FreeBSD system at the time, so I ported the compiler toolchain, runtime, and standard library to FreeBSD (with a good bit of education on debugging from Russ Cox<\/a>).<\/p>\n\n
    However, most of my work has been on low-latency systems software – mostly written in C – and FreeBSD has not been supported by any of my employers since 2007. Since I wasn’t really getting a chance to write new software in Go, and I eventually was no longer interested in the overhead of maintaining knowledge of an OS I was only using for fun, my use of and contributions to Go fell by the wayside.<\/p>\n\n
    Now that I work at Google, I’ve finally had some opportunities to write code in Go. While I still enjoy the language, there are a few experience report<\/a>-style things that ended up keeping me from using the language for the last 5-6 years, and things I currently find a little cumbersome. At the suggestion of some colleagues, I thought I’d document at least one of them.<\/p>\n\n
    One thing I miss<\/span>\n is something like C’s malloc<\/code> idiom. In C, allocated memory generally comes from calls to malloc<\/code>-family functions, which give you at least enough memory to fit at least one of the thing you want, or none at all. The idiom for doing this looks like:<\/p>\n\n
    T *p = malloc(sizeof *p);\n<\/code><\/pre>\n\n
    Notice how the type of p (T *<\/code>) only appears once. This line of code exploits how sizeof<\/code> behaves when its operand appears to dereference a pointer – which isn’t what happens, because the result of sizeof<\/code> mustKate Flavel<\/a> reminds me that this is not true for VLA types, which are so useless, I always forget. Such a type has its expression evaluated.<\/span> \n always be discernable at compile-time. What the language defines instead for this syntax is that sizeof<\/code> yields the size of the pointed-to object; it doesn’t turn into some runtime thing. The nice thing about the C idiom is that if I change the type represented by T<\/code>, I only change the type in the declaration or definition. No changes need to be made in the site(s) the pointer object is assigned to the result of the allocation. While the example above is trivial, consider a more complicated case where a structure member points to some type:<\/p>\n\n
    struct set {\n    size_t cap;\n    size_t nmemb;\n    int members[];\n}\n\nstruct set *\nset_create(size_t sz)\n{\n    struct set *n = malloc(sizeof *n + (sz * sizeof *n->members));\n    if (n == NULL) {\n        return NULL;\n    }\n\n    n->cap = sz;\n    n->nmemb = 0;\n\n    return n;\n}\n<\/code><\/pre>\n\n
    If, later, we want to change struct set<\/code> to support members with other than int<\/code>, we might change members<\/code> to a union, and add some enum<\/code> specifying the fields we wanted to add. And we can do this without changing any<\/em> of our code in set_create<\/code>.<\/p>\n\n
    This has bitten me in Go literally every time I’ve created some struct<\/code> type that embeds a thing that must be allocated, like a slice or a map. In Go, we are forced to repeat ourselves in expressing the type of thing we’d like to allocate, despite that type being well known to the compiler, and type inference is so idiomatic (consider expressions like a := b<\/code>), I sometimes have to dig to figure out what the type of a thing is. Let’s look at what’s involved in creating a struct with a map embedded:<\/p>\n\n
    type NamedMap struct {\n    name string\n    m    map[string]string\n}\n\nfunc NewNamedMap(name string) *NamedMap {\n    return &NamedMap{name: name, m: map[string]string{}}\n}\n<\/code><\/pre>\n\n
    We could alternatively use make<\/code> in NewNamedMap<\/code>, but we’re still left with return &NamedMap{name: name, m: make(map[string]string)}<\/code> – again, repeating the type. With well-thought-out code, there should be only one (additional) place where we need to specify the type to allocate it, but this still requires touching multiple points of code when types change. This bites me when I’m prototyping and haven’t thought fully about the amount of state I need to keep in a map. I’ve found myself needing to change map[string]string<\/code> to map[string]T<\/code> on several occasions, and it bugs me every time I have to change more than one line.<\/p>\n\n
    One could argue I should think more about what I need before writing code, and that’d be fair. I’d still counter that it’s not uncommon for additional state requirements to develop within a project’s lifetime, like in my set<\/code> example above. It’s also possible that constraints of a system change over time such that a type that was perfectly fine initially eventually becomes unusable. In Go, the set<\/code> structure might look like this:<\/p>\n\n
    type Set struct {\n    nmemb   int\n    cap     int\n    members []int\n}\n\nfunc NewSet(sz int) *Set {\n    return &Set{cap: sz, members: []int{}}\n}\n<\/code><\/pre>\n\n
    You might ask why I’m not just using a slice, and the answer is that this is a short example for demonstrating the problem. Anyway, we may later want to support a slice of different types, and we run into the same problem we had previously. With a slice, we might be able to ignore the initialization, assuming we only ever read from the slice after appending to it. With e.g. maps and channels, the situation is more complicated because we really must<\/em> allocate before use. So it’s not that uncommon to repeat type information somewhere<\/em>.<\/p>\n\n
    I’m not entirely sure how to go about solving this. For compound literals, it perhaps it would be possible to add a syntax like:<\/p>\n\n
    return &Set{cap: sz, members: Set.members{}}\n<\/code><\/pre>\n\n
    If you had a pointer to a slice for whatever reason:<\/p>\n\n
    return &Set{cap: sz, members: &Set.members{}}\n<\/code><\/pre>\n\n
    I don’t know if I like these compound literal syntaxes. Fixing make<\/code> to support the sizeof<\/code> behavior in C feels more expressive and obvious:<\/p>\n\n
    return &Set{cap: sz, members: make(Set.members)}\n<\/code><\/pre>\n\n
    But maybe this is just me programming in C for too long.<\/p>\n\n
    I don’t know that it is useful or valuable to change new<\/code> to have similar behavior; I suspect its use is uncommon enough that it is not. In any case, it does seem clear to me that this would reduce the overhead of refactoring software, and of writing new software.<\/p>\n\n
    This particular issue<\/span>\n isn’t so bad, but fixing it would definitely make my life better. Go is already much more expressive than C in many cases, and than C++ (which I still can’t stand to learn). I think if support for type inference in allocation was added to the language, it’d become more appealing to C systems programmers who are still holding out for reasons other than GC. (Personally, I’d also like to see a better story around support for system-level concurrency, but that’s best left for a separate post.)<\/p>\n\n
    I’ve edited this article<\/span>\n to fix an error in the C example. When I originally wrote the example, the struct set<\/code> did not make use of flexible array members. Anmol Sethi wrote to ask about this feature, and pointed out that I was erroneously allocating and assigning to the FAM again. I had forgotten to remove that code. Oops.<\/p>\n"},{"title":"Rote Learning in CS","link":"https:\/\/9vx.org\/post\/rote-learning-in-cs\/","pubDate":"Sun, 20 Aug 2017 00:59:00 +0000","guid":"https:\/\/9vx.org\/post\/rote-learning-in-cs\/","description":"
    Does rote learning<\/span>\n have a place in computer science education? In my talk<\/a> at QCon, I stated:<\/p>\n\n
    \n
    [Learning through discovery] is so much better than this typical, learn-by-rote method of teaching that we all sort of think of as teaching…<\/p>\n<\/blockquote>\n\n
    I just read a blog post<\/a> by Zed Shaw suggesting that rote learning is basically required to become skillful in the field. I came into the article wholly skeptical – I haven’t always been a fan of what Zed writes, and my own research seemed to contradict what I thought the thesis would be. After reading it, I maintain that rote learning is useless as a teaching strategy. So I was surprised to find that I actually agreed with large parts of Zed’s post. What gives?<\/p>\n\n
    Consider for instance this sentence from Zed:<\/p>\n\n
    \n
    The problem with [the fear of rote learning] is that nearly every creative thing you do requires rote practice.<\/p>\n<\/blockquote>\n\n
    Now, let’s compare that sentence to what I said:<\/p>\n\n
    \n
    To gain skill, we have to practice at the edge of our ability.<\/p>\n<\/blockquote>\n\n
    Zed and I agree on this idea that practice is required to gain skill – although if you bring in some context, it might not appear that we do. We coincidentally both contextualize this around learning guitar. Zed writes, “The idea that I\u2019m going to learn the major scale on a guitar by just learning the concept of a major scale is laughable.” I think it’s entirely possible for someone to conclude I disagree based on what I said:<\/p>\n\n
    \n
    [F]or example, I\u2019ve played guitar for twenty-five years. And I suck.<\/p>\n\n
    And fundamentally, this is because I rarely challenge myself. I learn some song or – the coolest thing I ever learned was some sweep picking pattern, and I can\u2019t even do it very fast. But I rarely challenge myself to do something new. We don\u2019t get better at guitar by playing one chord or plucking one string time after time after time. In fact, I don\u2019t think that would even help us master that chord because you\u2019ve got to do transitions in and out of it, et cetera.<\/p>\n<\/blockquote>\n\n
    While it sounds like we’re saying very different things here, I don’t think we are. Undoubtedly, mastery of guitar requires repetition. You’re going to play scales a lot. I mean, a lot<\/em>. I’ve played them a lot<\/em>, and I’m still terrible. I’m so bad, I have exactly 0 positions memorized. I could probably noodle out major\/minor\/blues scales given some time, but it wouldn’t be pretty. Why? Part of this is that my practice is inconsistent; I’ll speak more about what kinds of practice are effective in the closing paragraphs.<\/p>\n\n
    In my own context, I seem to be saying that practice doesn’t help you learn guitar at all. But this isn’t at all what I meant – I really failed to communicate clearly. I meant that playing guitar isn’t really a single skill. You aren’t good at guitar simply when you can play a single chord or a single song. If you practice one single thing until your death, playing that one thing absolutely perfectly every time, you’ve mastered exactly 0% of guitar. The greats in guitar have skills outside the realm of guitar playing as well as a mastery of the instrument. They can improvise. They can play many different styles of music. They’re fundamentally diverse<\/em> in their skillset.<\/p>\n\n
    Zed goes on<\/span>\n to tie his ideas into CS education. He writes:<\/p>\n\n
    \n
    If you\u2019re imagining yourself at 12 trying to learn to code, then I\u2019m betting you had either a book or website with code that you copied and made work. This should just be how we start people in programming, and not the current method of conceptual “weed out” classes.<\/p>\n<\/blockquote>\n\n
    I identify closely with this hypothetical. As a kid, I remember copying programs out of books on QBasic and making simple changes to try to understand how things worked. I do think we need to be careful about making an assumption this is the only way people could successfully learn to program – most of the people I know who learned this way were white, middle-class boys. I would also note that I was a pretty terrible programmer for a really long time until I had people mentoring me, and I thought I was a much better programmer than I actually was until years after I had people who were mentoring me. I’m not entirely sure that I agree this is an effective way to start. It took me about 10 years of professional programming to get anywhere close to being something that most folks would call good, and I’m positive that a more formal grounding in computer science would have helped.<\/p>\n\n
    I suggest this caution specifically because I’m not a fan of Zed’s LCTHW book for many<\/a> reasons<\/a>. The book no longer appears to be freely available, but in a previous iteration, it contained a section called “Make is your new Python.” While factually incorrect, this section title is an example of Meaningful Learning, the idea that we gain new knowledge by building and expanding upon things we already know. I generally think this is a good idea; it’s why I suggest using mail as a metaphor when discussing indirection in C<\/a>.<\/p>\n\n
    But Tim also points out, “What if the reader has no experience with Python at all, and doesn’t even know what it is? The author is making an assumption here that not only is the reader a novice in terms of C, but Python also.” I think this is a fair criticism. This assumption of knowledge is why, although I identify with the hypothetical scenario of learning to program at age 12, I think we should be careful in assuming its validity for the purposes of teaching others. If that knowledge is not foundational (and is potentially exclusive), we may continue with the shitty status quo in diversity in our field. This is why I also said we need to operate from curiosity: we have to figure out where it is to start teaching folks.<\/p>\n\n
    That all said, I agree that the “weed out” classes are problematic: they challenge people without promoting an incremental self-theory. They’re basically designed to be anti-diversity machines. They’re designed to be this sort of pedagogical manifestation of laziness by instructors who don’t actually want to help people learn.<\/p>\n\n
    Anyway,<\/span>\n this bit from Zed’s post is what really spurred me to write this post:<\/p>\n\n
    \n
    I think the main reason why this is ignored or vilified in CS is the same reason that most programmers simply can\u2019t teach: They are so far removed from their beginner experience that they forget that they actually learned to code via rote learning.<\/p>\n<\/blockquote>\n\n
    Compare again this statement to what I said about the pedagogy of debugging:<\/p>\n\n
    \n
    As we gain experience, we forget what it was like to learn a thing, and we forget even more the more experience we gain.<\/p>\n<\/blockquote>\n\n
    Zed doesn’t appear to be stating that repetitive practice should be the only way that CS education happens, but rather that it should be explored as part of it.<\/p>\n\n
    Is repeated practice<\/span>\n entirely absent in CS education? Programs that are heavy on lab projects effectively promote such practice by having students use their new knowledge to solve multiple different problems. I think this effectively merges the repetitive practice aspect with the creative learning aspect. So I don’t agree entirely that practice is globally “ignored or vilified,” we just need to look at how it actually manifests in some curricula.<\/p>\n\n
    And how does it manifest? I was talking to a high school programming class earlier this year. As part of my presentation, I illustrated how problems that don’t at all look similar actually end up being the same problem. The example I gave started with Scrabble: we need some kind of dictionary structure to look up words. It turns out this is exactly the same problem that you face if you want to implement key-based purging in a content delivery network. By understanding how to handle a Scrabble dictionary, I was able to implement the Surrogate Keys feature at Fastly.<\/p>\n\n
    So we find repetition in solving lots of related problems; this is why programs heavy in lab work can be successful. And we make that less boring (which is the main complaint against this pedagogical practice of rote learning, outside of its total lack of efficacy) by being creative in the tasks that we have folks solve. This seems similar to Zed’s suggestion to engage in “…training that involves a mixture of rote (scales, chords, ear training) followed by copying and modifying (learn a song and try to improvise).”<\/p>\n\n
    Was I off-base<\/span>\n in my talk? Do I agree with Zed? I wasn’t off-base, and I think I agree with Zed, but the reason is because of semantics. You may’ve noticed that I keep saying “repetitive practice” and not “rote.” The word “rote” means a form of mechanical, habitual, unthinking practice. This isn’t what I’m talking about. It doesn’t seem to be what Zed is talking about. I really don’t agree that rote learning works, and I don’t think he actually does either. It doesn’t jive with his own examples of learning guitar and painting: skills he is motivated to practice deliberately.<\/p>\n\n
    When I say “learn-by-rote,” I really do mean “rote” in this mechanical, mindless context. This is not at all effective; there’s significant, neurological evidence<\/a> to support the idea that it is ineffectual. As a quick aside, a fantastic segment<\/a> earlier this year on KQED’s Forum program explored the problems with lecture-based education, in an interview with Nobel laureate Carl Wieman. (Shameless self-promotion: I called into this program at 16:49.) I think it’s relevant because lectures are basically unengaging, mechanical, habitual, unthinking ways of throwing entirely too much information at a person and having them remember none of it.<\/p>\n\n
    Finally, it’s not just repetition in practice that is required to gain skill in a craft. Peter Norvig’s Teach Yourself Programming in Ten Years<\/a> cites numerous papers in explaining that deliberation<\/em> is also required:<\/p>\n\n
    \n
    The key is deliberative practice: not just doing it again and again, but challenging yourself with a task that is just beyond your current ability, trying it, analyzing your performance while and after doing it, and correcting any mistakes. Then repeat. And repeat again. There appear to be no real shortcuts…<\/p>\n<\/blockquote>\n\n
    Based on his post, I don’t think Zed actually believes that repetitive, undirected, automatic practice builds skill. Past experience reading Zed’s responses to people who argue with him about semantics haven’t been pleasant, so I don’t really want to engage in a semantic argument. However, I do wonder whether this incorrect terminology is the reason he thinks that “rote learning” is such a taboo in education. Ultimately, I think we’re both talking about the same thing, and I agree that it works.<\/p>\n\n
    Edit:<\/span>\n I somewhat tongue-in-cheek tagged Zed when posting this article on Twitter. He replied and, far from a flame war, we had a pretty interesting discussion<\/a>. Zed does mean “mindless” when he says “rote” – but it’s more of a Buddhist, meditative interpretation of the word. I think I know what he means; I can understand how a sort of culturally different view on “mindlessness” might be beneficial in learning. Western culture has a romance with the idea of smart, and smart means thinking. As such, learning is often stressful. This makes the sort of meditative version of “mindlessness” difficult to attain, if at all attainable given the stress on children to perform academically. This was definitely an area of unconscious bias for me, and I’m happy to have learned.<\/p>\n"},{"title":"Go Go Gadget Co-Ro","link":"https:\/\/9vx.org\/post\/go-go-gadget-co-ro\/","pubDate":"Mon, 05 Jun 2017 16:00:00 +0000","guid":"https:\/\/9vx.org\/post\/go-go-gadget-co-ro\/","description":"
    I like coroutines.<\/span>\n The other day on my team’s Slack channel, someone introduced Adam Dunkels’ protothreads<\/a>. Either I hadn’t seen these before or had forgotten them, but it looked cool. A short conversation ensued, and at some point a colleague offered:<\/p>\n\n
    \n
    i can’t ever decide if i like [coroutines] or not. so ignoring that, i think there’s a decent heuristic: something like tatham\u2019s coroutines will provoke a conversation every time you use it. i figure if something produces so much discussion, it’s certainly non-obvious to understand<\/p>\n<\/blockquote>\n\n
    This colleague is an excellent software engineer and computer scientist. She’s definitely familiar enough with threads, and so this comment struck me as odd. Programs implemented using threads are, in my view, much more difficult to understand than those implemented atop coroutines. If you can understand how threads work, you can understand coroutines. In this post, I’d like to describe what threads and coroutines actually are, the similarities and differences between them, and why I think coroutines provoke so much discussion.<\/p>\n\n
    We’ll start out<\/span>\n with some definitions. What are threads; what are coroutines? The easiest way I find to introduce concepts is to present definitions based on the properties of the things the word describes. So we’ll start at a low level, and work our way up.<\/p>\n\n
    A computer is some hardware that provides state, and can execute at least one program. A program is simply a specification for how state is changed over time. A program executed by a computer is called a process. The context in which the program is executed is called a “thread,” and every process contains at least one thread. Processes containing multiple threads may see those threads execute in parallel. The state provided by the computer may or may not be shared between processes, but within a process, this state is shared. Threads are also typically given some “protected” state: state intended for use by the thread, but not expressly prohibited from sharing between threads.<\/p>\n\n
    As a program executes, state changes, and time progresses. These tuples of (execution unit, computer state)<\/code> may be arranged over the vector of time to describe what is called an “execution history.” Each thread is a container of an independent 
    So we have this hierarchy of “computer” and “process” and “thread” – but all of these things are just containers of independent execution histories. The computer has one or more processors that execute instructions to modify state over time. It executes processes that are comprised of one or more execution histories, which also describes what a thread is. If these are all the same thing, why in the world do we have different words for them?<\/p>\n\n
    Each word describes the same thing, but at a different level of abstraction. This abstraction is great, because it provides us a means to approximate a new Turing machine, which means we get a little extra state to do something useful each time we abstract. So although we usually say that a computer doesn’t contain another computer, we have VMs! Processes don’t contain other processes, but they do contain threads – which are effectively little processes. And threads don’t contain threads. But a thread comprising Turing-complete execution units running on an abstraction of a Turing machine can obviously implement another Turing machine. Therefore, a thread may contain another thread, just not at the same abstraction level!<\/p>\n\n
    Our computer is our base-level abstraction for a Turing machine (we don’t actually have one of these because nobody has infinite tape). Our process is another abstraction that allows us to pretend our computer runs multiple Turing machines at once: this is useful because we don’t want the computation of one of them to influence the computation of another. And threads exist for effectively the same reason – they’re useful for multiplexing the same kind of computation on discrete bits of state.<\/p>\n\n
    Coroutines are what we get when we define an abstraction on top of threads. In a novel way, coroutines are<\/em> threads, in the same way that threads are processes and processes are computers – all abstractions for a Turing machine. Coroutines recognize that the fundamental unit of execution is this tuple of (execution unit, computer state)<\/code>, that multiple vectors of these can execute independently with an almost arbitrary interleaving. Furthermore, since they exist at an abstraction above<\/em> threads, that abstraction should be providing something that makes them easier to understand.<\/p>\n\n
    To understand why we might like this abstraction, let’s take a deeper look at some of the problems with writing purely threaded software.<\/p>\n\n
    Threads provide an abstraction<\/span>\n above the machine or process level, representing multiple independent execution histories. This abstraction became popular in recent years as multicore and manycore systems became ubiquitous. Threads existed even before such systems were terribly common, so what is it about the abstraction that makes them useful?<\/p>\n\n
    Many tasks we solve with software are repetitive. We rarely try to solve tasks that involve taking a single specific input and producing a single specific output. Such a transformation only needs to occur once, and we only write software for something this specific when the input and \/ or output sets are large. In network servers, for example, requests from a client arrive that direct the software to do some work. These requests will all be very similar, and so it makes sense for us to use the same codepath to process each request. Additionally, network requests tend to take time to satisfy, and request arrival often follows a Poisson distribution which (practically) means that requests will usually occur in clusters.<\/p>\n\n
    Let’s say we’re serving requests for image thumbnails. This means we’re serving a lot of small content across a potentially large library. For simplicitly, let’s assume a each request can only result in a single thumbnail, and every thumbnail is equally likely to be requested. A random disk seek on a fast spinning disk takes about 10 milliseconds. This seek time dominates the time we spend on the request, so we can approximate the request time as 10ms + \u03b5. The simplest solution, to serve each request from a single thread of execution as it arrives, has two consequences:<\/p>\n\n
      \n
    1. The oldest request accumulates the latency of every request in front of it.<\/li>\n
    2. We are maximally able to serve 100 requests per second.<\/li>\n<\/ol>\n\n
      Because request arrival rate tends to follow a Poisson distribution, it’s likely that a large number of requests experience the negative of the first point. And if we receive over 100 requests within the same 10 milliseconds, the folks with the latest requests will wait over a second for their thumbnail. In fact, even if we add more disks in a RAID configuration, we can never beat this problem, if we insist on serving requests serially.<\/p>\n\n
      The abstraction of the thread allows us to take advantage of parallelism in hardware. Let’s now assume we have a RAID1+0 configuration (mirrored and striped), and so our objects are stored twice, and distributed across many disks. Adding threads allows us to amortize the cost of disk seeks over multiple disks, reducing the probable latency for any individual request. Because the time for processing the request (compared to the disk seek) is \u03b5, our mental model now says that the probability of any two requests requiring the same disk is proportional to the number of disks. Let’s say that we have 10 disks. Now with 100 clustered requests, it’s entirely possible we can serve all of them within 100ms instead of 1s.<\/p>\n\n
      This is fantastic for read-only and read-mostly workloads. However, when requests require us to mutate state, things go downhill quickly. As an abstraction on top of some state (either a machine or a process), threads have access to everything in the parent’s state. Traditionally, this has meant that threads are able to share stateful things like memory that tend to be protected between instances of their parent abstraction. This is problematic when the shared state is mutable. Because multiple threads of execution may occur in parallel, some actions may occur at the same time. It’s well known that conflicts occur when multiple threads attempt to access some shared mutable state, and one of those accesses is a write. Consider the following code:<\/p>\n\n
      #include <threads.h>\n#include <stdint.h>\n#include <stdio.h>\n\nstatic volatile uint64_t counter;\n#define NTHRD 2\n\nthrd_start_t\nthread(void *arg)\n{\n        (void) arg;\n        for (int i = 0; i < 1000000; i++) {\n            counter++;\n        }\n        return NULL;\n}\n\nint\nmain(void)\n{\n        thrd_t tds[NTHRD];\n        for (int i = 0; i < NTHRD; i++) {\n                if (thrd_create(&tds[i], thread, NULL) != thrd_success) {\n                        return -1;\n                }\n        }\n        for (int i = 0; i < NTHRD; i++) {\n                if (thrd_join(tds[i], NULL) != thrd_success) {\n                        return -1;\n                }\n        }\n        printf("counter: %" PRIu64 "\\n", counter);\n}\n<\/code><\/pre>\n\n
      In this example, the resulting value of counter is unclear.
        \n
      1. Read the value of the counter from memory.<\/li>\n
      2. Modify the value of the counter by adding one to the value read.<\/li>\n
      3. Write the value of the counter back to memory.<\/li>\n<\/ol>\n\n
        Even though this read-modify-write operation is expressed in C (and possibly at the hardware level) as a single “increment” instruction, it actually decomposes into these three operations. And each of them could happen interleaved with similar operations executed by other threads. If our first and second threads consistently execute each operation in lock-step with each other (112233), each loop only increments the counter by one despite having run twice.<\/p>\n\n
        The reason that this can happen has to do partially with how computer hardware handles memory accesses, and partially due to how threads are scheduled (by the machine or an OS scheduler). Because reads from and writes to memory are expensive (when compared to the cost of performing some operations in registers), processors tend to cache these reads and writes. On a system with multiple processors, with each thread simultaneously executing on a different processor, the threads will be operating on cached copies of the memory (despite our use of volatile<\/code>). If a processor did not provide an increment instruction, the assembly for counter++<\/code> might look like:<\/p>\n\n
        load    %reg, (counter) ; load counter from memory into register\nadd     %reg, 1         ; add 1 to value in register\nstore   (counter), %reg ; store register back into memory\n<\/code><\/pre>\n\n
        One property of threads is that they’re often pre-emptable. This means that whatever is running the threads (whether the machine or an OS scheduler) may stop the process from executing while it is in-between instructions. In the case where our increment is implemented as three separate instructions, the thread could be paused between any of them. This pre-emption further exacerbates the problem of the data race.<\/p>\n\n
        Typically, we solve this problem by using synchronization constructs. In this case, the hardware might provide us an instruction to do a synchronized increment. In other cases, we might need to use a mutex (or other serializing construct) to ensure the correctness of our implementation of the “increment a counter” protocol. This has its own issues (many of which are presented well in Samy Al Bahra’s article<\/a> on non-blocking synchronization) that I might cover in a separate blog post.<\/p>\n\n
        For now, let’s go back to our thumbnail serving example. Implemented in somewhat pseudocode, we might have the following.<\/p>\n\n
        thrd_start_t\nserve_thumbnails(void *arg)\n{\n        while (true) {\n                struct request_state *state = get_actionable_request();\n\n                if (read_request(state) ||\n                    get_thumbnail(state) ||\n                    send_response(state)) {\n                        handle_error(state);\n                }\n        }\n        cleanup(state);\n}\n<\/code><\/pre>\n\n
        Ignoring caching, pre-emption is the primary effect that allows a threaded implementation of our thumbnail service to operate efficiently. However, if we take a deeper look, read_request<\/code> and send_response<\/code> are also potentially latency-intensive. Our 10ms + \u03b5 estimate wasn’t exactly correct because although get_thumbnail<\/code> is approximately bounded at 10ms, it could take us dozens or hundreds of milliseconds to read the request and send the response. During that time, the thread is consuming resources (every abstraction requires some amount of state to represent), while not providing any computational benefit. Schedulers recognize this and run something else during the time the thread is blocked waiting on input, or waiting for the output side to become ready for more data.<\/p>\n\n
        Unfortunately, a thread consumes resources (at least memory) even when it is not being executed. Even with threads being lighter-weight than processes, their mere existence isn’t necessarily cheap when you have a lot of them. Additionally, the cost to “context switch” between threads of execution is usually somewhat expensive. Context switches are pretty awful because during that time, the machine isn’t doing anything particularly useful. It is just changing the environment around a bit to switch to a different Turing machine abstraction. When we’re doing I\/O intensive operations, these operations might occur while we still have time to run on the scheduler (that is, we would not have been pre-empted if we had not done some I\/O task).<\/p>\n\n
        To address this problem, we can make our I\/O operations asynchronous. In the example above, we’ve already created an abstraction for our request state. We can use that to create a finite state machine to know what to execute at any point.<\/p>\n\n
        thrd_start_t\nserve_thumbnails(void *arg)\n{\n        while (true) {\nnext_state:\n                struct request_state *state = get_actionable_request();\n\n                switch (state->fsm_state) {\n                case STATE_READ_REQUEST:\n                        switch (read_request(state)) {\n                        case BLOCKED:\n                                queue_for_input(state);\n                                goto next_state;\n                        case ERROR:\n                                state->fsm_state = STATE_ERROR;\n                                break;\n                        case SUCCESS:\n                                state->fsm_state = STATE_GET_THUMBNAIL;\n                                break;\n                        }\n                        break;\n\n                case STATE_GET_THUMBNAIL:\n                        switch (get_thumbnail(state)) {\n                        case BLOCKED:\n                                queue_for_input(state);\n                                goto next_state;\n                        case ERROR:\n                                state->fsm_state = STATE_ERROR;\n                                break;\n                        case SUCCESS:\n                                state->fsm_state = STATE_SEND_RESPONSE;\n                                break;\n                        }\n                        break;\n\n                case STATE_SEND_RESPONSE:\n                        switch (send_response(state)) {\n                        case BLOCKED:\n                                queue_for_output(state);\n                                goto next_state;\n                        case ERROR:\n                                state->fsm_state = STATE_ERROR;\n                                break;\n                        case SUCCESS:\n                                state->fsm_state = STATE_DONE;\n                                break;\n                        }\n                        break;\n\n                case STATE_ERROR:\n                        handle_error(state);\n                        break;\n\n                case STATE_DONE:\n                        cleanup(state);\n                        break;\n                }\n        }\n\n        return NULL;\n}\n<\/code><\/pre>\n\n
        This works, but it’s not particularly easy to read. We no longer have a linear flow of code. There’s tons of branching, and we have to be very careful to advance our state correctly to avoid cases where we might cause a loop in our state graph, fail to handle a state, or something else bad. There are a couple of opportunities to avoid repetition in this code, but that won’t drastically improve its readability. The property we have lost here is called “composability.”<\/p>\n\n
        Composability is a property<\/span>\n of a system or API that describes how easy it is to implement, consume, and understand that system or interface. Examples of composable interfaces include functions for working with common data structures like queues, hash tables, and graphs of various shapes. These structures tend to have add<\/code>, remove<\/code>, and find<\/code> interfaces. We can “compose” these interfaces with relative ease to get correct structures that efficiently manage and represent our data sets or program flow.<\/p>\n\n
        In our example above, we see that C isn’t the best language we could possibly use to compose a finite state automaton. When a language, system, or interface becomes too difficult to put together, too difficult to read, or too difficult to understand, we say that it is not composable. We generally solve this by introducing additional layer of abstraction. For example, the wonderful library and tools provided by libfsm<\/a> give us several pleasant means to compose finite state automata through languages, tools, and libraries.<\/p>\n\n
        Let’s look at our original example again:<\/p>\n\n
        thrd_start_t\nserve_thumbnails(void *arg)\n{\n        while (true) {\n                struct request_state *state = get_actionable_request();\n\n                if (read_request(state) ||\n                    get_thumbnail(state) ||\n                    send_response(state)) {\n                        handle_error(state);\n                }\n        }\n        cleanup(state);\n}\n<\/code><\/pre>\n\n
        In an ideal world, this would be expressive enough to encode our state transitions. If read_request<\/code> would block, it would be nice if it could simply stop its execution and retry exactly where it left off, instead of us having to design an entire machine to do that. The finite state machine we created does exactly this, but it isn’t a very nice way to abstract the problem.<\/p>\n\n
        This is precisely where coroutines help.<\/span>\n The point of the abstraction of coroutines is to offload much of this complexity into the scheduler. Unlike threads, coroutines are not pre-empted; they are instead “cooperatively” scheduled. That is to say, a coroutine (unlike a thread) runs until it explicitly yields its runtime to another coroutine. Like any abstraction, it requires some additional state, but this state is relatively small. Of course, we still have to “context switch” between coroutines, but this is a much simpler and faster operation than having the OS context switch between threads.<\/p>\n\n
        A coroutine doesn’t typically have a “protected” state like a thread does. When this is useful, this protected state is usually much smaller than that of a thread. On many machine implementations, coroutines only require a subset of machine registers to be saved and restored. Coroutines exist in a very small number of different states: running, runnable, or blocked on a resource. Because of this, multiple coroutines may execute concurrently on a single thread.<\/p>\n\n
        Concurrency and parallelism are often confused to mean the same thing. Parallelism describes a system in which multiple independent execution histories may occur at literally the same time. This is the case with threads, where (for example) parallel independent execution histories may be executed simultaneously on multiple processors. Concurrency is weaker: it simply states that multiple independent execution histories may be interleaved such that they appear to be occurring at the same time. This is roughly the same feature operating system schedulers provide to give the illusion of multitasking many processes on the same system.<\/p>\n\n
        Using coroutines, our original example still works just fine. We might have multiple threads of execution (so that we can utilize our hardware in parallel) with each thread multiplexing several coroutines. In this world, the complexity moves around, but it is tightly contained in a few spots. This lends itself nicely to composability. As an example, our read_request<\/code> function might look like this:<\/p>\n\n
        int\nread_request(struct request_state *state)\n{\n        ssize_t r;\n\n        do {\n                r = read(state->fd, state->buf, state->bufsiz);\n                if (r == -1) {\n                        if (errno == EAGAIN) {\n                                yield_on_read(state);\n                        } else {\n                                return 1;\n                        }\n                }\n        } while (request_not_complete(state));\n\n        return 0;\n}\n<\/code><\/pre>\n\n
        The special function yield_on_read<\/code> is provided by the coroutine implementation. It snapshots the current machine state of the coroutine, updates the coroutine’s state to become runnable when more data exists on the file descriptor, and returns to the scheduler with a lightweight “context switch.” Later on, when data arrives (or possibly when a timeout occurs) on that file descriptor, the scheduler returns directly into the coroutine where it left off (again with this lightweight “context switch”). The coroutine continues by testing request_not_complete<\/code> (which will be true because the request wasn’t completed the previous time), and will re-execute the read<\/code> system call, this time guaranteed to get at least some data. In the time between this, some other coroutines may have executed. Similar things could be implemented for get_thumbnail<\/code> and send_response<\/code>. Indeed, we could abstract out all blocking I\/O operations into a single API such that other coroutines don’t need to implement their own state handling in a loop like we’ve done here.<\/p>\n\n
        How do coroutines snapshot the machine state? This is usually done using functions like makecontext<\/code> and swapcontext<\/code>, or getcontext<\/code> and setcontext<\/code>. As previously mentioned, some machines do not require all execution context to be saved. In such cases, some assembly code to read or restore the minimal required machine state is written.<\/p>\n\n
        Why aren’t coroutines more widely used?<\/span>\n Although the resulting code often ends up being more composable, and although highly concurrent workloads like network servers tend to benefit from such an expression, there’s a fair amount of overhead involved in implementing the underlying API and scheduling. Threads are already relatively lightweight (at least compared to processes) and already have a scheduler to run them. Much of the time, the overhead of implementing APIs and scheduling is not seen as justification for the resulting composability and scalability win.<\/p>\n\n
        In my opinion, this is the reason that coroutines provoke a discussion. It looks like a difficult technical discussion about how complicated writing a scheduler is, but really it boils down to a discussion about where and how to hide complexity. It seems clear to me that the bulk of code written using coroutines is less complicated, and the composability benefits are huge.<\/p>\n\n
        But in some cases, coroutines have significant drawbacks. When part of a cooperatively scheduled task has high computational overhead, do we yield before performing the computation? All other coroutines are blocked while one is executing, so we need to be aware of this tradeoff. If we want to take advantage of parallelism with coroutines, we also face more challenges. If several coroutines are working with some shared state and<\/em> those coroutines can yield in the middle of something that should logically be a critical section, we still need locking. Problematically, if we also have added multiple threads on which to execute coroutines, this means that we need to introduce some kind of locking protocol. Several more corner cases like these exist, and none are particularly trivial to solve. All add complexity to the scheduler and have runtime overhead.<\/p>\n\n
        It’s frequently safer and easier to go with the devil you know. And so we have long conversations (and blog posts) about coroutines instead of actually using them.<\/p>\n\n
        That all said, the popularity and reasonably good performance of the Go programming language in the systems and network programming space shows that it is possible to get coroutines right.<\/p>\n\n
        If you’re interested in implementations or adopting coroutines into your work, there are several great resources. My two favorites (apart from homegrown implementations) are libmill<\/a>, which is a library that provides Go-style concurrency for C, and libtask<\/a> provides coroutines as well as appropriate non-blocking file and network I\/O interfaces as a portable version of Plan 9’s concurrent programming interface.<\/p>\n\n
        I’ve avoided covering<\/span>\n numerous related topics. In particular, message passing in a threaded environment combines the simplicity of existing pre-empted thread scheduling with the composability of data structure operations. In fact, message passing is a fantastic way to implement notification queues for systems built on either threads or coroutines (or both). Like any abstraction, the devil is in the details. Message passing in threaded environments moves the complexity away from the scheduler and the threading code, but into the algorithms and data structures used to pass messages. In environments with extremely high thread counts, this can be a non-starter when using an unmanaged language like C. Anyway – it’s an idea for another blog post!<\/p>\n"},{"title":"Discovering Undefined Behavior","link":"https:\/\/9vx.org\/post\/discovering-undefined-behavior\/","pubDate":"Mon, 17 Apr 2017 17:00:00 +0000","guid":"https:\/\/9vx.org\/post\/discovering-undefined-behavior\/","description":"
        The C language is infamous<\/span>\n for undefined behavior. Fantastic, multi-part articles by Chris Lattner<\/a> and John Regehr<\/a> have made their rounds over the years, describing several instances of undefined behavior. In particular, they describe how compilers are allowed to assume it does not occur in correct C programs, and therefore make optimizations that seem to remove correctness from programs. The intent of these posts seems to be to educate people that they should be aware of what consitutes undefined behavior.<\/p>\n\n
        But this isn’t always so easy.<\/p>\n\n
        When does<\/span>\n undefined behavior occur? The C11 draft<\/a> formalizes this in \u00a74p2:<\/p>\n\n
        \n
        If a \u2018\u2018shall\u2019\u2019 or \u2018\u2018shall not\u2019\u2019 requirement that appears outside of a constraint or runtime-constraint is violated, the behavior is undefined. Undefined behavior is otherwise indicated in this International Standard by the words \u2018\u2018undefined behavior\u2019\u2019 or by the omission of any explicit definition of behavior. There is no difference in emphasis among these three; they all describe \u2018\u2018behavior that is undefined\u2019\u2019.<\/p>\n<\/blockquote>\n\n
        Of note should be the text stating that undefined behavior occurs when “any explicit definition of behavior” is omitted. What happens if you try to run a C compiler on cheese? This is undefined by omission, and therefore this implicit undefined behavior becomes explicit.<\/p>\n\n
        Maybe that doesn’t sound so bad, but it results in tons of confusion amongst programmers and compiler implementors. For example, did you know that printf("%p", (void *)NULL);<\/code> yields undefined behavior? This is because \u00a77.1.4p1 states, “If an argument to a function has … a null pointer, … the behavior is undefined.” Later on, in the specification of the printf<\/code> family, no such exception appears.<\/p>\n\n
        This might seem innocuous, but it’s maybe more obviously problematic if you consider a case like printf("%s\\n", s);<\/code>, where s<\/code> is NULL<\/code>. Printf<\/code> functions have relatively high overhead: they’re variadic functions, which tend to be more expensive to call. They also contain format strings, which must be parsed at runtime. Some compilers therefore optimize calls like printf("%s\\n", s);<\/code> to puts(s);<\/code>, which are functionally equivalent. This is allowed because of the statement in \u00a77.1.4p1, but pisses folks off because most printf<\/code> implementations will output something like (nil)<\/code> if s<\/code> is NULL<\/code>. Puts<\/code>, on the other hand, probably crashes trying to dereference a null pointer.<\/p>\n\n
        You might think<\/span>\n all cases of undefined behavior have been discovered. I’d be surprised if this was true. The “undefined by omission” case allows for all sorts of things that are so circuitous to understand, it makes you wonder how the language works at all. For example, after a somewhat long discussion in ##C on Freenode, we came to the conclusion that there’s undefined behavior hidden in bit-fields. To get here, you have to jump through several hoops:<\/p>\n\n
        First of all, \u00a76.7.2p5 states that an implementation may choose whether a bit-field of type int<\/code> is represented as signed int<\/code> or unsigned int<\/code>. Given the following code,<\/p>\n\n
        struct ex {\n    int bf : 1;\n};\n<\/code><\/pre>\n\n
        we might expect to be able to represent the values -1<\/code> and 0<\/code>. Or maybe 0<\/code> and 1<\/code>. Or maybe -0<\/code> and 0<\/code>. But none of these are actually possible, because \u00a76.2.6.2p2 defines signed integers as “containing exactly one sign bit,” the purpose of which is to modify the value of the variable with either sign and magnitude<\/em>, two’s complement<\/em>, or ones’ complement<\/em> semantics. However, in the above example, ex.bf<\/code> clearly has no value bits to modify if it is represented as a signed integer: there is only enough space to represent the sign bit.<\/p>\n\n
        The spec could clear this up by specifying in \u00a76.7.2.1 that bit-fields of signed type must include enough space for at least one value bit. Or specifying something similar at or around \u00a76.2.6.2, where the representation of signed integers is discussed. As it stands, this is the kind of thing you end up running into on accident.<\/p>\n\n
        I don’t think<\/span>\n it’s great that programmers should be left to “discover” undefined behavior. To that end, I think it’d be great if things like this were spelled out more completely in the standard, or if Annex J.2 was considered normative. However, it seems the standard committee is fine leaving such behavior undefined by omission.<\/p>\n\n
        What undefined behavior have you discovered? I’d love to hear about it!<\/p>\n\n
        This post was edited<\/span>\n to fix a statement about printf<\/code> and puts<\/code> equivalency. In particular, I had stated that printf("%s", s);<\/code> was functionally equivalent to puts(s);<\/code>. It is not, but printf("%s\\n", s);<\/code> is. Thanks to lemonade\\<\/code>` on Freenode for pointing this out.<\/p>\n"},{"title":"Help Vampires Aren't","link":"https:\/\/9vx.org\/post\/help-vampires-arent\/","pubDate":"Fri, 14 Apr 2017 23:20:00 +0000","guid":"https:\/\/9vx.org\/post\/help-vampires-arent\/","description":"
        Help vampires.<\/span>\n It’s an idea apparently coined by Amy Hoy in an article she wrote<\/a> over ten years ago. The “vampires” described are common in open source communities. They’re also prevalent in online chat communities like IRC, especially in channels ostensibly for help purposes. The blog post describes a real problem that people experience. Some people in these communities (usually neophytes) engage in maladaptive learning behaviors that drain the energy out of those who might be able to help them (hence the term “help vampire”).<\/p>\n\n
        But I have a real problem with this terminology, the presentation of information in the article, and many of its suggestions for solving the problem.<\/p>\n\n
        The tone<\/span>\n of the first half of the article is glib, but inflammatory. The article claims to be a resource one might share with a so-called “help vampire,” but it spends the entire first half basically lambasting these individuals. This could be solved relatively easy by reversing the order of the article, such that the helpful information is placed first and the backstory comes at the end. If this were the case, the article would read to be more genuine about wanting to help.<\/p>\n\n
        Though the article claims that it can help people change their ways (and help others help those people change their ways), it subverts this goal entirely by first offending the person looking for help, and by training educators to recognize these people as malicious. For example, this appears in the fourth paragraph of the article: “[T]hese vampires suck the very life and energy out of people.”<\/p>\n\n
        Read that several times. If you were just called a “help vampire” – this is the first suggestion of how to deal with such individuals – would you feel good about yourself? Would you suddenly be open to changing? Or would you be confused why someone is trying to stop you from learning something? Would you feel offended? Would you become defensive? Would you shut down? Would you feel like this community is for you? Here, at the very beginning of an article that is supposed to be helpful to people of this classification, such individuals are already vilified. This is not an effective strategy for educating people, nor is it an effective strategy for convincing people to change their behaviors. In fact, it engages in the same kinds of “life sucking” activities it accuses the “vampire” of possessing!<\/p>\n\n
        The problem is that the whole idea of a “help vampire” (as specified) assumes malevolence. It completely disregards the popular aphorism of Hanlon’s razor