{"@attributes":{"version":"2.0"},"channel":{"title":"Bessey's Blog","description":"Technical blog of a full stack software engineer, working on all things web since 2013","link":"https:\/\/bessey.dev\/","pubDate":"Wed, 19 Jun 2024 09:20:06 +0000","lastBuildDate":"Wed, 19 Jun 2024 09:20:06 +0000","generator":"Jekyll v3.9.5","item":[{"title":"Github Releases are no match for CHANGELOG.md","description":"

I have noticed a trend recently of library authors abandoning their CHANGELOG.md<\/a> files, in favour of Github Releases.<\/p>\n\n

\"Example<\/p>\n\n

For a library author, the choice is understandable. A Github Release message contains much the same information as a changelog, so why duplicate effort and risk having no source of truth by maintaining both? But there\u2019s a pretty big problem with this: Github Releases today sucks UX wise for your library\u2019s users<\/strong>.<\/p>\n\n

TLDR?<\/strong> Get the best of both worlds by generating your changelog from your Releases with rhysd\/changelog-from-release<\/strong><\/a>!<\/em><\/p>\n\n\n\n

First off, let me clarify what I use changelogs for, and what I expect of them. By far the most common use case I have for a changelog is when upgrading a library, to understand what significant (to me<\/a>) changes have occurred between the version I\u2019m on and the version I\u2019m upgrading to.<\/p>\n\n

Pretty simple stuff, but let\u2019s see how the difficulty of completing this task differs between Github Releases and a CHANGELOG.md file.<\/p>\n\n

Github Releases<\/h3>\n\n

We navigate to our repository\u2019s Releases page, where we can see the 10 most recently released versions. Most of them are patches, so we only get one minor release on the whole page.<\/p>\n\n

\"\"<\/p>\n\n

This isn\u2019t the version we\u2019re upgrading from though, obviously. It\u2019s ok though, there\u2019s a search bar, so we can just search for my exact version! Except\u2026<\/p>\n\n

\"\"<\/p>\n\n

Nope, the Releases search bar behaviour is\u2026 unintuitive<\/em>. Passing the exact Release tag string doesn\u2019t guarantee it comes up first, or even on the first page. Adding quotation marks doesn\u2019t either. It turns out<\/a> the magic incantation is tag:blah<\/code>:<\/p>\n\n

\"\"<\/p>\n\n

Great! We have the version we\u2019re running now. Now we just need to page forwards through the newer versions. Except, that is not possible from this view:<\/p>\n\n

\"\"<\/p>\n\n

So I guess we\u2019ll have to return to the index page, and page backwards from the latest version all the way to the version we are currently on after all. In batches of 10. Fun.<\/p>\n\n

\"\"<\/p>\n\n

Well this is awful. Oh, at least there\u2019s a &page=X<\/code> query param. We can use that to skip back and forth through pages homing in on the page featuring our version. We go to page=3<\/code> and the last version is too new. On page=8<\/code> the first version is too old. How about page=5<\/code>? Last version too new again\u2026 Somehow we have gone from upgrading a library to executing a human powered binary search<\/strong> \ud83e\udd22.<\/p>\n\n

Eventually we find the right page, and can now start paging forwards through the newer versions, studying the associated notes. Even now though we run into an annoying behaviour we need to be aware of. Since releases are ordered chronologically by publish<\/strong> date there may be releases \u201cbetween\u201d versions in the list that do not apply to us because they are not present in the version we are running, because they are from another branch:<\/p>\n\n

\"\"<\/p>\n\n

Mercifully it is possible to suppress pre-releases with prerelease:false<\/code> in the search UI. However, this doesn\u2019t solve the issue where older major versions are maintained alongside newer versions as is common for bigger dependencies like languages and frameworks. In those cases, you just have to skip past these versions.<\/p>\n\n

All in all, quite the ordeal to complete a seemingly simple task. So how does this modern clean solution<\/em> compare to the ancient archaic text file<\/em> approach?<\/p>\n\n

CHANGELOG.md<\/h3>\n\n

We navigate to our repository\u2019s CHANGELOG.md in its root directory, where we can see all versions ever. Next, we Ctrl+F<\/code> for our version, type its name into the browser find in page UI, and we are instantly scrolled down to it. If you prefer mice to keyboards, we also have a list of jump links to every version.<\/p>\n\n

\"\"<\/p>\n\n

It\u2019s not uncommon to have the entire repository\u2019s history in this single page. React<\/a> for example has 11 years of history<\/strong> on this one page.<\/p>\n\n

And to see changes between the upgraded version and ours? Scroll up. That\u2019s it \ud83d\ude04. By virtue of being branch specific, changelogs sidestep the aforementioned pre-release problem, because the repository\u2019s trunk branch changelog only contains changes that have landed on the trunk. If you are<\/em> a pre-release tester, the release branch\u2019s changelog has what you need.<\/p>\n\n

\n\n

I think the UX chasm between these two workflows is obvious, and that is why I am surprised and concerned to see CHANGELOG.md falling out of fashion. So what\u2019s to be done? Well on the off chance you are a Github employee, please, improve this experience! If Releases are to replace changelogs, as I think they reasonably could, their UX needs to be better<\/em> than changelogs, not worse. A UI to see changelogs between releases X and Y would be a game changer.<\/p>\n\n

However, in the much likelier event you are a repo maintainer that\u2019s all in on Github Releases, please consider maintaining a CHANGELOG.md<\/strong> also. If that\u2019s too much effort consider generating a CHANGELOG.md from your existing Github Releases<\/strong>. Thanks to rhysd\/changelog-from-release<\/strong><\/a> this is a pretty simple thing to do. It can even be configured to ignore certain releases, enabling you to e.g. exclude pre-releases from your trunk changelog, but include them on your pre-release branch changelog.<\/p>\n\n

Thanks for reading, and above all, much love to all open source maintainers out there! \u2764\ufe0f<\/p>\n\n

PS I used to have Disqus comments on but going forward I\u2019m disabling them as they are full of really tasteless ads. Please discuss on HN \/ Reddit \/ Twitter<\/del> X instead.<\/em><\/p>\n","pubDate":"Tue, 18 Jun 2024 13:01:00 +0000","link":"https:\/\/bessey.dev\/blog\/2024\/06\/18\/github-releases-vs-changelogs\/","guid":"https:\/\/bessey.dev\/blog\/2024\/06\/18\/github-releases-vs-changelogs\/","category":["changelog","github","ux"]},{"title":"Why, after 6 years, I\u2019m over GraphQL","description":"

GraphQL is an incredible piece of technology that has captured a lot of mindshare since I first started slinging it in production in 2018. You won\u2019t have to look far back on this (rather inactive) blog to see I have previously championed this technology. After building many a React SPA on top of a hodge podge of untyped JSON REST APIs, I found GraphQL a breath of fresh air. I was truly a GraphQL hype train member.<\/p>\n\n

However, as the years have gone on and I have had the opportunity to deploy to environments where non functional requirements like security, performance, and maintainability were more of a concern, my perspective has changed. In this article I would like to take you through why today, I would not recommend GraphQL to most people<\/strong>, and what I think are better alternatives.<\/p>\n\n\n\n

Throughout I will use Ruby code with the excellent graphql-ruby<\/a> library for examples, but I believe many of these problems are ubiquitous across choice of language \/ GraphQL library.<\/p>\n\n

If you know of better solutions and mitigations, please do leave a comment. Now, lets begin\u2026<\/p>\n\n

Attack surface<\/h2>\n\n

It was obvious from GraphQL\u2019s beginning that exposing a query language<\/strong> to untrusted clients increases the attack surface of the application. Nevertheless, the variety of attacks to consider was even broader than I imagined, and mitigating them is quite a burden. Here\u2019s the worst I\u2019ve had to deal with over the years\u2026<\/p>\n\n

Authorisation<\/h3>\n\n

I think this is the most widely understood risk of GraphQL, so I won\u2019t go into too much depth here. TLDR: if you expose a fully self documenting query API to all clients, you better be damn sure that every field<\/strong> is authorised against the current user appropriately to the context in which that field is being fetched. Initially authorising objects<\/strong> seems like enough, but this quickly becomes insufficient. For example, say we are the Twitter<\/del> X \ud83d\ude44 API:<\/p>\n\n

query<\/span> <\/span>{<\/span>\n  <\/span>user<\/span>(<\/span>id<\/span>:<\/span> <\/span>321<\/span>)<\/span> <\/span>{<\/span>\n    <\/span>handle<\/span> <\/span># \u2705 I am allowed to view Users public info<\/span>\n    <\/span>email<\/span> <\/span># \ud83d\uded1 I shouldn't be able to see their PII just because I can view the User<\/span>\n  <\/span>}<\/span>\n  <\/span>user<\/span>(<\/span>id<\/span>:<\/span> <\/span>123<\/span>)<\/span> <\/span>{<\/span>\n    <\/span>blockedUsers<\/span> <\/span>{<\/span>\n      <\/span># \ud83d\uded1 And sometimes I shouldn't even be able to see their public info,<\/span>\n      <\/span># because context matters!<\/span>\n      <\/span>handle<\/span>\n    <\/span>}<\/span>\n  <\/span>}<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n\n
Compare this to the REST world where generally speaking you would authorise every endpoint<\/strong>, a far smaller task.<\/p>\n\n
Rate limiting<\/h3>\n\n
With GraphQL we cannot assume that all requests are equally hard on the server. There is no limit to how big a query can be. Even in a completely empty schema, the types exposed for introspection<\/a> are cyclical, so its possible to craft a valid query that returns MBs of JSON:<\/p>\n\n
query<\/span> <\/span>{<\/span>\n  <\/span>__schema<\/span>{<\/span>\n    <\/span>types<\/span>{<\/span>\n      <\/span>__typename<\/span>\n      <\/span>interfaces<\/span> <\/span>{<\/span>\n        <\/span>possibleTypes<\/span> <\/span>{<\/span>\n          <\/span>interfaces<\/span> <\/span>{<\/span>\n            <\/span>possibleTypes<\/span> <\/span>{<\/span>\n              <\/span>name<\/span>\n            <\/span>}<\/span>\n          <\/span>}<\/span>\n        <\/span>}<\/span>\n      <\/span>}<\/span>\n    <\/span>}<\/span>\n  <\/span>}<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n\n
I just tested this attack against a very<\/strong> popular website\u2019s GraphQL API explorer and got a 500 response back after 10 seconds. I just ate 10 seconds of someone\u2019s CPU time running this (whitespace removed) 128 byte<\/strong> query, and it doesn\u2019t even require me to be logged in.<\/p>\n\n
    \n  
  1. Estimate the complexity of resolving every single field in the schema<\/strong>, and abandon queries that exceed some maximum complexity value<\/li>\n  
  2. Capture the actual complexity of the run query and take it out of bucket of credits that resets at some interval<\/li>\n<\/ol>\n\n
    This calculation is a delicate affair to get right<\/a>. It gets particularly tricky when you are returning list fields whose length is not known prior to execution. You can make an assumption about the complexity of these, but if you are wrong, you may end up rate limiting valid queries or not rate limiting invalid queries.<\/p>\n\n
    To make matters worse, its common for the graph that makes up the schema to contain cycles. Lets say you run a blog with Articles which each have multiple Tags, from which you can see associated Articles.<\/p>\n\n
    type<\/span> <\/span>Article<\/span> <\/span>{<\/span>\n  <\/span>title<\/span>:<\/span> <\/span>String<\/span>\n  <\/span>tags<\/span>:<\/span> <\/span>[<\/span>Tag<\/span>]<\/span>\n<\/span>}<\/span>\n<\/span>type<\/span> <\/span>Tag<\/span> <\/span>{<\/span>\n  <\/span>name<\/span>:<\/span> <\/span>String<\/span>\n  <\/span>relatedTags<\/span>:<\/span> <\/span>[<\/span>Tag<\/span>]<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n\n
    query<\/span> <\/span>{<\/span>\n  <\/span>tag<\/span>(<\/span>name<\/span>:<\/span> <\/span>\"security\"<\/span>)<\/span> <\/span>{<\/span>\n    <\/span>relatedTags<\/span> <\/span>{<\/span>\n      <\/span>relatedTags<\/span> <\/span>{<\/span>\n        <\/span>relatedTags<\/span> <\/span>{<\/span>\n          <\/span>relatedTags<\/span> <\/span>{<\/span>\n            <\/span>relatedTags<\/span> <\/span>{<\/span> <\/span>name<\/span> <\/span>}<\/span>\n          <\/span>}<\/span>\n        <\/span>}<\/span>\n      <\/span>}<\/span>\n    <\/span>}<\/span>\n  <\/span>}<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n
    You expect a complexity of 5^5 = 3,125<\/code>. If an attacker is able to find an Article with 10 tags, they can trigger a query with a \u201ctrue\u201d complexity of 10^5 = 100_000<\/code>, 20x<\/strong> greater than estimated.<\/p>\n\n
    Compare this to rate limiting a REST endpoint, which generally have comparable response times. In this case all you need is a bucketed rate limiter that prevents a user exceeding, say, 200 requests per minute across all endpoints. If you do<\/strong> have slower endpoints (say, a CSV report or PDF generator) you can define more aggressive rate limits for these. With some HTTP middleware this is pretty trivial:<\/p>\n\n
    Rack<\/span>::<\/span>Attack<\/span>.<\/span>throttle<\/span>(<\/span>'API v1'<\/span>,<\/span> limit: <\/span>200<\/span>,<\/span> period: <\/span>60<\/span>)<\/span> do<\/span> |<\/span>req<\/span>|<\/span>\n  if<\/span> req<\/span>.<\/span>path<\/span> =~<\/span> '\/api\/v1\/'<\/span>\n    req<\/span>.<\/span>env<\/span>[<\/span>'rack.session'<\/span>][<\/span>'session_id'<\/span>]<\/span>\n  end<\/span>\nend<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
    Query parsing<\/h3>\n\n
    Before a query is executed, it is first parsed. We once received a pen-test report evidencing that its possible to craft an invalid query string that OOM\u2019d the server. For example:<\/p>\n\n
    query<\/span> <\/span>{<\/span>\n  <\/span>__typename<\/span> <\/span>@a<\/span> <\/span>@b<\/span> <\/span>@c<\/span> <\/span>@d<\/span> <\/span>@e<\/span> <\/span>...<\/span> <\/span># imagine 1k+ more of these<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n\n
    This is a syntactically<\/strong> valid query, but invalid for our schema. A spec compliant server will parse this and start building an errors response containing thousands of errors which we found consumed 2,000x<\/strong> more memory than the query string itself. Because of this memory amplification, its not enough to just limit the payload size, as you will have valid queries that are larger than the the smallest dangerous malicious query.<\/p>\n\n
    Performance<\/h2>\n\n
    When it comes to performance in GraphQL people often talk about it\u2019s incompatibility with HTTP caching. For me personally, this has not been an issue. For SaaS applications, data is usually highly user specific and serving stale data is unacceptable, so I have not found myself missing response caches (or the cache invalidation bugs they cause\u2026).<\/p>\n\n
    The major performance problems I did<\/strong> find myself dealing with were\u2026<\/p>\n\n
    Data fetching and the N+1 problem<\/h3>\n\n
    I think this issue is pretty widely understood nowadays. TLDR: if a field resolver hits an external data source such as a DB or HTTP API, and it is nested in a list containing N items, it will do those calls N times.<\/p>\n\n
    This is not a unique problem to GraphQL, and actually the strict GraphQL resolution algorithm has allowed most libraries to share a common solution: the Dataloader pattern<\/a>. Unique to GraphQL though is the fact that since it is a query language, this can become<\/strong> a problem with no backend changes when a client modifies a query. As a result, I found you end up having to defensively introduce the Dataloader abstraction everywhere just in case<\/em> a client ends up fetching a field in a list context in the future. This is a lot of boilerplate to write and maintain.<\/p>\n\n
    Meanwhile, in REST, we can generally hoist nested N+1 queries up to the controller, which I think is a pattern much easier to wrap your head around:<\/p>\n\n
    class<\/span> BlogsController<\/span> <<\/span> ApplicationController<\/span>\n  def<\/span> index<\/span>\n    @latest_blogs<\/span> =<\/span> Blog<\/span>.<\/span>limit<\/span>(<\/span>25<\/span>).<\/span>includes<\/span>(<\/span>:author<\/span>,<\/span> :tags<\/span>)<\/span>\n    render<\/span> json: <\/span>BlogSerializer<\/span>.<\/span>render<\/span>(<\/span>@latest_blogs<\/span>)<\/span>\n  end<\/span>\n\n  def<\/span> show<\/span>\n    # No prefetching necessary here since N=1<\/span>\n    @blog<\/span> =<\/span> Blog<\/span>.<\/span>find<\/span>(<\/span>params<\/span>[<\/span>:id<\/span>])<\/span>\n    render<\/span> json: <\/span>BlogSerializer<\/span>.<\/span>render<\/span>(<\/span>@blog<\/span>)<\/span>\n  end<\/span>\nend<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
    Authorisation and the N+1 problem<\/h3>\n\n
    But wait, there\u2019s more N+1s! If you followed the advice earlier of integrating with your library\u2019s authorisation framework, you\u2019ve now got a whole new category of N+1 problems to deal with. Lets continue with our X API example from earlier:<\/p>\n\n
    class<\/span> UserType<\/span> <<\/span> GraphQL<\/span>::<\/span>BaseObject<\/span>\n  field<\/span> :handle<\/span>,<\/span> String<\/span>\n  field<\/span> :birthday<\/span>,<\/span> authorize_with: :view_pii<\/span>\nend<\/span>\n\nclass<\/span> UserPolicy<\/span> <<\/span> ApplicationPolicy<\/span>\n  def<\/span> view_pii?<\/span>\n    # Oh no, I hit the DB to fetch the user's friends<\/span>\n    user<\/span>.<\/span>friends_with?<\/span>(<\/span>record<\/span>)<\/span>\n  end<\/span>\nend<\/span>\n<\/code><\/pre><\/div><\/div>\n\n
    query<\/span> <\/span>{<\/span>\n  <\/span>me<\/span> <\/span>{<\/span>\n    <\/span>friends<\/span> <\/span>{<\/span> <\/span># returns N Users<\/span>\n      <\/span>handle<\/span>\n      <\/span>birthday<\/span> <\/span># runs UserPolicy#view_pii? N times<\/span>\n    <\/span>}<\/span>\n  <\/span>}<\/span>\n<\/span>}<\/span>\n<\/span><\/code><\/pre><\/div><\/div>\n\n
    This is actually trickier to deal with than our previous example, because authorisation code is not always run in a GraphQL context. It may for example be run in a background job or an HTML endpoint. That means we can\u2019t just reach for a Dataloader naively, because Dataloaders expect to be run from within GraphQL (in the Ruby implementation anyway).<\/p>\n\n
    In my experience, this is actually the biggest source of performance issues<\/strong>. We would regularly find that our queries were spending more time authorising data than anything else. Again, this problem simply does not exist in the REST world.<\/p>\n\n
    Coupling<\/h2>\n\n
    In my experience, in a mature GraphQL codebase, your business logic is forced into the transport layer<\/a>. This happens through a number of mechanisms, some of which we\u2019ve already talked about:<\/p>\n\n
    I have mitigated this using nasty things like request level globals<\/a> to memoise data across policy calls, but its never felt great.<\/p>\n\n
    If your server exposes a concept of maximum number of errors to accrue before abandoning parsing, this can be mitigated<\/a>. If not, you\u2019ll have to roll your own solution. There is no REST equivalent to this attack of this severity.<\/p>\n\n
    A partial mitigation here is to prevent deeply nested queries<\/a>. However, the example above demonstrates that this is not really a defense, as it\u2019ts not an unusually deep query. GraphQL Ruby\u2019s default maximum depth is 13<\/strong>, this is just 7.<\/p>\n\n
    When estimating the complexity of Tag.relatedTags<\/code>, you might assume that an article will never have more than 5 tags, so you set this fields complexity to 5<\/strong> (or 5 * its children\u2019s complexity<\/a>). The problem here is that Article.relatedTags<\/code> can be its own child, so your estimate\u2019s inaccuracy can compound exponentially. The formula is N^5 * 1<\/code>. So given this query:<\/p>\n\n
A common mitigation1<\/a><\/sup> for this attack is to<\/p>\n\n
One wonders how much GraphQL holds responsibility for Broken Access Control climbing to the OWASP Top 10\u2019s #1 spot<\/a>. One mitigation here is to make your API secure by default by integrating with your GraphQL library\u2019s authorisation framework<\/a>. Every object returned and\/or field resolved, your authorisation system is called to confirm that the current user has access.<\/p>\n\n