Getting started with Claude Code, on MacOS, with chruby and rspec

I’ve been writing ruby and rails for nearly 20 years. A couple weeks ago, I had gotten code snippets from copy-paste in a chat window, but I hadn’t even experimented with Claude Code or similar “can write code to your file system” tools.

I know some people are now using LLM’s to write all their code, which I’m not excited about, but I decided I couldn’t hold off any longer, and I had to at least understand how it worked to be able to decide when/where to use it. Everything in here is probably (?) old news for people already way into using LLMs to write code.

I decided that a project to speed up my rspec test suite (using the amazing test-prof for profiling and performance patterns!) was a great first application of it — because it will probably involve both analysis and writing many files, if nothing else Claude is probably great at editing many files according to my instructions doing much more than a regex grep can do (yes, indeed it was great at this).

Since I’m optimizing the test suite, I definitely want Claude Code to be able to run rspec — but really for any task, I gather you do, because you definitely want it to be able to run tests to make sure they pass, and iterate if it did something to break tests.

I somewhat unorthodoxly use chruby as my ruby version manager, and I had a bit of trouble getting claude to run rspec (and any other ruby tools I might want) with chruby, and then a bit more trouble when I realized that capybara with selenium-chromedriver was running into trouble with default sandbox that in June 2026 a MacOS Claude Code runs in.

tldr, here’s the PR with the settings/configuration I ended up with.

I was not used to tools that work like Claude, and it all seems to be somewhat under-documented (perhaps because it’s changing so fast) and under-blogged about (do people blog anymore when they can just ask an LLM to solve it so nobody is reading blogs?), or just confusing to me — it took me a day or two to figure it out honestly, and I kept wondering if I was doing it wrong/different from anyone else… but I think what I ended up with is reasonable? If you know better/different, please do let me know!

I definitely kept thinking “surely I’m not the only one trying to do this, why is this so confusing to me and why are others so confused when i ask about? Am I missing something obvious?” I’m still not sure! But I share what I figured out in case it will help.

Specify to run with chruby-exec in a CLAUDE.md

I could not get Claude to run the normal source files for chruby — editing various .bash or .zsh config files (yes I know about ~/.zshenv) did not seem to have any effect. Perhaps Claude Code doesn’t use a ‘real’ shell that uses any config files? When I asked Claude Code itself what to do, it suggested configuration to try to get Claude Code to use config files.. but none seemed to work?

One thing Claude kept suggesting was hard-coding the ENV variables set by chruby in the claude settings.json — which I’m sure would have worked, but I just didn’t like it as a solution. Maybe this is what everyone else is doing? I thought surely we can do better. Plus I’d ideally like it to auto choose based on .ruby-version, not be something I have to update everytime I update ruby (frequent), or have Claude accidentally using a different ruby than my other tools are!

Thanks to @havenwood for helping me think through it on chruby github discussions, and for suggesting using chruby-exec, with a little shell substitution with cat .ruby-version. This in a CLAUDE.md (rather than other things in settings.json) seems to work great:

Prefix Ruby shell commands (ruby, bundle, rake, etc.) with chruby-exec $(cat .ruby-version) --:

“`

chruby-exec $(cat .ruby-version) — bundle exec rake

“`

Running chrome does not work in sandbox used on MacOS

I can’t speak for other OS’s, and I don’t totally understand what’s going on (MacOS “Seatbelt” I guess?), but Claude Code executed rspec was refusing to bring up headless chrome,which i use for system/feature specs via selenium/selenium-webdrivers.

Trying multiple things Claude suggested to specifically allow-list chrome(driver) through the sandbox, definitely none of them worked. Btw, did try switching to cuprite (with Claude Code’s help of course to do it fairly quickly) — despite some reddit suggestions, it seemed to still have the exact same sandbox issue, and at least in my project actually ran my test suite somewhat slower than chromedriver.

Eventually, with more confusing reddit discussion where nobody else had any idea what I was talking about or why I was having a problem, I decided that everyone else must just be exempting rspec itself from the sandbox. (Because surely having Claude Code run rspec is very normal, right? It’s just so useful!) (Thank you to redditors who tried to help!)

It’s just straight rspec, but rspec with many possible arguments, running just certain files/examples, possibly with profiling arguments for test-prof etc. I need it exempted from the sandbox so it can run chrome(driver), but I also need it not to be asking me “Is it okay to run this set of argumetns with rspec” all the time?

Two different settings in Claude’s settings.json, both accept wildcards — for both I want to apply to rspec executions but not accidentally extra stuff, want to try to stay secure-ish here. The chruby rigamorole above makes that somewhat more confusing.

I forget if it was my idea or claude’s idea, but we wrote a wrapper script for chruby-exec-rspec, so we could more cleanly allow-list just that. Claude definitely wrote the implementation of the bash wrapper script. And when I realized that all the test-prof inline ENV vars for profiling (like FPROF=1) messed up my attempted left-anchored allow-listing, I asked claude to work that out by letting the wrapper rearrange an arg into an inline ENV prefix, something my bash skills were def not up to.

The best way to see how I did all that is just the PR.

A word on Claude Desktop

I initially started work in Claude Desktop “code” tab, rather than the CLI. I think this actually made it more confusing to solve these problems above? I am not sure if sandboxing works differnetly in Claude Desktop vs claude CLI? I think the desktop may just be running the claude CLI in various directories?

Just starting out and not being sure how things were working… I found trying to ask Claude [how/] to fix the problems I was having, made things very confusing. Claude does not know whether it’s running as Claude Desktop or not, and was not really sure if the answer is different ha (Claude Desktop probably post-dates Claude Sonnet 4.6’s knowledge base?). Most blogs etc you find googling also pre-date Claude Desktop.

I switched to claude CLI and I can’t totally explain why but things seemed to get simpler. All the fixes I figured out worked when I switched back to Claude Desktop.

And contrary to what you might find googling, claude CLI and Claude Desktop do share sessions now, you can start a session in either place, then move to the other tool to continue it, in either direction. To start claude CLI and choose an existing sessiont to resume, you need to launch as claude --resume.

The CLI is a very neat UI actually! It definitely still seems to be the most popular way to use Claude (whether direct or in a panel in an editor), Claude Desktop “code” tab is I guess fairly new and not as popular, although I like it too and still am mostly using it.

How it worked for the task?

Pretty amazingly actually. Even having read about what it could do, I was kind of amazed.

Once you get it able to run rspec (including with test-prof profiling), this prompt is pretty amazing and fun:

Please use various test-prof profiling commands to identify current best opportunities for speeding up test suite.

Come back ~20 minutes later (my full test suite took ~4 minutes to run at the beginning) and it had some stuff. It tended to just go ahead and make the changes not outline them to me first (I was not in “plan” mode, haven’t tried that much yet), but I’m in a git-controlled dir I can git diff to see what it did — and ask it about it.

By the time I thought to use this general one, i had already implemented some low-hanging optimizations, so that may be why, to be fair, this prompt alone didn’t find much actually significant at that point, honestly.

Here are some others that were pretty amazing:

I am looking to speed up the test suite in this Rails app using rspec and factorybot.

To begin with, let’s focus on the system specs in spec/system. I don’t think they have any obvious performance improvement opportunities. But I’m wondering if they are all necessary. Can you identify any that may be testing something that is not necessary to test, or could be tested by a different kind of spec that is faster?

for specs in spec/components, let’s try changing factory data from create to build_stubbed. Change it for setup where tests still pass. For tests that break when you do that, list them, and if it’s clear let me know why they failed. Analyze performance gains.

[didn’t actually get any gains there, but found that out quickly with very little manual effort, which is a win!]

in our rspec setup, switch from chromedriver to cuprite. make sure tests still pass, if not identify why not.

using AnyFixture, I’ve created a :standard_work fixture, that’s just a generic public tiff-based work.

Can you identify model or service specs it would work well for?

[Didn’t actually end up using AnyFixture yet, but Claude Code helped me make that decision much quicker than I could have without it, based on how much benefit we got vs complexity]

My Rails app uses rspec and Factorybot.

There is an :asset factory with an :inline_promoted_file trait. it turns out this is really bad for performance, and we in fact rarely need to actually create assets with inline promoted files. That should only be used in cases where we really need to test end-to-end derivative and characterization.

In most other cases, we can use a faster “faked file” approach instead. Instead of a trait, we’ve implemented this with a sub-factory, :asset_with_faked_file.

Can you find uses of the :asset factory with :inline_promoted_file, and, if there’s no reason they need to test end-to-end derivative creation, change them to use :asset_with_faked_file sub-factory instead?

[It was able to identify the ones that would work pretty well, was the amazing part — and explain to me exactly why the other ones wouldn’t]

Tell me if I’m doing something weird?

Some of the stuff with chruby/rspec, I am still surprised I had so much trouble getting started, and am wondering if I’m doing something weird/wrong!

But I think probably it’s just that I have been writing code so long, that dealing with these tools that work very differently requires my brain to get out of it’s rut… also that I’m kind of a perfectionist and want to understand whats’ going on and be comfortable with it and that it’s the best way, when increasingly others are just vibing? I don’t know!

But feedback welcome!

ActiveRecord neighbor vector search, with per-document max

I am doing LLM “RAG” with rails ActiveRecord, postgres with the pgvector extension for vector similarity searches, and the neighbor gem. I am fairly new to all of this stuff, figuring it out by doing it.

I realized that for a particular use, I wanted to get some document diversity — so i wanted to do a search of my chunks ranked by embedding vector similarity, getting the top k (say 12) chunks — but in some cases I only want, say, 2 chunks per document. So the top 12 chunks by vector similarity, such that only 2 chunks per interview max are represented in those 12 top chunks.

I decided I wanted to do this purely in SQL, hey, I’m using pgvector, wouldn’t it be most efficient to have pg do the 2-per-document limit?

Note: This may be a use case that isn’t a good idea! I have come to realize that maybe I want to just fetch 12*3 or *4 docs into ruby, and apply my “only 2 per document” limit there? Because I may want to do other things there anyway that I can’t do in postgres, like apply a cross-model re-ranker? So I dunno, but for now I did it anyway.

So this was some fancy SQL, i was having trouble figuring out how to do it myself, so I asked ChatGPT, sure. It gave me an initial answer that worked, but…

Turns out was over-complicated, a simpler (to my understanding anyway) approach was possible
Turns out was not performant, it was not using my postgres ‘HNSW’ indexes to make vector searches higher performance, and/or was insisting on sorting the entire table first defeating the point of the indexes. How’d I know? Well, I noticed it was being slower than expected (several seconds or at times much more to return), and then I did postgres explain/analyze… which I had trouble understanding… so i fed the results to ChatGPT and/or Claude, who confirmed, yeah buddy, this is a bad query, it’s not using your vector index properly.

I had to go on a few back and forths with both ChatGPT and Claude (this is just talking to them in a GUI, not actually using Claude Code or whatever), to get to a pattern that did use my index effectively. They kept suggesting things to me that either just didn’t work, or didn’t actually use the index, etc. I had to actually understand what they were suggesting, and tweak it myself, and have a dialog with them…

But i eventually got to this cool method that can take an arbitrary ActiveRecord relation which already has had neighbor nearest_neighbors query applied to it… and wraps it in a larger query using CTE’s that can limit the results to max-per-document.

I wondered if I should try to share this somewhere (would neighbor gem want a PR?), except… I’m realizing like I said above maybe this is not actually a very useful use case, better to do it in ruby… I’m still not necessariliy getting the performance I expected either, although the analyze/explain says the indexes should be used properly.

So I just share here. Note the original base_relation may be it’s own internal joins to enforce additional conditions on retrieval etc. Assuming each Chunk ActiveRecord model has a document_id attribute which we are using to group for max-per-document.

# We need to take base_scope and use it as a Postgres CTE (Common Table Expression)
    # to select from, but adding on a ROW_NUMBER window function, that let's us limit
    # to top max_per_interview
    #
    # Kinda tricky, especially to do with good index usage. Got solution from google and talking
    # to LLMs, including having them look at pg explain/analyze output.
    #
    # @param base_relation [ActiveRecord::Relation] original relation, it can have joins and conditions.
    #   It MUST have already had vector distance ordering applied to it with `neighbor` gem.
    #
    # @param max_per_interview [Integer] maximum results to include per interview (oral_history_content_id)
    #
    # @param inner_limit [Integer] how many to OVER-FETCH in inner limit, to have enough even after
    #    applying max-per-interview.
    #
    # @return [ActiveRecord::Relation] that's been in a query to enforce max_per_interview limits. It does
    #   not have an overall limit set, caller should do that if desired, otherwise will be effectively
    #   limited by inner_limit.
    def wrap_relation_for_max_per_interview(base_relation:, max_per_interview:, inner_limit:)
      # In the inner CTE, have to fetch oversampled, so we can wind up with
      # hopefully enough in outer. Leaving inner unlimited would be peformance problem,
      # cause of how indexing works it doesn't need to calculate them all if limited.
      base_relation = base_relation.limit(inner_limit)

      # Now we have another CTE that assigns doc_rank within partitioned
      # interviews, from base. Raw SQL is just way easier here.
      partitoned_ranked_cte = Arel.sql(<<~SQL.squish)
        SELECT base.*,
          ROW_NUMBER() OVER (
            PARTITION BY document_id
            ORDER BY neighbor_distance
          ) AS doc_rank
        FROM base
      SQL

      # A wrapper SQL that incorporates both those CTE's, limiting to
      # doc_rank of how many we want per-interview, and overall making sure to
      # again order by vector neighbor_distance that must already have been included
      # in the base relation.
      base_relation.klass
        .select("*") # just pass through from underlying CTE queries.
        .with(base: base_relation)
        .with(partitioned_ranked: partitoned_ranked_cte)
        .from("partitioned_ranked")
        .where("doc_rank <= ?", max_per_document)
        .order(Arel.sql("neighbor_distance"))
    end

Like I said, I am new to this LLM stuff, curious what others have to say here.

Help fund attorney for artist charged with transporting zines(?!?)

i know Des Revol, and know them to be an incredibly kind, solid, reliable person.

For real he’s facing federal charges and threat of deportation because of subversive political pamphlets found in his trunk.

Des was not at the Prairieland demonstration. Instead, on July 6, after receiving a phone call from his wife in jail (one of the initial ten), Des was followed by Federal Bureau of Investigation (“FBI”) agents in Denton, Texas. They pretextually pulled him over due to a minor traffic violation and quickly arrested him at gunpoint. He was later charged with alleged “evidence tampering and obstruction of justice” based on a box of political pamphlets that he purportedly moved in his truck from his home (not his wife’s) to another house. This type of literature can be found in any activist house or independent bookstore. Des was briefly held at the Johnson County Jail, and then transferred to a federal prison, FMC Fort Worth, where he has been held ever since.

He is also currently on an ICE hold, and has been publicly targeted and doxxed on social media by both prominent fascists and ICE. Moreover, right after his arrest, his family experienced a brutal and intimidating nine-hour FBI raid of their home. Police confiscated everything from electronics to stickers and more zines.

I’m a librarian (and software engineer, but I have a librarian’s MLIS degree and have made a career in libraries). I know that if collecting and distributing controversial, dissident, and even “subversive” political literature is subject to this kind of state repression, our entire society is in trouble.

Attorneys are expensive. And they are all so busy right now.

If you can spare a few bucks, care about a free society, and feel that supporting Des is a good way to do it, please help contribute at his GoFundMe.

More info in this article from the Intercept, and at Des’ support website.

Des says:

I want to be very clear. I did not participate. I was not aware nor did I have any knowledge about the events that transpired on July 4 outside the Prairieland Detention Center. Despite not having any knowledge or not having been near the area at all, I was violently arrested at gunpoint for allegedly making a “wide turn.” My feeling is that I was only arrested because I’m married to Mari Rueda, who is being accused of being at the noise demo showing support to migrants who are facing deportation under deplorable conditions. For this accusation, she’s being threatened with a life sentence in prison.

My charge is allegedly having a box containing magazine “zines,” books, and artwork. Items that are in the possession of millions of people in the United States. Items that are available free online, and available to purchase at stores and online even at places like Amazon. Items that should be protected under the First Amendment “freedom of speech.” If this is happening to me now, it’s only a matter of time before it happens to you.

I believe there’s been almost 20 people arrested in supposed relation to this public noise demo. More than half of those were arrested days later despite not being in the area and are now facing a slew of outrageous charges, in what seems like a political persecution to instill fear on people exercising their First Amendment right.

Whisper-generated transcripts used in presentation of archival video

Here at the Science History Institute, we have a fairly small, but growing, body of video/film in our Digital Collections, at present just over 100 items, around 70 hours total.

We wanted to add transcripts/captions to these videos, for accessibility to those who are hearing impaired, for searchability of video transcript content, and for general usability. We do not have the resources to do any manual transcription or even really Quality Assurance, but we decided that OpenAI whisper automated transcription software was of sufficient quality to be useful.

We have implemented whisper-produced transcriptions. We use them for on-screen text track captions; for an accompanying on-the-side transcript; and for indexing for searching in our collection.

I’ll talk about some of the choices we made and things we discovered, including: our experience using whisper to transcribe; implementing a text track for captions in the video screen (and some Safari weirdness with untitled empty track); synchronized transcript elsewhere on the page; improving the default video.js skin/theme; and trying to encourage Google to index transcript text.

*Baseline: The Chemist*, an amusing 1970s kind of impressionistic/conceptual promotional video for… chemists being really cool?

Some other interesting videos in our collection

Atomic Energy As a Force For Good (1955)
Putting Scientific Information to Work (1967) (of interest to my librarian colleagues, an early marketing video for ISI (then the “Institute for Scientific Information”) cutting edge citation indexing database. (Go to 31:53 for a weird 60s style recap of important events of the 60s?)
Pick of the Pod (1939)
Proposition 65: Troubled Waters for California (1986)

OpenAI Whisper Hosted API

Many of our library/museum/archives peers use the open source Whisper implementation, or a fork/variation of it, and at first I assumed I would do the same. However, we deploy largely on heroku, and I quickly determined that the RAM requirements (at least for medium and above models, and disk space requirements (a pip install openai-whisper added tens of gigs) were somewhere in between inconvenient and infeasible on the heroku cedar platform, at least for our budget.

These limitations and costs change on the new heroku fir platform, so at first I thought we might have to wait until we migrate there — but then I noticed whisper also existed, of course, on the commercial OpenAI API platform.

This is not exactly the same product as OpenAI whisper, and exactly how it differs is not public. The hosted whisper does not let (or require?) you to choose a model, it just uses whatever it uses. It has fewer options — and in the open source realm, there are forks or techniques with even more options and features, like diarization or attempting to segment multi-lingual recordings by language. With the hosted commercial implementation, you just get what you get.

But on the plus side, it’s of course convenient not to have to provison your own resources. It is priced at $0.006 per minute of source audio, so that’s only around $25 to transcribe our meager 70 hour corpus, no problem, and no problem if we keep adding 70-200 hours of video a year as currently anticipated. If we start adding substantially more, we can reconsider our implementation.

Details of whisper API usage implementation

Whisper hosted API has a maximum filesize of 25 MB. Some of our material is up to two hours in length, and audio tracks simply extracted from this material routinely exceeded this limit. But by using ffmpeg to transcode to the opus encoding in an ogg container, using the opus voip profile optimized for voice, at a 16k bitrate — even 2 hours of video is comfortably under 25MB. This particular encoding was found often recommended on forums, with reports that downsampling audio like this can even result in better whisper results; we did not experiment, but it did seem to perform adequately.

ffmpeg -nostdin -y -i input_video.mp4 -vn -map-metadata -1 -ac 1 -c:a libopus -b:a 16k -application voip ./output.oga

Whisper can take a single source language argument — we have metadata already in our system recording language of source material, so if there is only one listed, we supply that. Whisper can’t really handle multi-lingual content. Almost all of our current video corpus is only English, but we do have one video that is mixed English and Korean, and fairly poor audio quality — whisper API actually refused to transcribe that, actually returning an error message (after a wait). When I tried that with opensource whisper just out of curiosity, it did transcribe it, very slowly — but all the Korean passages were transcribed as “hallucinated” English. So error-ing out may actually be a favor to us.

You can give whisper a “prompt” — it’s not conversational instructions, but is perhaps treated more like a glossary of words used. We currently give it our existing metadata “description” field, and that resulted in successful transcription of a word that never caught on, “zeugmatography” (inventor of MRI initially called it that), as well as correct spelling of “Eleuthère Irénée”. If it’s really just a glossary, we might do even better by taking all metadata fields, and just listing unique words once per word (or even trying to focus on less common words). But for now description as-is works well.

Here’s our ruby implementation, pretty simple, using the ruby-openai gem for convenience.

I had at one point wanted to stream my audio, stored on S3, directly to a HTTP POST to API, without having to download the whole thing to a local temporary copy first. But ruby’s lack of a clear contract/API/shape of a “stream” object strikes again, making interoperability painful. This fairly simple incompat was just the first of many I encountered; patching this one locally just let me onto the next one, etc. One of my biggest annoyances in ruby honestly!

Results?

As others have found, the results of whisper are quite good, better than any other automated tool our staff had experimented with, and we think the benefits to research and accessibility remain despite what errors do eist. There isn’t much to say about all the things it gets right, by listing the things it doesn’t you might get the wrong idea, but it really does work quite well.

As mentioned, it can’t really handle multi-lingual texts
Errors and hallucinations were certainly noticed. In one case it accurately transcribed a musical passage as simply ♪, but oddly labelled it as “Dance of the Sugar Plum Fairies” (it was not). An audience clapping was transcribed as repeated utterances of “ok”. This example might be more troubling: some totally imaginary dialog replacing what is pretty unintelligible dialog in the original.
Perhaps the most troubling noticed is invented copyright attributions, such as © transcript Emily Beynon (apparently a common one?) — and some other names too. Putting imaginary erroneous copyright declarations in is not great. I am contemplating post-processing to strip any cue beginning with ©, which I think can’t possibly be legitimate?
Wide differences in how long the cues are, although consistent within a piece. But some pieces are transcribed with long paragraph-sized cues, and others just phrase by phrase. I am considering post-processing to join tiny phrase cues into sentences, up to so many words.
It seems to not infrequently, well into a video, start losing the synchronization of timing, getting 5, 10, or even 15 seconds behind? This is weird and I haven’t seen it commented upon before. The text is still as correct as ever, so mostly an inconvenience. See for instance at 9:09 in Baseline: The Chemist, definitely annoying. By 10:23 it’s caught up again, but quickly gets behind again, etc.

We don’t really have the resources to QA even our fairly small collection, so we are choosing to follow in the footsteps of WGBH and their American Archive of Public Broadcasting, and publish it anyway, with a warning influenced by theirs:

I think in the post-pandemic zoom world, most users are used to automatically generated captions and all their errors, and understand the deal.

WGBH digitizes around 37K items a year, far more than we do. They also run an instance of FixIt+ for public-contributed “crowd-sourced” transcription corrections. While I believe FixIt+ is open source (or a really old version of it is?) and some other institutions may run it, we don’t think we’d get enough public attention and only have a small number of videos, we can’t really afford to stand up our own FixitPlus even if it is available. But it does seem like there is an unfilled need for someone to run a crowd-hosted FixitPlus to charge a reasonable rate for hosting for someone that only will need a handful a year?

We did implement an admin feature to allow upload of corrected WebVTT, which will be used in preference to the direct ASR (Automated Speech Recognition) ones. As we don’t anticipate this being done in bulk, right now staff just downloads the ASR WebVTT, uses the software of their choice to edit it, and then uploads a corrected version. This can be done for egregious errors as noticed, or using whatever policy/workflow our archival team thinks appropriate. We also have an admin feature to disable transcoding for material it does not work well for, such as multi-lingual, silent, or other problems.

Text Track Captions on Video

We were already using video.js for our video display. It provides API’s based on HTML5 video API’s, in some cases polyfilling/ponyfilling, in some cases just delegating to underlying APIs. It has good support for text tracks. At present, by default it uses ‘native’ text tracks instead of it’s own implementation (maybe only on?) Safari — you can force emulated text tracks, but it seemed advisable to stick to default native. This does mean it’s important to test on multiple browsers, there were some differences in Safari that required workarounds (more below).

So, for text tracks we simply provide a WebVTT file in a <track> element under the <video> element. Auto-generated captions (ASR, or “Automated Speech Recognition”, compare to OCR), don’t quite fit the existing categories of “captions” vs “subtitles” — we label them as kind captions and give them an English label “Auto-captions”, which we think/hope is a common short name for these.

Safari adding extra “Untitled” track for untagged HLS

For those most part, this just works, but there was one idiosyncracy that took me a while to diagnose and determine appropriate fix. We deliver our video as HLS with a .M3U8 playlist. There is a newer metadata element in .m3u8 playlist that can label the presence or absence of subtitles embedded in the HLS. But in the absence of this metadata — Safari (both MacOS and iOS I believe) insists on adding a text caption track called “Untitled”, which in our case will be blank. This has been noticed by some, but not as much discussion on the internet as I’d expect to be honest!

One solution would be adding the metadata saying no text track is present embedded in HLS (since we want to deliver text tracks as external in <track> element instead). Somewhat astoundingly, simply embedding an EXT-X-MEDIA tag with a fixed static value of CLOSED-CAPTIONS=NONE — on AWS Elemental MediaConvert (which I use) seems to takes you into the “Professional Tier” costing 60% more! I suppose you could manually post-process the .m3u8 manifests yourself… including my existing ones…

Instead, our solution is simply, when on Safari, hook into events on video element to remove a text track with empty string language and title, which is what characterizes these. I adapted from similar solution in ramp, who chose this direction. They wrote theirs to apply to “mobile which is not android”; I found it actually was needed on Safari (iOS or MacOS Safari too), and indeed not Android Chrome (or iOS Chrome!).

I lost at least a few days figuring out what was going on here and how to fix it, hopefully you, dear reader, won’t have to!

Synchronized Transcript on page next to video

In addition to the text track caption in the video player, I wanted to display a synchronized transcript on the page next to/near the video. It should let you scroll through the transcript independent of the video, and click on a timestamp to jump there.

Unsure of how best to fit this on the screen with what UX — I decided to look at YouTube and base my design on what they did. (On YouTube, you need to expand description and look for a “show transcript” button at bottom of it — I did make my ‘show transcript’ button easier to find!)

It shows up next the video, or when on a narrow screen right below it. In a ‘window in window’ internal scrolling box. Used some CSS to try to make the video and the transcript fit wholly on the screen at any screen size — inner scrolling window that’s higher than the parent window I consider a UX nightmare to avoid!

Looking at YouTube, I realized that feature that highlighted current cue as the video played was also one I wanted to copy. That was the trickiest thing to implement.

I ended up using the HTML5 media element api and the events emitted by it and associated child objects, based on the text track with cues I had already loaded in my video.js-enhanced html5 video player. I can let the browser track cue changes and listen for events when they change, to highlight current cue.

If a track is set to mode hidden, then the user agent will still track the text cues and emit events for when they change, even though they aren’t displayed. Video.js (and probably native players) by default have UI that toggles between shown and disabled (which does not track cue changes), so I had to write a bit of custom code to switch non-selected text tracks to hidden instead of disabled
- (Some browsers and/or video.js polyfill code may have been emitting cueChange events even on disabled tracks, contrary to or not required by spec — important to test on all browsers!)
After that, it’s just listening to the cueChange HTML5 video event emitted on the track of our auto-captions, to know that we need to de-highlight any old cues, and highlight the new ones.
Had to write code to map from the HTML5 video Cue object returned as active cue, and find the div/span on page to highlight. as simple as putting start time in a data- attribute, and matching it to startTime on Cue — except we’re string-matching, so important to output identically including digits after decimal place etc.
At first I didn’t realize I could use the user-agent’s own cue-tracking code, and was trying to catch an event on every timeUpdate event, and calculate which cues included that timestamp myself. In addition to being way more work than required (the HTML5 video API has this feature for you to use) — safari wasn’t emitting timeUpdate events unless the status bar with current time was actually on screen!
In general, the media element api and events seemed to an area with, for 2025, unusual level of differences between browsers — or at least between more native Safari and more emulated video.js in other browsers. It definitely is important to do lots of cross browser testing. While I use it rarely, when I do I couldn’t do without BrowserStack and its free offerings for open source.

Improved Video Controls

The default video.js control bar seems to me undesirably small buttons and text, and just not quite right in several ways. And there don’t seem to be very many alternative open source theme or skins (video.js seems to use both words for this), and what do exist are often kind of pushing on “interesting” aesthetics instead of being neutral/universal?

Adding the caption button was squeezing the default control bar tight, especially on small screens. With that and the increased attention to our videos that transcripts would bring, we decided to generally improve the UX of the controls, but in a neutral way that was still generic and non-branded. Again, I was guided by both youtube and the ramp player (here’s one ramp example), and also helped by ramp’s implementation (although beware some skin/theme elements are dispersed in other CSS too, not all in this file).

Before (default video.js theme)

After (local tweaked)

Scrubber/progress bar extends all the way across the the screen, above the control bar (ala youtube and ramp)
- Making sure captions stay above the now higher controls was tricky. I think this approaching using translateY works pretty well, but hadn’t seen it before? Also required a bit of safari-specific css for safari’s “native text tracks”. And some nice slide up/down animation on control bar show/hide matching youtube seems nice.
- buttons split between right and left, like again both youtube and ramp. Volume on right only cause it was somewhat easier.
Buttons themselves made bigger by default, and the icons on the buttons take up a larger portion of the button square. (They were all so tiny before!)
Underline the CC button when a text track is visible. From both youtube and ramp.
- Required some simple javascript to add/remove the visibility class appropriately.
Current time showing as current / total instead of by default elapsed, now matching youtube and what some of our users asked for. (Default video.js has some weird spacing that you have to really trim down once you show current and total).
Use newer CSS @container queries to make buttons smaller and/or remove some buttons when screen is smaller (had some weird problems with this actually crashing the video player in my actual markup though).

While fairly minor changes, I think it results in much better look and usability for a general purpose neutral theme/skin than video.js ships with out of the box. While relatively simple, it still took me a week or so to work through.

If there’s interest, I would find time to polish it up further and release it as more easily re-usable open source product, let me know?

Google indexable transcripts

One of the most exciting things about adding transcripts for our videos, is that text is now searchable and discoverable in our own web app.

It would be awfully nice if Google would index it too, so people could find otherwise hidden mentions of things they might want in videos. In the past, I’ve had trouble getting Google to index other kinds of transcripts and item text like OCRs. While hypothetically Google is visiting with javascript and can click on things like tabs or disclosure “show” buttons — conventional wisdom seems to be that Google is doens’t like to index things that aren’t on the initial page and require a click to see — which matches my experience, although others have had other experiences.

In an attempt to see if I could get google to index, I made a separate page with just transcript text — it links back to the main item page (with video player), and even offers clickable timecodes that will link back to player at that time. This transcript-only page is the href on the “Show Transcript” button, although a normal human user ordinarily would get JS executing to show transcript on same page instead when clicking on that link, you can right-click “open in new tab” to get it if you want. These extra transcript pages are also listed in my SiteMap.

There are already a few of these transcript pages showing up in google, so it seems to be a potentially useful move.

That isn’t to say how much SEO juice they have; but first step is getting them in the index, which I had trouble doing before with things that required a tab or ‘show’ click to be shown. So we’ll keep an eye on it! Of course, another option is making the transcript on-page right from the start without requiring a click to show, but I’m not sure if that really serves the user?

We also marked up our item pages with schema.org content for video, including tags around the transcript text (which is initially in DOM, but requires a ‘show transcript’ click to be visible). I honestly would not expect this to do much for increasing indexing of transcripts… I think according to google this is intended to give you a “rich snippet” for video (but not to change indexing)… but some people think Google doesn’t do too much of that anyway, and to have any chance I’d probably have to provide a persistent link to video as a contentUrl which I don’t really do. Or maybe it could make my content show up in Google “Video” tab results… but no luck there yet either. Honestly I don’t think this is going to do much of anything, but it shouldn’t hurt.

Acknowledgements

Thanks to colleagues in Code4Lib and Samvera community slack chats, for sharing their prior experiences with whisper and with video transcripts — and releasing open source code that can be used as a reference — so I didn’t have to spend my time rediscovering what they already had!

Especially generous were Mason Ballengee and Dananji Withana who work on the ramp project. And much thanks to Ryan “Harpo” Harbert for two sequential years of Code4Lib conference presentations on whisper use at WGBH (2024 video, 2025 video), and also Emily Lynema for a 2025 whisper talk.

I hope I have helped pass on a portion of their generosity by trying to share all this stuff above to keep others from having to re-discover it!

Using CloudFlare Turnstile to protect certain pages on a Rails app

I work at a non-profit academic institution, on a site that manages, searches, and displays digitized historical materials: The Science History Institute Digital Collections.

Much of our stuff is public domain, and regardless we put this stuff on the web to be seen and used and shared. (Within the limits of copyright law and fair use; we are not the copyright holders of most of it). We have no general problem with people scraping our pages.

The problem is that, like many of us, our site is being overwhelmed with poorly behaved bots. Lately one of the biggest problems is with bots clicking on every possible combination of facet limits in our “faceted search” — this is not useful for them, and it overwhelms our site. “Search” pages are one of our most resource-constrained category of page in our present site, adding to the injury. Peers say even if we scaled up (auto or not) — the bots sometimes scale up to match anyway!

One option would be putting some kind of “Web Application Firewall” (WAF) in front of the whole app. Our particular combination of team and budget and platform (heroku) makes a lot of these options expensive for us in licensing, staff time to manage, or both. Another option is certainly putting the the whole thing behind (ostensibly free) CloudFlare CDN and using its built-in WAF, but we’d like to avoid giving our DNS over to CloudFlare, I’ve heard mixed reviews of CloudFlare free staying free, and generally am trying to avoid contributing to CloudFlare’s monopoly unaccountable control of the internet.`

Although ironically then, the solution we arrived at is still using CloudFlare, but Cloudflare’s Turnstile “captcha replacement”, one of those things that gives you the “check this box” or more often entirely interactive “checking if you are a bot” UXs.

[If you’re a tldr look at the code type, here’s the initial implementation PR in our open repo, there are some bug fixes since then
Update March 18 2025: There is now a gem implementation, bot_challenge_page. It is pre-1.0 and still evolving as we learn more about the problem space]

While this still might unfortunately lock people using unconventional browsers etc out (just the latest of many complaints on HackerNews), we can use this to only protect our search pages. Most of our traffic comes directly from Google to an individual item detail page, which we can now leave completely out of it. We have complete control of allow-listing traffic based on whatever characteristics, when to present the challenge, etc. And it turns out we had a peer at another institution who had taken this approach and found it successful, so that was encouraging.

How it works: Overview

While typical documented Turnstile usage involves protecting form submissions, we actually want to protect certain urls, even when accessed via GET. Would this actually work well? What’s the best way to implement it?

Fortunately, when asking around on a chat for my professional community of librarian and archivist software hackers, Joe Corall from Lehigh University said they had done the exact same thing (even in response to the same problem, bots combinatorially exploring every possible facet value), and had super usefully written it up, and it had been working well for them.

Joe’s article and the flowchart it contains is worth looking it. His implementation is as a Drupal plugin (and used in at least several Islandora instances); the VuFind library discovery layer recently implemented a similar approach. We have a Rails app, so needed to implement it ourselves — but with Joe paving the way (and patiently answering our questions, so we could start with the parameters that worked for him), it was pretty quick work, bouyed by the confidence this approach wasn’t just an experiment in the blue, but had worked for a similar peer.

Meter the rate of access, either per IP address, or as Joe did, in buckets per sub-net of client IP address.
Once client has crossed a rate limit boundary (in Joe’s case 20 requests per 24 hour period), redirect them to a page which displays the Turnstile challenge — and has the original destination in a query param in url —
Once they have passed the Turnstile challenge, redirect them back to their original destination, which now lets them in because you’ve stored their challenge pass in some secure session state.
In that session state record that they passed, and let them avoid a challenge again for a set period of time.

Joe allow-listed certain client domain names based on reverse IP lookup, but I’ve started without that, not wanting the performance hit on every request if I can avoid it. Joe also allow-listed their “on campus” IPs, but we are not a university and only have a few staff “on campus” and I always prefer to show the staff the same thing our users are seeing — if it’s inconvenient and intolerable, we want to feel the pain so we fix it, instead of never even seeing the pain and not knowing our users are getting it!

I’m going to explain and link to how we implemented this in a Rails app, and our choices of parameters for the various parameterized things. But also I’ll tell you we’ve written this in a way that paves the way to extracting to a gem — kept everything consolidated in a small number of files and very parameterized — so if there’s interest let me know. (Code4Lib-ers, our slack is a great place to get in touch, I’m jrochkind).

Ruby and Rails details, and our parameters

Here’s the implementing PR. It is written in such a way to keep the code conslidated for future gem extraction, all in the BotDetectController class, which means kind of weirdly there is some code to inject in class methods in the controller. While it does turnstile now, it’s written with variable/class names such that analagous products could be made available.

Rack-attack to meter

We were already using rack-attack to rate-limit. We added a “track” monitor with our code to decide when a client had passed a rate-limit gate to require a challenge. We start with allowing 10 requests per 12 hours (Joe at Lehigh did 20 per 24 hours), batched together in subnets. (Joe did subnets too, but we do smaller /24 (ie x.y.z.*) for ipv4 instead of Joe’s larger /16 (x.y.*.*)).

Note that rack-attack does not use sliding/rolling-windows for rate limits, but fixed windows that reset after window period. This makes a difference especially when you use such a long period as we are, but it’s not a problem with our very low count per period, and it does keep the RAM extremely effiicent (just an integer count per rate limit bucket).

When the rate limit is reached, the rack-attack block just sets a key/value in the rack_env to tell another component that a challenge is required. (setting in the session may have worked, but we want to be absolutely sure this will work even if client is not storing cookies, and this is really only meant as this-request state, so rack env seemed the good way to set state in rack-attack that could be seen in a rails controller)

Rails before_action filter to enforce challenge

There’s a Rails before_action filter that we just put on the application-wide ApplicationController, that looks for the “bot challenge key” required in the rack env — if present, and there isn’t anything in the session saying they have already passed a bot challenge, then we redirect to a “challenge” page, that will display/activate Turnstile.

We simply put the original/destination URL in a query param on that page. (And include logic to refuse to redirect to anything but a relative path on same host, to avoid any nefarious uses).

The challenge controller

One action in our BotDetectController just displays the turnstile challenge. The cloudflare turnstile callback gives us a token we need to verify server-side with turnstile API to verify challenge was really passed.

the front-end does a JS/xhr/fetch request to the second action in our BotDetectController. The back-end verify action makes the API call to turnstile, and if challenge passed, sets a value in Rails (encrypted and signed, secure) session with time of pass, so the before_action guard can give the user access.

if the JS in front gets a go-ahead from back-end, it uses JS document.replace to go to original destination. This conveniently removes the challenge page from the user’s browser history, as if it never happened, browser back button still working great.

In most cases the challenge page, if non-interactive, wont’ be displayed for more than a few seconds. (the language has been tweaked since these screenshots).

We currently have a ‘pass’ good for 24 hours — once you pass a turnstile challenge, if your cookies/session are intact, you won’t be given another one for 24 hours no matter how much traffic. All of this is easily configurable.

If the challenge DOES fail for some reason, the user may be looking at the Challenge page with one of two kinds of failures, and some additional explanatory text and contact info.

Limitations and omissions

This particular flow only works for GET requests. It could be expanded to work for POST requests (with an invisible JS created/submitted form?), but our initial use case didn’t require it, so for now the filter just logs a warning and fails for POST.

This flow also isn’t going to work for fetch/ajax requests, it’s set up for ordinary navigation, since it redirects to a challenge then redirects back. Our use case is only protecting our search pages — but the blacklight search in our app has a JS fetch for “facet more” behavior. Couldn’t figure out a good/easy way to make this work, so for now we added an exemption config, and just exempt requests to the #facet action that look like they’re coming from fetch. Not bothered that an “attacker” could escape our bot detection for this one action; our main use case is stopping crawlers crawling indiscriminately, and I don’t think it’ll be a problem.

To get through the bot challenge requires a user-agent to have both JS and cookies enabled. JS may have been required before anyway (not sure), but cookies were not. Oh well. Only search pages are protected by the bot challenge.

The Lehigh implementation does a reverse-lookup of the client IP, and allow-lists clients from IP’s that reverse lookup to desirable and well-behaved bots. We don’t do that, in part because I didn’t want the performance hit of the reverse-lookup. We have a Sitemap, and in general, I’m not sure we need bots crawling our search results pages at all… although I’m realizing as I write this that our “Collection” landing pages are included (as they show search results)… may want to exempt them, we’ll see how it goes.

We don’t have any client-based allow-listing… but would consider just exempting any client that has a user-agent admitting it’s a bot, all our problematic behavior has been from clients with user-agents appearing to be regular browsers (but obviously automated ones, if they are being honest).

Possible extensions and enhancements

We could possibly only enable the bot challenge when the site appears “under load”, whether that’s a certain number of overall requests per second, a certain machine load (but any auto-scaling can make that an issue), or size of heroku queue (possibly same).

We could use more sophisticated fingerprinting for rate limit buckets. Instead of IP-address-based, colleague David Cliff from Northeastern University has had success using HTTP user-agent, accept-encoding, and accept-language to fingerprint actors across distributed IPs, writing:

I know several others have had bot waves that have very deep IP address pools, and who fake their user agents, making it hard to ban.

We had been throttling based on the most common denominator (url pattern), but we were looking for something more effective that gave us more resource headroom.

On inspecting the requests in contrast to healthy user traffic we noticed that there were unifying patterns we could use, in the headers.

We made a fingerprint based on them, and after blocking based on that, I haven’t had to do a manual intervention since.

def fingerprint
result = “#{env[“HTTP_ACCEPT”]} | #{env[“HTTP_ACCEPT_ENCODING”]} | #{env[“HTTP_ACCEPT_LANGUAGE”]} | #{env[“HTTP_COOKIE”]}”
Base64.strict_encode64(result)
end

…the common rule we arrived at mixed positive/negative discrimination using the above

request.env["HTTP_ACCEPT"].blank? && request.env["HTTP_ACCEPT_LANGUAGE"].blank? && request.env["HTTP_COOKIE"].blank? && (request.user_agent.blank? || !request.user_agent.downcase.include?("bot".downcase))

so only a bot that left the fields blank and lied with a non-bot user agent would be affected

We could also base rate limit or “discriminators” for rate limit buckets on info we can look up from the client IP address, either a DNS or network lookup (performance worries), or perhaps a local lookup using the free MaxMind databases that also include geocoding and some organizational info.

Does it work?

Too early to say, we just deployed it!

I sometimes get annoyed when people blog like this, but being the writer, I realized that if I wait a month to see how well it’s working to blog — I’ll never blog! I have to write while it’s fresh and still interesting to me.

But encouraged that colleagues say very similar approaches have worked for them. Thanks again to Joe Corral for paving the way with a drupal implementation, blogging it, discussing it on chat, and answering questions! And all the other librarian and cultural heritage technologists sharing knowledge and collaboration on this and many other topics!

I can say that already it is being triggered a lot, by bots that don’t seem to get past it. This includes google bot and Meta-ExternalAgent (which I guess is AI-related; we have no particular use-based objections we are trying to enforce here, just trying to preserve our resources). While Google also has no reason to combinatorially explore every facet combination (and has a sitemap), I’m not sure if I should exempt known resource-considerate bots from the challenge (and whether to do so by trusting user-agent or not; our actual problems have all been with ordinary-browser-appearing user-agents).

Update 27 Jan 2025

Our original config — allowing 10 search results per IP subnet before turnstile challenge — was not enough to keep the bot traffic from overwhelming us. Too many botnets had enough IPs making apparently fewer than 10 requests each.

Lowering that to 2 requests was enough to reduce traffic enough. (Keep in mind that a user should only get one challenge per 24 hours unless IP address changes — although that makes me realize that people using Apple’s “private browsing” feature may get more, hmm).

Pretty obvious on these heroku dashboard graphs where our succesful turnstile config was deployed, right?

I think I would be fine going down to challenge on first search results, since a human user should still only get one per 24 hour period — but since the “success passed” mark in session is tied to IP address (to avoid session replay for bots to avoid the challenge), I am now worried about Apple “private browsing”! In today’s environment with so many similar tests, I wonder if private browsing is causing problems for users and bot protections?

You can see on the graph a huge number of 3xx responses — those are our redirects to challenge page! The redirect to and display of the challenge page seem to be cheap enough that they aren’t causing us a problem even in high volume — which was the intent, nice to see it confirmed at least with current traffic.

We are only protecting our search result page, not our item detail pages (which people often get to directly to google) — this also seems succesful. The real problem was the volume of hits from so many bots trying to combinatorially explore every possible facet limit, which we have now put a stop to.

Accessing capybara-screenshot artifacts on Github CI

We test our Rails app with rspec and capybara.

For local testing, we use the capybara-screenshot plugin which “Automatically save screen shots when a Capybara scenario fails”, even when the tests were running in a headless browser you couldn’t see at all. This can be very helpful in debugging tricky capybara failures, especially ones that are “flaky” and hard to reproduce failure on.

We run all our tests automatically as CI in Github Actions.

I was running into some capybara browser tests that were failing flakily and inconsistently on Github Actions, but I could not manage to reproduce locally at all. What was going on? It would be super helpful to have access to the capybara-screenshots generated on the github actions run.

Is there a way to do it? Yes! Store them as Github Actions “artifacts“. My last two steps of my github workflow .yml look like this, the one that runs rspec, and then the one that saves any capybara-screenshot screenshot artifacts!

        - name: Run tests
          run: |
            bundle exec rspec

        - name: Archive capybara failure screenshots
          uses: actions/upload-artifact@v4
          if: failure()
          with:
            name: dist-without-markdown
            path: tmp/capybara/*.png
            if-no-files-found: ignore

I already had capybara-screenshot set up.

Now, if a capybara test fails, I can look at the screenshot filename reported for that particular failed test in the Github CI log.

And, then down under the “Archive capybara failure screenshots” action, I can find a clickable URL, which when clicked on, downloads a zip file that contains any/all archived capybara screenshots. If there are more than one, I can match filenames to the filename reported in a certain spec failure.

And I confirmed that last step with an if: failure() does not change the failure status of the job — the job is still marked by Github CI as failed, as it should be, but the archiving step still runs to archive the failure artifacts.

Very handy!

Getting rspec/capybara browser console output for failed tests

I am writing some code that does some smoke tests with capybara in a browser of some Javascript code. Frustratingly, it was failing when run in CI on Github Actions, in ways that I could not reproduce locally. (Of course it ended up being a configuration problem on CI, which you’d expect in this case). But this fact especially made me really want to see browser console output — especially errors, for failed tests, so I could get a hint of what was going wrong beyond “Well, the JS code didn’t load”.

I have some memory of being able to configure a setting in some past capybara setup, to make error output in browser console automatically fail a test and output? But I can’t find any evidence of this on the internet, and at least I’m pretty sure there is no way to do that with my current use of selenium-webdrivers and with the headless chrome to run capybara tests.

So I worked out this hacky way to add any browser console output to the failure message on failing tests only. It requires using some “private” rspec API, but this is all I could figure out. I would be curious if anyone has a better way to accomplish this goal.

Note that my goal is a bit different than “make a test fail if there’s error output in browser console”, although I’m potentially interested in that too, here I wanted: for a test that’s already failing, get the browser console output, if any, to show up in failure message.

# hacky way to inject browser logs into failure message for failed ones
  after(:each) do |example|
    if example.exception
      browser_logs = page.driver.browser.logs.get(:browser).collect { |log| "#{log.level}: #{log.message}" }

      if browser_logs.present?
        # pretty hacky internal way to get browser logs into 
        # existing long-form failure message, when that is
        # stored in exception associated with assertion failure
        new_exception = example.exception.class.new("#{example.exception.message}\n\nBrowser console:\n\n#{browser_logs.join("\n")}\n")
        new_exception.set_backtrace(example.exception.backtrace)

        example.display_exception = new_exception
      end
    end
  end

I think by default, with selenium headless chrome, you should get browser console that only includes error/warn log levels but not info, but if you aren’t getting what you want or want more you need to make a custom Capybara driver with custom loggingPrefs config that may look something like this:

Capybara.javascript_driver = :my_headless_chrome

Capybara.register_driver :my_headless_chrome do |app|
  Capybara::Selenium::Driver.load_selenium
  browser_options = ::Selenium::WebDriver::Chrome::Options.new.tap do |opts|
    opts.args << '--headless'
    opts.args << '--disable-gpu'
    opts.args << '--no-sandbox'
    opts.args << '--window-size=1280,1696'

    opts.add_option('goog:loggingPrefs', browser: 'ALL')
  end
  Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
end

Run your Rails gem CI on rails main branch

attr_json is basically an ActiveRecord extension. It works with multiple versions of Rails, so definitely runs CI on each version it supports.

But a while ago on attr_json, i set up CI to run on Rails main unreleased branch. I already was using appraisal to test under multiple Rails versions.

(which I recommend; sure it seems easy enough to do this ‘manually’ with conditionals in your Gemspec or separate Gemfiles and BUNDLE_GEMFILE — but as soon as you start needing things like different extra dependencies (version of rspec-rails anyone?) for different Rails versions… stop reinventing the wheel, appraisal just works).

So I added one more appraisal block for rails-edge, pretty straightforward. (This example also uses combustion which I don’t necessarily recommend, I think recent Rails dummy app generated by rails plugin new is fine, unlike Rails back in 5.x or whatever).

The “edge rails” CI isn’t required to pass for PR’s to be merged. I put it in it’s own separate Github Actions workflow, in part so I can give it it’s own badge on the README. (The way things are currently set up, I think you don’t even get “edge rails CI” feedback on the PR — it would be ideal to get it as feedback, but make it clear it’s in its own category and failures aren’t a blocker).

I intend this to tell the person looking at the README considering using the gem, and evaluating it’s health and making guesses about its maintenance level and effective cost of ownership: Hey, this maintainer is continually testing on unreleased Rails Edge. That’s a pretty good sign! Especially that it’s green, means it’s working on unreleased rails edge. And when the next Rails release happens, we already know it’s in a state to work on it, I won’t have to delay my Rails upgrade for this dependency.

And if a change happens on Rails edge main branch that breaks my build — I find out when it happens. If you don’t look at whether your code passes the build on (eg) Rails 7.2 until it’s released, and you find a bunch of failures — it turns out that was basically deferred maintenance waiting for you.

I find out about breakages when they happen. I fix them when I have time, but seeing that red build breakage on “Future Rails Versions” is a big motivator to get it green. (I might have called that “edge Rails” in retrospect, I think that’s a generally understood term?). And when Rails 7.2 really is released — I just need to change my gemspec to allow Rails 7.2, and release attr_json, I don’t have deferred maintenance on compat with latest Rails release piling up for me, and I can release an attr_json supporting the new Rails release immediately, and not be a blocker for my users upgrading to latest Rails release on their schedule.

This has worked out very well for me, and I would really encourage all maintainers of Rails plugins/engines to run CI on Rails edge.

Consider a small donation to rubyland.news?

I started rubyland.news a few years ago because it was a thing I wanted to see for the Ruby community. I had been feeling a shrinking of the ruby open source collaborative community, it felt like the room was emptying out.

If you find value in Rubyland News, just a few dollars contribution on my Github Sponsors page would be so appreciated.

I wanted to make people writing about ruby and what they were doing with it visible to each other and to the community, in order to try to (re)build/preserve/strengthen a self-conception as a community, connect people to each other, provide entry to newcomers, and just make it easier to find ruby news.

I develop and run rubyland.news in my spare time, as a hobby project, all by myself, on custom Rails software. I have never and will never accepted money for editorial placement — the feeds included in rubyland.news are exclusively based on my own judgement of what will serve readers and the community well.

Why am I asking for money?

The total cost of Rubyland News, including hosting and the hostname itself, are around $180 a month. Current personal regular monthly donations add up to about $100 a year — from five individual sponsors (thank you!!!!)

I pay for this out of my pocket. I’m doing totally fine, no need to worry about me, but I do work for an academic non-profit, and don’t have the commercial market software engineer income some may assume.

Sure, I could run it somewhere cheaper than heroku (and eventually might have to) — but I’m doing all this in my spare time, I don’t want to spend an iota more time or psychic energy on (to me) boring operational concerns than I need to.

Some donations would also help motivate me to keep putting energy into this, showing me that the project really does have value to the community. If I am funded to exceed my costs, I might also add resources necessary for additional features (like a non-limited DB to keep a searchable history around?)

You can donate one-time or monthly on my Github Sponsors page. The suggested levels are $1 and $5 per month. If I get an increase in $5-$10/month more contributions this year, I will consider it a huge success, it really makes a difference!

If you donate $5/month or more, and would like to be publicly listed/thanked, I am very happy to do so, just let me know!

If you don’t want to donate or can’t spare the cash, but do want to send me an email telling me about your use of rubyland news, I would love that too! I really don’t get much feedback! And would love to know any features you want or need. (With formerly-known-as-twitter being on the downslide, are there similar services you’d like to see rubyland.news published to?) jonathan at rubyland.news)

Thanks

Thanks to anyone who donates anything at all
also to anyone who sends me a note to tell me that they value Rubyland News (seriously, I get virtually no feedback — telling me things you’d like to be better/different is seriously appreciated too! Or things you like about how it is now. I do this to serve the community, and appreciate feedback and suggestions!)
To anyone who reads Rubyland News at all
To anyone who blogs about ruby, especially if you have an RSS feed, especially if you are doing it as a hobbyist/community-member for purposes other than business leads!
To my current monthly github sponsors, it means a lot!
To anyone contributing in their own way to any part of open source communities for reasons other than profit, sometimes without much recognition, to help create free culture that isn’t just about exploiting each other!

Beware sinatra, rails 7.1, rack 3, resque bundler dependency resolution

tldr practical advice for google: If you use resque 3.6.0 or less, and Rails 7.1, and are getting an error: cannot load such file -- rack/showexceptions — you probably need to add rack "~> 2.0" to your Gemfile!

The latest version of the ruby gem sinatra, as I write this, is 3.1.0, and it does not yet support the recently released rack 3. It correctly specifies that in it’s gemspec, with gem "rack", "~> 2.2", ">= 2.2.4”

[And as of this writing, that is true in sinatra github main branch too, no work has been done to allow rack 3.x]

The new Rails 7.1 does work with and allow Rack 3.x, as well as still working with Rack 2.x, it allows any rack >= 2.2.4 (specifying it will be compatible with a future rack 4.x too, which seems dangerous, for reasons, read on)

There is a version of sinatra that (wrongly) specifies working with rack 3.x: Sinatra 1.0 (Released March 2010!) specifies in it’s gemspec that it will work with any rack >= 1.0. They quickly corrected that in Sinatra 1.1a to say “~> 1.1”, meaning “1.x greater than or equal to 1.1 only”.

But sinatra 1.0 is still there in the repo, as a target for bundler dependency resolution, claiming to work fine with rack 3.x. By the way, sinatra 1.0 is wrong about that, it certainly does not work with rack 3.x. One error you might get from it is cannot load such file -- rack/showexceptions on boot, which is a lot better than a subtler error that only shows up at runtime, for sure!

Do you see where this is going?

I am in process of updating my app to Rails 7.1. I didn’t even know my app had a sinatra dependency… but it turns out it did, my app uses resque latest version 2.6.0, which has a dependency on sinatra sinatra >= 0.9.2

So okay, poor bundler has to take this dependency tree and create a resolution for it. Rails 7.1 allows rack 2 or 3; resque 2.6.0 allows any sinatra at all; sinatra 1.0 allows any rack, but sinatra 3.1.0 only allows rack 2.x.

There are two possible resolutions that satisfy those restrictions (really more than two if you can use any old version of a dependency), but the one bundler picked was:

rack 3.0.8
sinatra 1.0

Which then failed CI because sinatra 1.0 doesn’t really work with rack 3.x.

The other possible resolution would have been rack 2.2.8 and sinatra 3.1.0.

That’s the one I actually want.

To help it it along I just need to add rack "~> 2.0" to my Gemfile. This was a bit confusing to debug!

What is the problem? The danger of open-ended gem dependencies

So the problem here is sinatra 1.0 (ten years ago!) claiming it supported any rack version no matter how high! It should have said ~> 1.0 meaning “1.x, but not 2” — how could it possibly predict it would work with rack 2, or 3, or 4, or 9.0?

If sinatra 1.0 had put an upper bound on the version of rack it woudl work with, bundler would have done the ‘correct’ (to us humans) resolution out of the box, cause the ‘wrong’ one it did would not have been available as satisfying all restrictions. Doing an open-ended spec like this leaves a bomb that can get someone decades later, as it did here.

And Rails is still doing that! actionpack 7.1.x says it works with any rack >= 2.2.4 — it ought to add in a < 4 there, it knows it works with rack 2.x and 3.x, but how can it predict it works with rack 5.x or 6.x, which don’t exist at all yet? It’s leaving the same bomb for bundler dependency resolution in the future that sinatra 1.0 did, and there’s no real way to fix it once the versions are out there.

Alternately, if sinatra released a version that did support rack 3, and said so, bundler would preferentially choose that version, with rack 3, and we wouldn’t have a problem. (Bundler’s dependency resolution is actually really amazing, it’s amazing how often it makes the “right” choice among many possible versions that would satisfy all dependency restrictions) I’m not sure how much maintenance energy sinatra is getting, but eventually it’s going to have to get there or there’s going to be a conflict with something that has sinatra in it’s dependency tree and also has something that requires rack 3 in it’s dependency tree.

And more immediately… resque says it works with any sinatra >= 0.9.2 (released in 2009)…. but does it really? Who knows. Releasing a resque that says it needs, oh, sinatra >= 2.0 (released 2017) might help bundler come to a more satisfying dependency resolution… or could just result in bundler deciding to use an old version of resque so it can use an old version of sinatra which says (incorrectly!) it supprots rack 3…. hard to predict. But maybe I’ll PR resque. But resque is also not exactly overflowing with maintenance applied to it these days…

Eventually I just need to switch away from resque. I have my eye on good_job.