{"@attributes":{"version":"2.0"},"channel":{"title":{},"link":"https:\/\/blog.carrio.dev","description":{},"generator":"Zola","language":"en","lastBuildDate":"Fri, 21 Nov 2025 00:00:00 +0000","item":[{"title":"TypeScript Nested Type Selection","pubDate":"Fri, 21 Nov 2025 00:00:00 +0000","author":"Unknown","link":"https:\/\/blog.carrio.dev\/blog\/typescript-nested-type-selection\/","guid":"https:\/\/blog.carrio.dev\/blog\/typescript-nested-type-selection\/","description":"

In TypeScript, you often need to take subsets of types. There are several utility types that can help you achieve this, such as Pick<\/code>, Omit<\/code>, and Partial<\/code>.<\/p>\n

However, these utility types at scale can feel pretty clunky and can lead to verbose code. To address this, I worked on a feature called nested type selection<\/strong>.<\/p>\n

If you have ever worked with GraphQL, you might recognize the pattern of nested type selection. This pattern allows you to create a new type that is a subset of an existing type, while also allowing you to specify the properties that you want to include explicitly.<\/p>\n

In large, the need for this utility type arose from the code generation capabilities of GraphQL tools. These tools often generate types that are nested within other types, and it can be difficult to extract the necessary information from these deeply nested types.<\/p>\n

To that end, I showcase here the Select<\/code> utility type.<\/p>\n

The Current Condition<\/h2>\n

Let's start with the origin type definition. This will be a type that has various nested properties which we want to pick out from the type.<\/p>\n

The original types, which might be generated by an OpenAPI or GraphQL Codegen tool, could look something like this:<\/p>\n

interface <\/span>Post {\n<\/span>  <\/span>id<\/span>: string;\n<\/span>  <\/span>title<\/span>: string;\n<\/span>  <\/span>content<\/span>: string;\n<\/span>  <\/span>author<\/span>: User;\n<\/span>}\n<\/span>\n<\/span>interface <\/span>User {\n<\/span>  <\/span>id<\/span>: string;\n<\/span>  <\/span>email<\/span>: string;\n<\/span>  <\/span>name<\/span>: string;\n<\/span>  <\/span>age<\/span>: number;\n<\/span>  <\/span>address<\/span>: {\n<\/span>    <\/span>street<\/span>: string;\n<\/span>    <\/span>city<\/span>: string;\n<\/span>    <\/span>state<\/span>: string;\n<\/span>    <\/span>zip<\/span>: string;\n<\/span>  };\n<\/span>  <\/span>friends<\/span>: Array<User>;\n<\/span>  <\/span>posts<\/span>: Array<Post>;\n<\/span>};\n<\/span><\/code><\/pre>\n
Especially with GraphQL, which allows recursive type relationships, data can often become deeply nested. Defining a type based off this normally would require a combination of Pick<\/code> or Omit<\/code> and intercepting the type with itself, redefining properties as references to the original type.<\/p>\n
Let's start with a React component. Best practices would be to define a type for this component's props that require only what is referenced.<\/p>\n
function <\/span>Post<\/span>(<\/span>props<\/span>: PostComponentProps) {\n<\/span>  <\/span>const <\/span>{ <\/span>friends <\/span>} = <\/span>props<\/span>.<\/span>post<\/span>.<\/span>author<\/span>;\n<\/span>  <\/span>const <\/span>totalFriends <\/span>= <\/span>friends<\/span>.length;\n<\/span>  <\/span>const <\/span>friendsBlock <\/span>= <\/span>useMemo<\/span>(() <\/span>=>\n<\/span>    <\/span>totalFriends <\/span>> <\/span>1\n<\/span>      ? <<\/span>p<\/span>>Friends with <<\/span>b<\/span>><\/span>{<\/span>friends<\/span>[<\/span>0<\/span>].name<\/span>}<\/span><\/<\/span>b<\/span>> and <\/span>{<\/span>totalFriends <\/span>- <\/span>1<\/span>}<\/span> others<\/<\/span>p<\/span>>\n<\/span>      : <\/span>totalFriends <\/span>=== <\/span>1\n<\/span>        ? <<\/span>p<\/span>>Friends with <<\/span>b<\/span>><\/span>{<\/span>friends<\/span>[<\/span>0<\/span>].name<\/span>}<\/span><\/<\/span>b<\/span>><\/<\/span>p<\/span>>\n<\/span>        : <\/span>null<\/span>, [<\/span>totalFriends<\/span>]);\n<\/span>\n<\/span>  <\/span>return <\/span><<\/span>div<\/span>>\n<\/span>    <<\/span>h2<\/span>><\/span>{<\/span>props<\/span>.<\/span>post<\/span>.title<\/span>}<\/span><\/<\/span>h2<\/span>>\n<\/span>    <<\/span>p<\/span>>By <\/span>{<\/span>props<\/span>.<\/span>post<\/span>.<\/span>author<\/span>.name<\/span>}<\/span><\/<\/span>p<\/span>>\n<\/span>    <\/span>{<\/span>friendsBlock<\/span>}\n<\/span>  <\/<\/span>div<\/span>>;\n<\/span>}\n<\/span><\/code><\/pre>\n
Now let's define a type based on the API generated types. If you do this ad-hoc, you might have the, albeit redundant, simple type definition:<\/p>\n
interface <\/span>PostComponentProps {\n<\/span>  <\/span>post<\/span>: {\n<\/span>    <\/span>id<\/span>: string;\n<\/span>    <\/span>title<\/span>: string;\n<\/span>    <\/span>author<\/span>: {\n<\/span>      <\/span>id<\/span>: string;\n<\/span>      <\/span>name<\/span>: string;\n<\/span>      <\/span>friends<\/span>: Array<{\n<\/span>        <\/span>id<\/span>: string;\n<\/span>        <\/span>name<\/span>: string;\n<\/span>      }\n<\/span>    }\n<\/span>  }\n<\/span>}\n<\/span><\/code><\/pre>\n
This works, but it is not totally ideal. We had to redundantly define each property's type and ensure it matches the original type, otherwise the API data could not be utilized with it. This can be cumbersome and error-prone, especially during development, when dealing with deeply nested structures, or with complicated data structures which are often the case with generated types.<\/p>\n
If it is difficult, developers may try to escape the rules of the system by referring to types directly. However, a domain model or UI component should only specify the properties it truly needs to function, not egregious top-level type definitions that can contain circular references or other complex structures.<\/p>\n
This is an example where you are putting yourself in a position where not only is the type complicated, but also virtually impossible to satisfy:<\/p>\n
interface <\/span>PostComponentProps {\n<\/span>  <\/span>post<\/span>: Post;\n<\/span>}\n<\/span><\/code><\/pre>\n
Technically, all the data we need for the component will be available, but so will a lot more data we don't need, and likely don't have available, especially with GraphQL's \"request only what you need\" philosophy.<\/p>\n
Now, the component must be passed a fully satisfied Post<\/code> object. Unfortunately, this has circular references and much more data than is needed to function.<\/p>\n
An Example with Type Selection<\/h2>\n
The alternative I propose here works very similarly to the functionality of selection sets<\/strong> in GraphQL[todo ref].<\/p>\n
You define what properties of a defined type you want to select, and the resulting type will only contain those properties. This also works for nested objects and arrays. It is a very expressive and lightweight way of defining the shape you want from some type.<\/p>\n
import <\/span>{ <\/span>Select <\/span>} <\/span>from <\/span>'<\/span>.\/select<\/span>';\n<\/span>\n<\/span>interface <\/span>PostComponentProps {\n<\/span>  <\/span>post<\/span>: Select<Post, {\n<\/span>    <\/span>id\n<\/span>    <\/span>title\n<\/span>    <\/span>author<\/span>: {\n<\/span>      <\/span>id\n<\/span>      <\/span>name\n<\/span>      <\/span>friends<\/span>: {\n<\/span>        <\/span>id\n<\/span>        <\/span>name\n<\/span>      }\n<\/span>    }\n<\/span>  }>\n<\/span>}\n<\/span><\/code><\/pre>\n
The resulting PostComponentProps type is equivalent to the inline definition in our first example. However, the type of post<\/code> is entirely described by the shape of the data we need for the component based on some known supertype Post<\/code>, ensuring compatibility with the API and practicing least privileges.<\/p>\n
In terms of behavior around arrays, it follows the same rules as GraphQL's selection sets. If you select a field that is an array, you can also select fields within the array elements. The resulting type infers the shape of the array elements based on the selection set and intelligently handles cases of arrays vs objects vs primitives.<\/p>\n
Types and the LSP Server<\/h2>\n
A major developer experience improvement offered in this approach is that the TypeScript language server will provide auto-suggestions for properties as you are filling out the Select<\/code> type. This makes it easier to ensure that the shape of the data you are using matches the shape of the data you need for the component. You can also prompt the auto-suggest in your editor and what properties are available will be displayed.<\/p>\n
The Magic Type<\/h2>\n
Here is the source for the type. There are certain limitations to this code currently, but it covers the majority of use cases. Given some interface, a selection set can be provided that describes the shape of the data you want, and the resulting type will included only the properties provided in that selection set.<\/p>\n
type <\/span>Primitive = string | number | boolean | null | undefined;\n<\/span>type <\/span>IndexableObject = Record<string, unknown>;\n<\/span>type <\/span>AnyArray = Array<unknown>;\n<\/span>type <\/span>ArrayType<T> = T <\/span>extends <\/span>Array<infer U> ? U : never;\n<\/span>type <\/span>Sentinel = undefined;\n<\/span>\n<\/span>type <\/span>SelectionOfObject<Source <\/span>extends <\/span>IndexableObject> = Partial<{\n<\/span>    [K in keyof Source]: SelectionOfUnknown<Source[K]>;\n<\/span>}>;\n<\/span>\n<\/span>type <\/span>SelectionOfArray<Source <\/span>extends <\/span>AnyArray> = Partial<{\n<\/span>    [K in keyof ArrayType<Source>]: SelectionOfUnknown<ArrayType<Source>[K]>;\n<\/span>}>;\n<\/span>\n<\/span>type <\/span>SelectionOfUnknown<Source> = Source <\/span>extends <\/span>AnyArray\n<\/span>    ? SelectionOfArray<Source>\n<\/span>    : Source <\/span>extends <\/span>IndexableObject\n<\/span>      ? SelectionOfObject<Source>\n<\/span>      : Sentinel;\n<\/span>\n<\/span>export type <\/span>Select<Source <\/span>extends <\/span>IndexableObject | AnyArray, Selection <\/span>extends <\/span>SelectionOfUnknown<Source>> = {\n<\/span>    [K in keyof Selection & keyof Source]: Selection[K] <\/span>extends <\/span>Sentinel\n<\/span>        ? Source[K]\n<\/span>        : Source[K] <\/span>extends <\/span>AnyArray\n<\/span>          ? ArrayType<Source[K]> <\/span>extends <\/span>IndexableObject\n<\/span>              ? Selection[K] <\/span>extends <\/span>SelectionOfUnknown<ArrayType<Source[K]>>\n<\/span>                  ? ArrayType<Source[K]> <\/span>extends <\/span>IndexableObject\n<\/span>                      ? Array<Select<ArrayType<Source[K]>, Selection[K]>>\n<\/span>                      : ArrayType<Source[K]> <\/span>extends <\/span>Primitive\n<\/span>                        ? Array<SelectionOfUnknown<ArrayType<Source[K]>>>\n<\/span>                        : never\n<\/span>                  : never\n<\/span>              : never\n<\/span>          : Source <\/span>extends <\/span>IndexableObject\n<\/span>            ? Selection[K] <\/span>extends <\/span>SelectionOfUnknown<Source[K]>\n<\/span>                ? Source[K] <\/span>extends <\/span>IndexableObject\n<\/span>                    ? Select<Source[K], Selection[K]>\n<\/span>                    : Source[K] <\/span>extends <\/span>Primitive\n<\/span>                      ? SelectionOfUnknown<Source[K]>\n<\/span>                      : never\n<\/span>                : never\n<\/span>            : never;\n<\/span>};\n<\/span><\/code><\/pre>\n"},{"title":"TypeScript Object Constants: A Flexible Enum Alternative","pubDate":"Mon, 10 Nov 2025 00:00:00 +0000","author":"Unknown","link":"https:\/\/blog.carrio.dev\/blog\/typescript-enum-alternative-object-constants\/","guid":"https:\/\/blog.carrio.dev\/blog\/typescript-enum-alternative-object-constants\/","description":"
Enums in TypeScript are very useful and have some utility behavior, but they are not without their limitations.<\/p>\n
The primary one you may run into, especially if you are crossing package boundaries such as generated GraphQL types, internal type definitions, or stubbing test data; is that when an enum type is expected, only a direct reference to the enum can be utilized. This is true even if the values stored in that enum are identical.<\/p>\n
Showcase: Enum Limitations<\/h3>\n
Here is a quick showcase of this behavior for enums:<\/p>\n
enum <\/span>EnumTypes {\n<\/span>  <\/span>First <\/span>= "<\/span>FIRST<\/span>",\n<\/span>  <\/span>Second <\/span>= "<\/span>SECOND<\/span>",\n<\/span>}\n<\/span>\n<\/span>function <\/span>withEnum<\/span>(<\/span>enumType<\/span>: EnumTypes) {\n<\/span>  <\/span>return <\/span>enumType<\/span>;\n<\/span>}\n<\/span>\n<\/span>withEnum<\/span>(<\/span>EnumTypes<\/span>.<\/span>First<\/span>);  <\/span>\/\/ \ud83d\udfe9 You referenced the enum value via the enum reference. Type checker has PASSED.\n<\/span>withEnum<\/span>("<\/span>FIRST<\/span>");          <\/span>\/\/ \ud83d\udca5 You referenced the equivalent enum value via a string reference. Type checker has FAILED.\n<\/span><\/code><\/pre>\n
The Object Constant Pattern<\/h3>\n
There is another common pattern that you might have seen as an alternative to TypeScript enums: Object constants.<\/p>\n
If you are not familiar with as const<\/code> syntax, this is called a const assertion<\/a><\/strong>. A quick summary is that it makes a definition a literal, immutable value. That comes with convenient behavior in that TypeScript infers that type as exactly what is defined. In addition, just like an enum cannot be modified, an object constant cannot either.<\/p>\n
This pattern is utilized more commonly now than ever; libraries like graphql-codegen<\/code> output object constants instead of enums because of their flexibility.<\/p>\n
Showcase: Flexibility in Object Constants<\/h3>\n
An example extending on the above to showcase how these differ:<\/p>\n
const <\/span>ConstTypes <\/span>= {\n<\/span>  First: "<\/span>FIRST<\/span>",\n<\/span>  Second: "<\/span>SECOND<\/span>",\n<\/span>} <\/span>as const<\/span>;\n<\/span>type <\/span>ConstType = (typeof ConstTypes)[keyof typeof ConstTypes];\n<\/span>\n<\/span>function <\/span>withConst<\/span>(<\/span>constType<\/span>: ConstType) {\n<\/span>  <\/span>return <\/span>constType<\/span>;\n<\/span>}\n<\/span>\n<\/span>\/\/ TypeScript enum compatibility:\n<\/span>withEnum<\/span>(<\/span>EnumTypes<\/span>.<\/span>First<\/span>);   <\/span>\/\/ \ud83d\udfe9 You referenced the enum value via the enum reference. Type checker has PASSED.\n<\/span>withEnum<\/span>("<\/span>FIRST<\/span>");           <\/span>\/\/ \ud83d\udca5 You referenced the equivalent enum value via a string reference. Type checker has FAILED.\n<\/span>withEnum<\/span>(<\/span>ConstTypes<\/span>.<\/span>First<\/span>);  <\/span>\/\/ \ud83d\udca5 You referenced the equivalent enum value via the const reference. Type checker has FAILED.\n<\/span>\n<\/span>\/\/ Object constant compatibilty:\n<\/span>withConst<\/span>(<\/span>EnumTypes<\/span>.<\/span>First<\/span>);  <\/span>\/\/ \ud83d\udfe9 You referenced the const value via the enum reference. Type checker has PASSED.\n<\/span>withConst<\/span>("<\/span>FIRST<\/span>");          <\/span>\/\/ \ud83d\udfe9 You referenced the equivalent via a string reference. Type checker has PASSED.\n<\/span>withConst<\/span>(<\/span>ConstTypes<\/span>.<\/span>First<\/span>); <\/span>\/\/ \ud83d\udfe9 You referenced the equivalent via the const reference. Type checker has PASSED.\n<\/span><\/code><\/pre>\n
The trick is in the additional type ConstType<\/code> declaration, which equates to a union of all possible values in the object constant. In this case \"FIRST\" | \"SECOND\"<\/code>. This still ensures correctness of value, but doesn't complain about not having used ConstTypes.First<\/code> literally when you provide values that are technically correct. You can refer to ConstType<\/code> for typing parameters and variables where you would have reached for EnumTypes<\/code> before.<\/p>\n
This makes the system much easier to integrate across package boundaries.<\/p>\n
And in the end, they are still 100% type-safe<\/strong>.<\/p>\n
If you are shipping a consumable package, are generating code, or simply run into a scenario where you have an enum that is the cause of some headache because you are strictly required to reference the enum directly, consider refactoring to an object constant as a solution.<\/p>\n"},{"title":"Nix Deploy Actions","pubDate":"Sun, 04 May 2025 00:00:00 +0000","author":"Unknown","link":"https:\/\/blog.carrio.dev\/blog\/nix-deploy-actions\/","guid":"https:\/\/blog.carrio.dev\/blog\/nix-deploy-actions\/","description":"
This topic is not a new one for those who have been following Nix. Between the Cachix Nix Action and the Determine Nix Installer Action, there are not only options but enterprise-level backing for Nix on GitHub. This post will focus on a specific example: _How I stabilized my CI deployment processes, reduced configuration drift between CI and local development, and fixed an issue that left my blog CI broken for 10 months.<\/p>\n
The Situation<\/h2>\n
I was making use of the Zola GitHub Action<\/a> (props to @shalzz<\/a>) when I initially started publishing my Zola-based blog. This was very simple to get started, and worked well for me for several months. But then one day, while the Zola binary on my system was working great, the CI action started to utilize a different version that was incompatible with my configuration as it was. This brought my deployment process to its knees, and I was reminded again of the random pain you will often suffer with systems that don't have strong consistency across environments.<\/p>\n
\n
It works on my machine \ud83e\udd37<\/em><\/p>\n<\/blockquote>\n
Reproducible Development Shells<\/h2>\n
I am already making use of Nix for a development environment. A little bit of direnv<\/code>, Nix flakes, and every time I pull my repository down, regardless if it's my Linux or macOS system, as long as I have the Nix tooling configured I will pull down the exact same development environment I had when I last worked on the project. It's truly an amazing feat. Even Docker doesn't provide reproducibility<\/em>, it gets you pretty far though. But here, just the configuration definition and the use of flakes and the locked inputs, I will always resolve the same environment.<\/p>\n
Reproducible Packages<\/h2>\n
My Nix flake defined a development shell with the packages I need, with consistency in versioning and PATH. Now I wanted to apply the same package consistency but for a command I would run. In a Nix flake, you can define a package in many different ways. In this case, it being a shell script, I was going to make use of the makeWrapper<\/code> package's wrapProgram<\/code> command and the writeScriptBin<\/code> package in Nix to take a Bash script, provide a consistent PATH of packages, and ensure the shebangs are patched to use the correct version of the shell as well. Everything would be controlled, exactly to what I need.<\/p>\n
Initial Version: Fork It!<\/h2>\n
My first version, largely to prove that it works, simply forked the source of the Zola action's entrypoint.sh<\/code>. After all, it's not like anything was really wrong with the script, just the environment it executes in. Taking this verbatim (with license and reference to upstream tacked in the header for good measure), I can use Nix to simply setup the runtime. I provide it the necessary coreutils<\/code>, git<\/code>, and zola<\/code> of course. I condense the logic largely to make the actual package line more digestible:<\/p>\n
bundleShellScript <\/span>{ <\/span>name <\/span>= "<\/span>deploy.sh<\/span>"; <\/span>filePath <\/span>= <\/span>.\/ci\/deploy.sh<\/span>; <\/span>buildInputs <\/span>= <\/span>with <\/span>pkgs<\/span>; [ <\/span>git zola coreutils <\/span>]; }\n<\/span><\/code><\/pre>\n
How does this work? Well, it orchestrates a bit of the wrapping necessary for the command. Let's see the function definition I wrote:<\/p>\n
{ <\/span>name, filePath, buildInputs, ... <\/span>}<\/span>:\n<\/span>let\n<\/span>    <\/span>command <\/span>= (<\/span>pkgs<\/span>.<\/span>writeScriptBin name <\/span>(<\/span>builtins<\/span>.<\/span>readFile filePath<\/span>)).<\/span>overrideAttrs<\/span>(old: {\n<\/span>    <\/span>buildCommand <\/span>= "<\/span>${<\/span>old<\/span>.<\/span>buildCommand<\/span>}<\/span>\\n<\/span> patchShebangs $out<\/span>";\n<\/span>    });\n<\/span>in <\/span>pkgs<\/span>.<\/span>symlinkJoin <\/span>{\n<\/span>    <\/span>inherit <\/span>name<\/span>;\n<\/span>    <\/span>paths <\/span>= [ <\/span>command <\/span>] ++ <\/span>buildInputs<\/span>;\n<\/span>    <\/span>buildInputs <\/span>= [ <\/span>pkgs<\/span>.<\/span>makeWrapper <\/span>];\n<\/span>    <\/span>postBuild <\/span>= "<\/span>wrapProgram $out\/bin\/<\/span>${<\/span>name<\/span>}<\/span> --prefix PATH : $out\/bin<\/span>";\n<\/span>}\n<\/span><\/code><\/pre>\n
Now I can give it any file reference and build inputs necessary for the command and we'll get a packaged command with all the necessary environment configuration to resolve the exact zola<\/code>, git<\/code>, etc. that I need.<\/p>\n
The Diff: GitHub Workflow Edition<\/h2>\n
So how much change was necessary on the GitHub Action workflow I was using? Well, I needed to ensure Nix was installed, for starters. But beyond that, the environment variables I pass in remain the same, it's primarily how I invoke the command<\/em> that differs.<\/p>\n
diff --git a\/.github\/workflows\/publish.yaml b\/.github\/workflows\/publish.yaml\n<\/span>index 24b5d99..b1c2bcb 100644\n<\/span>--- a\/.github\/workflows\/publish.yaml\n<\/span>+++ b\/.github\/workflows\/publish.yaml\n<\/span>@@ -11,12 +11,10 @@ <\/span>jobs:\n<\/span>     runs-on: ubuntu-latest\n<\/span>     if: github.ref != 'refs\/heads\/main' && github.ref != 'refs\/heads\/gh-pages'\n<\/span>     steps:\n<\/span>       - name: Checkout main\n<\/span>         uses: actions\/checkout@v3.0.0\n<\/span>         with:\n<\/span>           submodules: true\n<\/span>+      - uses: DeterminateSystems\/nix-installer-action@main\n<\/span>       - name: Build only\n<\/span>-        uses: shalzz\/zola-deploy-action@master\n<\/span>+        run: nix run .#deployAction\n<\/span>         env:\n<\/span>           BUILD_ONLY: true\n<\/span>           BUILD_FLAGS: --drafts\n<\/span>@@ -25,12 +23,10 @@ <\/span>jobs:\n<\/span>     runs-on: ubuntu-latest\n<\/span>     if: github.ref == 'refs\/heads\/main'\n<\/span>     steps:\n<\/span>       - name: Checkout main\n<\/span>         uses: actions\/checkout@v3.0.0\n<\/span>         with:\n<\/span>           submodules: true\n<\/span>+      - uses: DeterminateSystems\/nix-installer-action@main\n<\/span>       - name: Build and deploy\n<\/span>-        uses: shalzz\/zola-deploy-action@master\n<\/span>+        run: nix run .#deployAction\n<\/span>         env:\n<\/span>           PAGES_BRANCH: gh-pages\n<\/span>           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n<\/span><\/code><\/pre>\n
Yes. It is actually just 6 lines of code changed to accomplish this. My commit ended up catching some formatting fixes and updating the version for the checkout action, but this captures that actual requirements for migrating to using a Nix command instead of a GitHub Action. It is wildly simple<\/strong>.<\/p>\n
    \n
  1. Install Nix<\/li>\n
  2. Run Nix commands<\/li>\n
  3. ???<\/li>\n
  4. Profit<\/li>\n<\/ol>\n"},{"title":"Archiving YouTube Videos","pubDate":"Thu, 23 Jan 2025 00:00:00 +0000","author":"Unknown","link":"https:\/\/blog.carrio.dev\/blog\/archiving-youtube-videos\/","guid":"https:\/\/blog.carrio.dev\/blog\/archiving-youtube-videos\/","description":"
    It all started with a wedding. You hire a videographer, you wait patiently for several weeks, and finally your video is available. It's hosted on YouTube, quick and easy to start watching right away! However I can never know if this precious video may disappear, if their account may get deleted, or some other event may occur that causes such an important moment in our lives to disappear. This post provides a straightforward solution to archiving YouTube videos, in my case with a focus on retaining as high of quality as possible.<\/p>\n
    Downloading content from YouTube<\/h2>\n
    We are not pirating content, and have permission from our videographer to store this video. I may even have an external hard drive sitting around with it. But who wants to risk that breaking or disappearing? There is an existing open source project for downloading videos from YouTube and other sites called yt-dlp<\/strong><\/a>. This works perfectly for our scenario.<\/p>\n
    Without diving too deep into the specifics, there are many formats of content available from YouTube even for a single \"video\". Each video on YouTube can be accessed at various resolutions and audio qualities. We start off by listing what is available with the following command:<\/p>\n
    yt-dlp --list-formats 'https:\/\/www.youtube.com\/watch?v=deadbeef'<\/code><\/p>\n
    This will output a table with the available audio and video formats for the YouTube video. It is not uncommon for the formats to be completely separate, as in you have audio-only and video-only formats. We will cover the steps necessary for this scenario.<\/p>\n
    Let's suppose the output from the command looked something like this:<\/p>\n
    ID  EXT   RESOLUTION FPS CH \u2502   FILESIZE    TBR PROTO \u2502 VCODEC           VBR ACODEC      ABR ASR MORE INFO\n<\/span>\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n<\/span>sb2 mhtml 48x27        0    \u2502                   mhtml \u2502 images                                   storyboard\n<\/span>sb1 mhtml 80x45        0    \u2502                   mhtml \u2502 images                                   storyboard\n<\/span>sb0 mhtml 160x90       0    \u2502                   mhtml \u2502 images                                   storyboard\n<\/span>233 mp4   audio only        \u2502                   m3u8  \u2502 audio only           unknown             [en] Default\n<\/span>234 mp4   audio only        \u2502                   m3u8  \u2502 audio only           unknown             [en] Default\n<\/span>139 m4a   audio only      2 \u2502    2.12MiB    49k https \u2502 audio only           mp4a.40.5   49k 22k [en] low, m4a_dash\n<\/span>249 webm  audio only      2 \u2502    2.16MiB    50k https \u2502 audio only           opus        50k 48k [en] low, webm_dash\n<\/span>250 webm  audio only      2 \u2502    2.87MiB    66k https \u2502 audio only           opus        66k 48k [en] low, webm_dash\n<\/span>140 m4a   audio only      2 \u2502    5.63MiB   129k https \u2502 audio only           mp4a.40.2  129k 44k [en] medium, m4a_dash\n<\/span>251 webm  audio only      2 \u2502    5.64MiB   130k https \u2502 audio only           opus       130k 48k [en] medium, webm_dash\n<\/span>602 mp4   256x144     12    \u2502 ~  4.00MiB    92k m3u8  \u2502 vp09.00.10.08    92k video only\n<\/span>269 mp4   256x144     24    \u2502 ~  7.76MiB   179k m3u8  \u2502 avc1.4D400C     179k video only\n<\/span>160 mp4   256x144     24    \u2502    3.53MiB    81k https \u2502 avc1.4D400C      81k video only          144p, mp4_dash\n<\/span>603 mp4   256x144     24    \u2502 ~  7.32MiB   169k m3u8  \u2502 vp09.00.11.08   169k video only\n<\/span>278 webm  256x144     24    \u2502    3.71MiB    85k https \u2502 vp09.00.11.08    85k video only          144p, webm_dash\n<\/span>229 mp4   426x240     24    \u2502 ~ 13.83MiB   319k m3u8  \u2502 avc1.4D4015     319k video only\n<\/span>133 mp4   426x240     24    \u2502    7.99MiB   184k https \u2502 avc1.4D4015     184k video only          240p, mp4_dash\n<\/span>604 mp4   426x240     24    \u2502 ~ 13.51MiB   311k m3u8  \u2502 vp09.00.20.08   311k video only\n<\/span>242 webm  426x240     24    \u2502    6.30MiB   145k https \u2502 vp09.00.20.08   145k video only          240p, webm_dash\n<\/span>230 mp4   640x360     24    \u2502 ~ 31.52MiB   726k m3u8  \u2502 avc1.4D401E     726k video only\n<\/span>134 mp4   640x360     24    \u2502   15.18MiB   349k https \u2502 avc1.4D401E     349k video only          360p, mp4_dash\n<\/span>605 mp4   640x360     24    \u2502 ~ 26.07MiB   601k m3u8  \u2502 vp09.00.21.08   601k video only\n<\/span>243 webm  640x360     24    \u2502   10.97MiB   253k https \u2502 vp09.00.21.08   253k video only          360p, webm_dash\n<\/span>231 mp4   854x480     24    \u2502 ~ 58.71MiB  1353k m3u8  \u2502 avc1.4D401E    1353k video only\n<\/span>135 mp4   854x480     24    \u2502   31.68MiB   729k https \u2502 avc1.4D401E     729k video only          480p, mp4_dash\n<\/span>606 mp4   854x480     24    \u2502 ~ 41.26MiB   951k m3u8  \u2502 vp09.00.30.08   951k video only\n<\/span>244 webm  854x480     24    \u2502   18.21MiB   419k https \u2502 vp09.00.30.08   419k video only          480p, webm_dash\n<\/span>232 mp4   1280x720    24    \u2502 ~106.77MiB  2460k m3u8  \u2502 avc1.64001F    2460k video only\n<\/span>136 mp4   1280x720    24    \u2502   61.14MiB  1408k https \u2502 avc1.64001F    1408k video only          720p, mp4_dash\n<\/span>609 mp4   1280x720    24    \u2502 ~ 76.00MiB  1752k m3u8  \u2502 vp09.00.31.08  1752k video only\n<\/span>247 webm  1280x720    24    \u2502   32.92MiB   758k https \u2502 vp09.00.31.08   758k video only          720p, webm_dash\n<\/span>270 mp4   1920x1080   24    \u2502 ~215.93MiB  4976k m3u8  \u2502 avc1.640028    4976k video only\n<\/span>137 mp4   1920x1080   24    \u2502  115.89MiB  2668k https \u2502 avc1.640028    2668k video only          1080p, mp4_dash\n<\/span>614 mp4   1920x1080   24    \u2502 ~124.34MiB  2865k m3u8  \u2502 vp09.00.40.08  2865k video only\n<\/span>248 webm  1920x1080   24    \u2502   56.64MiB  1304k https \u2502 vp09.00.40.08  1304k video only          1080p, webm_dash\n<\/span>620 mp4   2560x1440   24    \u2502 ~352.28MiB  8118k m3u8  \u2502 vp09.00.50.08  8118k video only\n<\/span>271 webm  2560x1440   24    \u2502  172.62MiB  3974k https \u2502 vp09.00.50.08  3974k video only          1440p, webm_dash\n<\/span>625 mp4   3840x2160   24    \u2502 ~634.40MiB 14620k m3u8  \u2502 vp09.00.50.08 14620k video only\n<\/span>313 webm  3840x2160   24    \u2502  332.22MiB  7649k https \u2502 vp09.00.50.08  7649k video only          2160p, webm_dash\n<\/span><\/code><\/pre>\n
    There are video resolutions of 320p, 480p, 720p, etc. all the way to 2160p (4K). Additionally, there are multiple audio options available, with some useful information providing a high-level quality such as \"low\" and \"medium\". In my case, I will prefer the highest quality available for both audio and video, and make some preferential decisions on the formats. Let's go with the Opus audio codec in medium-quality and WebM video format in 2160p, or format IDs 251<\/strong> and 313<\/strong> respectively.<\/p>\n
    I will use yt-dlp<\/strong> to download each of these formats to separate files before joining them together.<\/p>\n
    # download the audio file\n<\/span>yt-dlp -f 251 -o wedding.audio.webm 'https:\/\/www.youtube.com\/watch?v=deadbeef'\n<\/span>\n<\/span># download the video file\n<\/span>yt-dlp -f 313 -o cwedding.video.webm 'https:\/\/www.youtube.com\/watch?v=deadbeef'\n<\/span><\/code><\/pre>\n
    These are both saved to WebM files, as it is only a container format that supports various video and audio formats within itself. With both files downloaded, our next task will be to join the two files together into a single, 4K AV file.<\/p>\n
    Splicing our audio and video together<\/h2>\n
    A renowned tool for working with audio and video files is ffmpeg<\/strong><\/a>. We will use this to pull the audio from the audio-only WebM file and the video from the video-only WebM file. The command we will run for this is:<\/p>\n
    ffmpeg \\\n<\/span>  -i wedding.video.webm \\\n<\/span>  -i wedding.audio.webm \\\n<\/span>  -c:v copy \\\n<\/span>  -c:a copy \\\n<\/span>  -map 0:v:0 \\\n<\/span>  -map 1:a:0 \\\n<\/span>  carrio-wedding.webm\n<\/span><\/code><\/pre>\n
    I split out the lines mostly for readability, as the command becomes more self-describing to me with this logical grouping.<\/p>\n
    We take two input files, first the video and then the audio file.<\/p>\n
    We copy over any video and audio codecs as in- no transcoding will be performed. This retains the Opus audio codec and the VP9 video codec used in these files.<\/p>\n
    We provide two mappings which dictate explicitly video and audio. For the first video (0, in a zero-index tool like ffmpeg), which had the video, we will pull the video. For the second video (1), which had the audio, we will pull the audio. The target for both is 0, the output file.<\/p>\n
    Lastly, we tell ffmpeg<\/strong> where to save the video: at wedding.webm<\/code>.<\/p>\n
    Since there was no transcoding necessary for this, the operation should complete very<\/em> quickly. In my case: 468ms.<\/p>\n
    Verifying your content<\/h2>\n
    Load your content in a compatible video player for your content! If you decided to go with WebM & Opus and targeted a high resolution like 4K, make sure you use a hardware-accelerated tool. You can also upload your content to something like Google Drive and the web player in the preview tool should automatically work for playback as well. In my case, I loaded up mpv<\/strong><\/a> to watch this beautiful moment as it included the necessary hardware acceleration libraries for my system to playback 4K content.<\/p>\n
    nix-shell -p mpv --run 'mpv .\/wedding.webm'\n<\/span><\/code><\/pre>\n
    \"mpv<\/p>\n"},{"title":"Datadog and StatsD for Serverless","pubDate":"Wed, 15 May 2024 00:00:00 +0000","author":"Unknown","link":"https:\/\/blog.carrio.dev\/blog\/datadog-statsd-serverless\/","guid":"https:\/\/blog.carrio.dev\/blog\/datadog-statsd-serverless\/","description":"
    tl;dr<\/h2>\n
    \n
    Do not use StatsD metrics with Datadog for serverless applications.<\/strong> Instead, utilize the distribution<\/code> metric type available in DogStatsD.<\/p>\n<\/blockquote>\n
    You will save yourself many hours or even days of lost time trying to get it to work. The distribution<\/code> metric type is specifically designed for serverless applications and is more resilient to data loss and function termination.<\/p>\n
    Monitoring Types<\/h2>\n
    A quick review of available types of monitoring in applications:<\/p>\n