Understanding Go AI Inference: What is Inference?

Welcome to a new series! For most developers today, using a large language model means one thing: an HTTP call to somebody else’s computer. You send a prompt to an API, tokens come back, and …
Welcome! This is a blog about how things work under the hood. If you’ve ever wondered what happens when your code runs, how databases execute queries, or what compilers do with your source code, you’re in the right place.
I write deep dives into software internals—covering programming languages, compilers, databases, filesystems, and more—but with a twist: I aim to make complicated internal behaviors look simple. These aren’t exhaustive references; they’re approachable overviews that give you just enough understanding to appreciate the clever engineering behind the tools you use every day.
What to expect: Posts here explore topics like Go’s compiler phases (lexer, parser, SSA), Python’s object model and garbage collection, database query execution, filesystem structures, and runtime behaviors like memory allocation and goroutines. Each post takes a concrete example (like a “hello world” program) and walks through what happens internally.
Publishing cadence: I publish a new post every week, diving deep into a different aspect of software internals with each article.
Whether you’re a student, an intern, or a seasoned developer curious about the internals you don’t usually need to think about, I hope these posts give you those “aha!” moments that make programming even more fascinating.

Welcome to a new series! For most developers today, using a large language model means one thing: an HTTP call to somebody else’s computer. You send a prompt to an API, tokens come back, and …

In the previous article we saw how the scheduler decides which task gets the CPU — so at this point we know how programs get to run. But running isn’t enough: for a program to be useful, it …

When you build a storage engine in Go, sooner or later you need to answer a very plain question:
“How should the code read bytes from files?”
This sounds too low-level to matter. A …

In the previous article we took apart the reflect package and found that its magic is mostly the compiler leaving very good notes — type descriptors frozen into read-only data at build time, and a …

In the previous article we looked at how the kernel gives every process its own private view of memory. But memory is only half of what a process needs to actually run. The other half is the CPU …

In the previous article we watched the runtime rebuild an entire stack trace out of metadata the compiler and linker had frozen into the binary at build time. I told you at the end that reflect works …

In the previous article
we looked at how a user program crosses the ring 3 → ring 0 boundary to ask the kernel for help. The example we used was read() — a file descriptor, a buffer pointer, a byte …

In the previous article
we took apart the select statement and saw how it’s really two features in one, with the compiler rewriting the easy shapes away and only the hard cases falling through …

In the previous article
we followed the kernel from the very first instruction the bootloader handed us all the way to the moment kernel_init called execve() on /sbin/init. That was a long ride, but …

In the previous article we walked through slices, maps, and channels, and how each of them is structured under the hood. Out of those three, channels are probably the most involved one in terms of …