Charles 🎉 Frye (@charles

Charles 🎉 Frye

12.7K posts

Charles 🎉 Frye

@charles_irl

memer of technical staff at @modal. he/him. ex @full_stack_dl, @weights_biases (acq. @CoreWeave), phd Berkeley @Redwood_Neuro. try modal.com

SF // WWW

charlesfrye.github.io

Joined January 2018

Pinned
Charles 🎉 Frye
@charles_irl
Feb 9, 2024
Replying to @charles_irl
My new job is to get everyone else to see what I see in Modal: the future of data-driven computing powered by open generative models trained at the scale of the web and adapted to end-user needs by code, customization, and continual improvement. LFG.
Modal: High-performance AI infrastructure
From modal.com
104K
Charles 🎉 Frye
@charles_irl
Dec 1, 2022
10/10, no notes
Charles 🎉 Frye
@charles_irl
Aug 1, 2020
They say you die twice: the first time when your heart beats its last, the second time when the last person who knows your name dies. This indicates that life employs a garbage collector based on reference counts. In this essay, I will
Charles 🎉 Frye
@charles_irl
Dec 12, 2024
I think programming GPUs is too hard. Part of the problem is sprawling, scattered documentation & best practices. Over the past few months, we’ve been working to solve that problem, putting together a “Rosetta Stone” GPU Glossary. And now it’s live! My take-aways in thread.
242K
Charles 🎉 Frye
@charles_irl
Apr 18, 2025
Charlie Marsh
@charliermarsh
Apr 18, 2025
I would just build the whole computer out of L1 cache
81K
Charles 🎉 Frye
@charles_irl
Nov 13, 2023
PagedAttention, Virtual Context, Speculative Decoding, Register Tokens: the last year has seen many ideas from systems programming applied to LLMs. Not many folks live in that intersection, so I wrote an explainer post to make them a bit more accessible! charlesfrye.github.io/programming/20…
312K
Charles 🎉 Frye
@charles_irl
Mar 18, 2023
I've been working on neural networks for almost a decade. The best way to describe how I'm currently feeling is like a dog that caught up with the car it was chasing, sunk its teeth in the fender, and is now traveling at 80 mph -- tail wagging, jaw tiring.
212K
Charles 🎉 Frye
@charles_irl
Jun 27, 2023
"any other questions before we wrap up this interview?" "yeah - are y'all an Adderall CEO, LSD CTO startup or an LSD CEO, Adderall CTO startup?"
144K
Charles 🎉 Frye
@charles_irl
Aug 23, 2025
insane to me that 800 is divisible by 32, it just doesn't feel right
126K
Charles 🎉 Frye
@charles_irl
Jul 13, 2022
did an entire PhD on optimization just to end up here for most projects
Charles 🎉 Frye
@charles_irl
Sep 4, 2025
I like making GPUs go brrt at @modal. I wrote up what I've learned along the way in an extension to the GPU Glossary -- our "CUDA Docs for Humans". Introducing: the GPU 𝔓𝔢𝔯𝔣𝔬𝔯𝔪𝔞𝔫𝔠𝔢 Glossary. modal.com/gpu-glossary/p…
00:00
157K
Charles 🎉 Frye
@charles_irl
Aug 24, 2021
If you're like me, you've written a lot of PyTorch code without ever being entirely sure what's _really_ happening under the hood. Over the last few weeks, I've been dissecting some training runs using @PyTorch's trace viewer in @wandb. Read on to learn what I learned!
GIF
Charles 🎉 Frye
@charles_irl
Jun 21, 2024
New guide to using CUDA on @modal_labs just dropped. It began its life as a document called "I am fucking done not understanding the CUDA stack", and after readelf-ing CUDA binaries, RTFMing the driver docs, & writing homebrew kernels, I'm excited to share it with the world!
76K
Charles 🎉 Frye
@charles_irl
Sep 26, 2025
We reverse-engineered Flash Attention 4.
137K