I implemented an LLM end-to-end in hardware, and ran it on an FPGA.
Zero Python. Zero CUDA. Just pure SysVerilog.
All my progress + everything I learned from 200h of LLM chip design (demo at the end)👇
Got nerd-sniped into robotics research 2 months ago with no prior experience
Ended up reimplementing computer vision on my Roomba with SLAM and now it's mapping the lab like a mini-explorer
Here are my 3 most surprising takeaways:
I got carried away trying to implement radix sort in CUDA today.
Ended up getting to ~0.42 billion elements sorted per sec on GH200 - not terrible for a price of 130 lines of code.
Code for the cuda curious: (and other jobless Canadians on Mem Day) github.com/Sinestro38/gpu…
#1 - Many tasks in robotics are better done without ML
Coming from a software/ML background, I asked myself why traditional robotics hasn't been replaced with ML.
I learned that ML often comes at the cost of speed & predictability over many traditional statistical approaches.
Before we dive into the project log, some context:
I made this as part of a lab project for the most cracked course at Waterloo: ECE 327. Nachiket Kapre (the Prof) designed this lab.
Because of it, I went from knowing ZERO Verilog to now squashing delta‑cycle races caused by #0
Finally, here's the culmination of this lab project - feeding a prompt into the I-BERT model deployed natively on the FPGA running tokens in, tokens out!
#3 - You learn much more by debugging and trying to figure shit out than following a course
The main difficulty in this project was surprisingly a network issue through docker and navigating a complex build system. None of these are taught in ROS courses.
...
I'd highly recommend playing around with robotics - think of an easy project in actuation/perception/control and take the first step.
I'm very bullish on humanoid robotics, and psyched to play around with more robotics stuff when I'm back in the loo post-coop 🥘
I did 3 out of a 22-hour ROS course (theconstruct.ai) and just jumped into porting the SLAM codebase I was given to ROS2.
It takes much less time to get started than I thought.
#2 - ROS[2] makes robot SW a lot more fun
To get this to work, I had to convert an existing SLAM algo developed by Kevin Hu (VIP Lab) here at UWaterloo to ROS2.
It made me realize how much of the grunt work ROS2 alleviates from someone just trying to ship a robotics project.
Phase 0: Learning system verilog
To implement anything in hw, you need to know a hardware description language (HDL). Since the course used system verilog, I got cracking on hdlbits.01xz.net
Hdlbits taught me the core syntax & structure of verilog:
• What is a module
So grateful to be starting a new chapter of growth at @ZapataComputing in late June. I'll be working as an Intern in @aperdomoortiz team.
I'm excited to learn so much about QML as well as help out with Zapata's efforts to build meaningful quantum-enabling software.