{"@attributes":{"version":"2.0"},"channel":{"title":"Posts on Ethan Woodbury","link":"https:\/\/ewoodbury.com\/","description":"Recent content in Posts on Ethan Woodbury","generator":"Hugo -- gohugo.io","language":"en-US","managingEditor":"ethwoodbury@gmail.com (Ethan Woodbury)","webMaster":"ethwoodbury@gmail.com (Ethan Woodbury)","copyright":"Copyright \u00a9 2025, Ethan Woodbury","lastBuildDate":"Mon, 12 Oct 2026 00:00:00 +0000","item":[{"title":"Vectorized data processing, explained","link":"https:\/\/ewoodbury.com\/vectorized-data\/","pubDate":"Mon, 12 Oct 2026 00:00:00 +0000","author":"ethwoodbury@gmail.com (Ethan Woodbury)","guid":"https:\/\/ewoodbury.com\/vectorized-data\/","description":"

Vectorization is the technique of running a tight loop over a column of values, typically a few thousand values at a time. It’s used by ClickHouse, DuckDB, Numpy, Datafusion, and all modern analytical processing engines.<\/p>\n

This contrasts with how most backend data systems work. Postgres and MySQL walk one record at a time. Spark SQL also steps one record at a time inside a stage. The values are raw bytes, not an object each. The Parquet reader is the part that is actually columnar, and it turns those columns back into rows for the rest of the plan. For analytical use cases, where filters and aggregations on large columns are common, vectorization has become the standard because it is so much faster.<\/p>\n

Vectorization works in any language; it doesn’t need a native language like C++ or Rust (a common misconception). In fact, this post builds the entire engine in pure Scala to demonstrate; the principles apply no matter what language is in use.<\/p>\n

\n

1. Row-based Engine<\/h2>\n

Let’s start with a basic query operation with a filter<\/code> and map<\/code> stage in Scala:<\/p>\n

values.<\/span>filter(<\/span>_<\/span> ><\/span> 0<\/span>).<\/span>map(<\/span>_<\/span> *<\/span> 2<\/span>).<\/span>collect()<\/span>\n<\/span><\/span><\/code><\/pre><\/div>
To implement this in a basic query engine, we define operators for each of the filter<\/code> and the map<\/code>, each of which take a function f<\/code> as a parameter. Each operator here uses a simple iterator to walk over each element in values<\/code>.<\/p>\n
def<\/span> filter[<\/span>A<\/span>](<\/span>f:<\/span> A<\/span> =><\/span> Boolean<\/span>)<\/span>:<\/span> Stage<\/span>[<\/span>A<\/span>, A<\/span>]<\/span> =<\/span> {<\/span>\n<\/span><\/span> SingleOpStage<\/span> {<\/span> p =><\/span>\n<\/span><\/span> val<\/span> it =<\/span> p.<\/span>data.<\/span>iterator.<\/span>filter(<\/span>f)<\/span>\n<\/span><\/span> Partition<\/span>(<\/span>IterUtil<\/span>.<\/span>iterableOf(<\/span>it))<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span>}<\/span>\n<\/span><\/span>\n<\/span><\/span>def<\/span> map[<\/span>A<\/span>, B<\/span>](<\/span>f:<\/span> A<\/span> =><\/span> B)<\/span>:<\/span> Stage<\/span>[<\/span>A<\/span>, B<\/span>]<\/span> =<\/span> {<\/span>\n<\/span><\/span> SingleOpStage<\/span> {<\/span> p =><\/span>\n<\/span><\/span> val<\/span> it =<\/span> p.<\/span>data.<\/span>iterator.<\/span>map(<\/span>f)<\/span>\n<\/span><\/span> Partition<\/span>(<\/span>IterUtil<\/span>.<\/span>iterableOf(<\/span>it))<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span>}<\/span>\n<\/span><\/span><\/code><\/pre><\/div>
This query walks over all elements and calls the filter<\/code> function on each, allocates memory, and creates a new Partition<\/code> data structure for that result. It then does the same with the map<\/code> function on each element, again allocating a new Partition<\/code> for the result. That’s a lot of steps!<\/p>\n
For this case, the data is not physically located together in memory. Since each int is its own object, the next value is a pointer chase away rather than at the next 4 bytes. The CPU loads each value into memory, processes it, reads the pointer for the next value, then goes all the way back to memory to fetch it.<\/p>\n
Another potential slowdown is that of type conversions, or “boxing” as it’s known in the JVM. The simple row-based Scala engine converts from the java.lang.Integer<\/code> to a raw 4-byte int before doing the comparison, then boxes the result back into java.lang.Integer<\/code>. This is true (to some extent) for all high-level languages like Scala which manage types.<\/p>\n
Due to both one-by-one processing and boxing, the row-based engine on the 1 million element list is pretty slow! The row engine’s collect<\/code> of the same query took about 195 ms and allocated about 84 MB, on 4 threads. A million raw ints are 4 MB, so that is about 20\u00d7 the data.<\/p>\n
flowchart TD\nsubgraph filt[\"Filter loop, x 1,000,000\"]\nf1[\"Chase next pointer\"] --> f2[\"Unbox the Int\"]\nf2 --> f3{\"> 0?\"}\nf3 -->|no| f1\nf3 -->|yes| f4[\"Allocate list node\"]\nf4 --> f1\nend\nmid1[\"New list, in memory\"]\nsubgraph mapl[\"Map loop, x 500,000\"]\nm1[\"Chase next pointer\"] --> m2[\"Unbox the Int\"]\nm2 --> m3[\"Multiply by 2\"]\nm3 --> m4[\"Allocate new Int and list node\"]\nm4 --> m1\nend\nfilt --> mid1 --> mapl\nstyle filt fill:#fff3bf,stroke:#e6b800\nstyle mapl fill:#fff3bf,stroke:#e6b800\n<\/div>\n
\"Row-Based,<\/p>\n
2. Vectorized Engine<\/h2>\n
The key principle to understand for vectorization is that of contiguous data. When the engine sets the data to be processed as an entire column (or vector) at once, the next value is always 4 bytes away from the current value.<\/p>\n
The column is stored in a contiguous location of memory addresses: in our case the 1 million ints take 4MB of contiguous addresses.<\/p>\n
In the engine, that column is an Array[Int]<\/code>, and the kernels are plain while<\/code> loops over it. Each step handles one int. No SIMD instruction is required, and the filter stays one int wide even when HotSpot’s auto-vectorizer is on. This is the unfused columnar engine the earlier Scarlet timings measured: two passes, one value at a time.<\/p>\n
def<\/span> filterInt32(<\/span>in:<\/span> Array<\/span>[<\/span>Int<\/span>])<\/span>:<\/span> (<\/span>Array<\/span>[<\/span>Int<\/span>],<\/span> Int<\/span>)<\/span> =<\/span> {<\/span>\n<\/span><\/span> val<\/span> out =<\/span> new<\/span> Array<\/span>[<\/span>Int<\/span>](<\/span>in.<\/span>length)<\/span>\n<\/span><\/span> var<\/span> written =<\/span> 0<\/span>\n<\/span><\/span> var<\/span> row =<\/span> 0<\/span>\n<\/span><\/span> while<\/span> (<\/span>row <<\/span> in.<\/span>length)<\/span> {<\/span>\n<\/span><\/span> if<\/span> (<\/span>in(<\/span>row)<\/span> ><\/span> 0<\/span>)<\/span> {<\/span>\n<\/span><\/span> out(<\/span>written)<\/span> =<\/span> in(<\/span>row)<\/span>\n<\/span><\/span> written +=<\/span> 1<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span> row +=<\/span> 1<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span> (<\/span>out,<\/span> written)<\/span>\n<\/span><\/span>}<\/span>\n<\/span><\/span>\n<\/span><\/span>def<\/span> projectInt32(<\/span>in:<\/span> Array<\/span>[<\/span>Int<\/span>],<\/span> length:<\/span> Int<\/span>)<\/span>:<\/span> Array<\/span>[<\/span>Int<\/span>]<\/span> =<\/span> {<\/span>\n<\/span><\/span> val<\/span> out =<\/span> new<\/span> Array<\/span>[<\/span>Int<\/span>](<\/span>length)<\/span>\n<\/span><\/span> var<\/span> row =<\/span> 0<\/span>\n<\/span><\/span> while<\/span> (<\/span>row <<\/span> length)<\/span> {<\/span>\n<\/span><\/span> out(<\/span>row)<\/span> =<\/span> in(<\/span>row)<\/span> *<\/span> 2<\/span>\n<\/span><\/span> row +=<\/span> 1<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span> out\n<\/span><\/span>}<\/span>\n<\/span><\/span>\n<\/span><\/span>val<\/span> (<\/span>filtered,<\/span> survivors)<\/span> =<\/span> filterInt32(<\/span>column)<\/span>\n<\/span><\/span>val<\/span> projected =<\/span> projectInt32(<\/span>filtered,<\/span> survivors)<\/span>\n<\/span><\/span><\/code><\/pre><\/div>
When the CPU starts processing the data in the column, it knows that the next value is always 4 bytes ahead of the current one. Therefore, it pulls an entire chunk of data, 64 bytes or 16 ints, from memory into L1 cache, before the CPU processing loop even asks for it. Then the CPU always has the next value available in L1, whereas in the row-based approach, the CPU needs to read the pointer then pull the next value from memory, for every single value.<\/p>\n
Recall the standard data storage on a CPU (sizes for an Intel i7-8550U):<\/p>\n
    \n
  1. CPU register<\/strong>: directly on the CPU, accessible in 1 CPU clock cycle, for bytes being processed<\/li>\n
  2. L1 cache<\/strong>: 32KB, accessible in about 4 clock cycles<\/li>\n
  3. L2 cache<\/strong>: 256KB, accessible in 10-15 clock cycles<\/li>\n
  4. L3 cache<\/strong>: 8MB, accessible in ~20-40 clock cycles<\/li>\n
  5. Memory (RAM)<\/strong>: 12GB, accessible in ~100-400 clock cycles<\/li>\n<\/ol>\n
    In the 1-million int example, the data to process is always ready in L3 or L1 cache, so the CPU is fully saturated and speeds through the 1 million ints for the filter step, writing the filter result back to memory. Then it speeds through the map step as well. The data fetch from RAM to L3 is usually the bottleneck in reality.<\/p>\n
    flowchart TD\nsubgraph filt[\"Filter loop, x 1,000,000\"]\nf1[\"Load next int\"] --> f2{\"> 0?\"}\nf2 -->|no| f1\nf2 -->|yes| f3[\"Store survivor\"]\nf3 --> f1\nend\nmid[\"Column, in memory\"]\nsubgraph proj[\"Map loop, x 500,000\"]\np1[\"Load filtered int\"] --> p2[\"Multiply by 2\"]\np2 --> p3[\"Store result\"]\np3 --> p1\nend\nfilt --> mid --> proj\nstyle filt fill:#fff3bf,stroke:#e6b800\nstyle proj fill:#fff3bf,stroke:#e6b800\n<\/div>\n
    \"Row-Based<\/p>\n
    3. Vectorized Engine with Fusing<\/h2>\n
    In the previous example, the engine executes two CPU instructions on each element, so each value leaves the CPU in between the oerations. The data gets pulled from memory to L3 to L1 to CPU for the first filter, only to be pushed back to L3 then L1 then CPU again for the mutiply. With data movement being the bottleneck, the overall query is limited by moving values between cache levels up to 2 million times, for only 1 million values.<\/p>\n
    This can be optimized beforehand in the planning stage, by fusing the CPU operations into a single kernel instead. The CPU can run both the filter and the multiply step while the data is still in the register, so there is no intermediate array and no data movement.<\/p>\n
    The planner matches the chain outside the loop and picks one pipeline. For this query that is Int32Pipeline.FP<\/code>, and inline<\/code> pastes both functions into its loop. One pass, one output column.<\/p>\n
    case<\/span> List<\/span>(<\/span>Filter<\/span>(<\/span>predicate),<\/span> Project<\/span>(<\/span>mapper))<\/span> =><\/span>\n<\/span><\/span> Int32Pipeline<\/span>.<\/span>FP<\/span>(<\/span>batch,<\/span> predicate,<\/span> mapper)<\/span>\n<\/span><\/span>\n<\/span><\/span>var<\/span> written =<\/span> 0<\/span>\n<\/span><\/span>var<\/span> row =<\/span> 0<\/span>\n<\/span><\/span>while<\/span> (<\/span>row <<\/span> live)<\/span> {<\/span>\n<\/span><\/span> val<\/span> value =<\/span> in(<\/span>row)<\/span>\n<\/span><\/span> if<\/span> (<\/span>predicate(<\/span>value))<\/span> {<\/span>\n<\/span><\/span> out(<\/span>written)<\/span> =<\/span> mapper(<\/span>value)<\/span>\n<\/span><\/span> written +=<\/span> 1<\/span>\n<\/span><\/span> }<\/span>\n<\/span><\/span> row +=<\/span> 1<\/span>\n<\/span><\/span>}<\/span>\n<\/span><\/span><\/code><\/pre><\/div>
    Note that fusing is separate from vectorization, and it’s been a common operation in query engines for a long time, including in Spark. With vectorization and fusing together though, for large column-heavy workloads, engines see massive performance benefits.<\/p>\n
    flowchart TD\nsubgraph kern[\"One loop, x 1,000,000\"]\na[\"Load int into register\"] --> b{\"> 0?\"}\nb -->|no| a\nb -->|yes| c[\"Multiply by 2\"]\nc --> d[\"Store once\"]\nd --> a\nend\nstyle kern fill:#fff3bf,stroke:#e6b800\n<\/div>\n
    \"Row-Based<\/p>\n
    4. SIMD<\/h2>\n
    You might have immediately thought about SIMD (Single Instruction Multiple Data), which is the more well-known form of vectorized optimizations. However we leave this until the end, to illustrate the significant gains from the simpler column loop, and from fusion, first. SIMD lives one level even deeper, within the CPU instructions themselves.<\/p>\n
    The core idea here is exactly as it sounds: a single CPU instruction running over multiple values. In this example, the operation out[i] = in[i] * 2<\/code> becomes a single AVX2 instruction operating on 8 ints (32 bytes) at once. All 8 ints are loaded into adjacent lanes as a contiguous block, the CPU shifts them all, then all 8 results are stored together.<\/p>\n
    For a dense out(i) = in(i) * 2<\/code>, a plain while<\/code> over an Array[Int]<\/code> is not only vectorized, it is also set to automatically use SIMD on modern JVM compilers such as HotSpot. That pass is SuperWord Level Parallelism, on by default, and it can be enabled or disabled as compiler flags:<\/p>\n
    java -XX:+UseSuperWord ... # on, default\njava -XX:-UseSuperWord ... # off, scalar\n<\/code><\/pre>
    The filter and fused loops above stay one int per step with the flag either way. The multiply is the loop that changes. A Scala engine can also skip the auto-vectorizer and call the instruction explicitly, using jdk.incubator.vector<\/code>. Here SPECIES_256<\/code> represents 8 ints (256 bits) processed in parallel across lanes:<\/p>\n
    import<\/span> jdk.incubator.vector.{<\/span>IntVector<\/span>,<\/span> VectorOperators<\/span>}<\/span>\n<\/span><\/span>\n<\/span><\/span>def<\/span> explicitMul(<\/span>in:<\/span> Array<\/span>[<\/span>Int<\/span>],<\/span> out:<\/span> Array<\/span>[<\/span>Int<\/span>])<\/span>:<\/span> Unit<\/span> =<\/span>\n<\/span><\/span> val<\/span> species =<\/span> IntVector<\/span>.<\/span>SPECIES_256<\/span>\n<\/span><\/span> val<\/span> lanes =<\/span> species.<\/span>length()<\/span>\n<\/span><\/span> var<\/span> i =<\/span> 0<\/span>\n<\/span><\/span> val<\/span> bound =<\/span> species.<\/span>loopBound(<\/span>in.<\/span>length)<\/span>\n<\/span><\/span> while<\/span> i <<\/span> bound do<\/span>\n<\/span><\/span> val<\/span> v =<\/span> IntVector<\/span>.<\/span>fromArray(<\/span>species,<\/span> in,<\/span> i)<\/span>\n<\/span><\/span> v.<\/span>lanewise(<\/span>VectorOperators<\/span>.<\/span>LSHL<\/span>,<\/span> 1<\/span>).<\/span>intoArray(<\/span>out,<\/span> i)<\/span>\n<\/span><\/span> i +=<\/span> lanes\n<\/span><\/span> while<\/span> i <<\/span> in.<\/span>length do<\/span>\n<\/span><\/span> out(<\/span>i)<\/span> =<\/span> in(<\/span>i)<\/span> *<\/span> 2<\/span>\n<\/span><\/span> i +=<\/span> 1<\/span>\n<\/span><\/span><\/code><\/pre><\/div>
    The 8th-gen i7 used for testing doesn’t support the newer instruction set (AVX-512) which would let it use SIMD on the filter and map together due to needing to drop the filtered values, so this test only measured the impact of SIMD onthe map. SIMD showed some improvement due to the faster CPU operations, but since this data is large enough to not fit entirely into cache, the operation is still bottlenecked by the fetch from memory which is not impacted by SIMD.<\/p>\n
    flowchart TD\nsubgraph kern[\"Dense multiply, x 125,000\"]\na[\"Load 8 ints\"] --> b[\"Shift all 8\"]\nb --> c[\"Store 8 ints\"]\nc --> a\nend\nstyle kern fill:#fff3bf,stroke:#e6b800\n<\/div>\n
    \"Map<\/p>\n
    On Languages<\/h2>\n
    I wrote these examples here in Scala, rather than a lower-level language, to illustrate the point that vectorization itself is language agnostic. In fact, my entire for-fun processing engine project, Scarlet<\/a>, is in Scala yet still beats Apache Spark easily thanks in large part due to vectorization.<\/p>\n
    Language choice has no effect once code is compiled down to the equivalent instructions within the processing loop. Of course, we saw that the JVM provides auto-vectorization and other optimizations during the compile step itself, which is in contrast to interpreted languages like Python which are stuck with executing operations as specified (although this is starting to change with the in-progress Python JIT<\/a>).<\/p>\n
    Still, low-level languages like C++ and Rust provide more direct memory control, specifically in being able to avoid the boxing and unboxing conversion cost of values, whereas JVM languages pay the boxing tax at the beginning and end of the operation when converting raw bytes. They can also specify the exact instruction: for the filter step, a C++\/Rust could the AVX-512 compress on a CPU that has it, instead of leaving that choice to the JIT compiler.<\/p>"},{"title":"Reviving a 9-year-old laptop with Linux and new hardware","link":"https:\/\/ewoodbury.com\/laptop-revival\/","pubDate":"Mon, 21 Sep 2026 00:00:00 +0000","author":"ethwoodbury@gmail.com (Ethan Woodbury)","guid":"https:\/\/ewoodbury.com\/laptop-revival\/","description":"
    I have a 2017 HP Pavilion 15-cc6xx, with an Intel i7-8550U (8th-gen) and 12GB RAM. This machine has been through a lot: it’s what got me through my last 3 years of my Chemical Engineering degree, plus my first internship and job searches. I had replaced it with a Macbook Pro M1 back in 2022, so the HP has been sitting collecting dust for 5 years now!<\/p>\n
    I had actually tried out Linux before on a very old laptop while back in high school; it didn’t go great as the hardware was even worse, I set it as dual-boot to keep Windows which caused problems, but most of all I just didn’t keep at it long enough to get it to a great state.<\/p>\n
    This time around, I did a bit more research upfront, but most importantly, coding agents exist now! I do a lot of compute cluster and infra work at work now, and agents have changed the game from Googling and searching through forums to simply asking the agent. Linux has a similar problem profile of fiddly config knobs, so with agents it seemed like a great new opportunity to try it out. It led me to some hardware upgrades as well. Spoiler - it went great!<\/p>\n
    Final results:<\/p>\n