You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 21f99d6
Browse filesBrowse the repository at this point in the historyBrowse files
- Attention: each spelling's forward pass and gradient is now one complete
SQL statement. The gradient with respect to Q, K and V is a single query
(grad of three columns of a join), as fast as three statements sharing a
backward pass.
- nanoGPT: match model.py. Layer norms have learned gains (ln_1, ln_2,
ln_f), the MLP uses GELU (its tanh approximation; DataFusion has no erf),
and the residual projections use GPT-2's scaled initialization. The
hyperparameters name their GPTConfig counterparts, and the docstring
lists what still differs: size, optimizer, dropout.
- TODO(xqlsystems/ddx#142) on each window MAX wrapped in
ddx_stop_gradient.
- Introduce every project, paper and term before using it, in the demos,
their README and xarray_sql.ad; record the rule, and the TODO exception
to the no-issue-numbers rule, in AGENTS.md.
- Drop the README's "Gradients in SQL" section until the feature is ready
to announce, and the stale --lazy flag from benchmarks/ml/README.md.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01NyN7idACeDMh94qrnFMu6h
Copy file name to clipboardExpand all lines: README.md
-24Lines changed: 0 additions & 24 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -237,30 +237,6 @@ Some highlights:
237
237
* Dense tensor operations are slower in SQL than Xarray. It seems like the vanilla HashJoin implementation in standard DB engines are not as performant as array ops (they don't take advantage of contiguous array layouts), though this gives us some idea for how it could [one day be possible](https://github.com/xqlsystems/einfold).
238
238
* Sparse tensor operations are just as fast (or faster!) than Xarray today. Specifically, SQL is an excellent way to perform regridding and reprojecting between different spatial scales (see also: we include a `proj` udf via the `xarray-sql[geo]` extra).
239
239
240
-
## Gradients in SQL
241
-
242
-
With the `ml` extra (`pip install "xarray-sql[ml]"`), a `Context` takes
243
-
gradients of queries with [ddx](https://github.com/xqlsystems/ddx). In a
244
-
select list, `grad(loss, column)` is, at each row, the derivative of the
245
-
number the CTE `loss` computes with respect to that row's `column`. A
246
-
gradient-descent step on a parameter table is then one statement:
247
-
248
-
```python
249
-
ctx.sql("""
250
-
WITH pred AS (
251
-
SELECT x.sample, SUM(x.value * w.val) AS y
252
-
FROM x JOIN w ON x.feature = w.feature GROUP BY x.sample),
253
-
loss AS (
254
-
SELECT AVG(power(pred.y - t.value, 2)) AS l
255
-
FROM pred JOIN t ON pred.sample = t.sample)
256
-
SELECT feature, val - 0.1 * grad(loss, val) AS val FROM w
257
-
""")
258
-
```
259
-
260
-
[`benchmarks/ml`](benchmarks/ml) trains a small GPT and runs attention and
261
-
a GraphCast-shaped weather model this way, checked against JAX and timed on
262
-
several engines.
263
-
264
240
## Why does this work?
265
241
266
242
Underneath Xarray, Dask, and Pandas, there are NumPy arrays. These are paged in
0 commit comments