Pinned
Happy to release Accelerated Scan, a kernel library for first order parallel associative scans in vanilla @PyTorch, Triton 2.2.0 and CUDA C++.
pip install accelerated-scan🧵
Volodymyr Kyrylov
16.2K posts
Volodymyr Kyrylov
@darkproger
unsupervised learning and rl
- super excited to have contributed to gpt-oss. We have put a lot of love into both training the model and making the developer examples, check them out:
- Backpropagation as dynamic programming over ΣΠ is generalized to compute other statistics: - max,* computes top gradient path - expectation semiring gives entropy (entropy is high when all paths are "active")
arxiv.org/abs/2307.03056 can't believe I have never heard of this before. - the model is very smart! after the release we found that the model scores 82.2 on GPQA with tools if we improve answer extractiongpt-oss is our new open-weight model family! the bigger one runs on a single GPU, you can run the small one on your laptop. Go install it right now, seriously! Telling your laptop to do something and watching it happen made me feel the AGI like nothing since ChatGPT.
- Як козаки в ЄС подавались. Proud!
- Excited for the new recurrent model in town! Implementing Hawk and Griffin in Torch is easy with Accelerated Scan:
I am incredibly proud to be able to put this paper out finally! This paper shows that hybrid linear RNNs (Griffin) combined with local attention (or sliding window attention) can be incredibly efficient at language modeling. 
- Made a demo for my new streaming Conformer Transducer for Ukrainian
00:00 - Bayesian Flow Networks link denoising diffusion and recursive Bayesian estimation. I made a tiny demo using Siren: proger.github.io/posts/bfn/norm…
- Happy to be going to @NeurIPSConf for the first time. Looking forward to meeting you there! I will be presenting a poster on Tuesday at 10:45 for our ReScience C paper [Re] VAE Approximation Error: ELBO and Exponential Families
- i had a fun conversation where moshi told me they were frustrated
00:00
