This month I got PRs merged into cutlass, flash-attention, xformers, transformers, peft and vllm.
All relatively trivial changes, but still feels good.
FYI:
flash-attn 2.1.0 was released, with inference fixes
xformers 0.0.21 was released, with FA2 now bundled
vLLM 0.1.4 was released, with bumped xformers dependency
ffmpeg's hls (http live streaming) demuxer used to speak only http/1.0
taught it to use http keepalive and pipelining, and overall streaming throughput increased by 3x
ffmpeg.org/pipermail/ffmp…
Hacked up line_profiler to capture VRAM usage instead of clock time, and its already pretty useful!
In this example, training_step uses 4GB VRAM and optimizer state uses 160MB
github.com/pyutils/line_p…
