Thanks @karpathy, for putting together an even friendlier GPT repo (again ๐).
I also see you are using @wandb for your experiment tracking on nanoGPT ๐.
github.com/karpathy/nanoGโฆ
If you need something, please let us know.
I wish you a happy new year ๐ !
I have been using @OpenAI whisper on my Mac the last couple of days. With the base.en model, transcribing a 5 minute audio takes less than 20 seconds on my Mac (using the CPU!)
This ๐ C++ port by @ggerganov is what you need! It's fast and very easy to install.
How are model providers making money serving Llama 405B?
A 8xH100 node costs around 1k per day. It can serve Llama 405B at ~300tok/s (with ten batched requests).
> That's 26M tokens per day, at $3/Mtokens = $78
๐ฅNew PyTorch 1.11 is out ๐ฅ so I decided to try the torchdata API to build data processing pipelines.
wandb.me/torchdata
This feature enable the user to construct reusable blocks to process data, it has so many potential uses ๐ฑ!
If you are still confused by all the benchmarks and evaluation pipelines for LLMs, you should definitely watch @dk21 latest video from the LLM fine-tune course.
llm-course.wandb.events
He covers all the standard used benchmarks, like HumanEval, HellaSwag, ARC and standard
We are hiring a Machine Learning Engineer on our Team!!
If you have been finetunning LLMs, deploying models into productions or curating the best possible datasets and want to work on a remote company that thrives, apply here:
jobs.lever.co/wandb/117d7374โฆ
If you want to run Stable Diffusion natively on your Mac, the fastest option is Tf/Keras right now.
You can check my analysis here:
wandb.me/mac_diffusion
The model Screenshot 2022-10-06 at 10.32.31compilation of Tf enables way better Metal GPU optimisation of the model.
There is a great interest in running LLMs locally. From a corporate and personal perspective, keeping your data within your computer/infra could be beneficial. In recent months, a project from @ggerganov to make LLMs run as fast as possible has taken the ML world by storm.
