UWNLP ❤️ BLOOM, which utilizes two UWNLP developments:
ALiBi (position embedding that allows models to extrapolate to longer sequences) by @OfirPress et al.
bitsandbytes (8-bit optimization) by @Tim_Dettmers et al.
Theory of Mind reasoning remains challenging for language models.
Check out FANToM's, a benchmark for ToM in conversations where LLMs still haven't made ~any progress, yet the task is quite easy for humans.
Developed by researchers at AI2 & UW!
⁉️ Let's check how GPT-4o, Gemini, Llama3, Mixtral, and Claude perform on theory of mind, shall we?🌟We report new results on Benchmark FANToM👻
- GPT-4o tops the chart by finally achieving score of 2.0/100 (vs. Human 87.5)
- Huge boost for Gemini-1.5-flash compared to
Finetuning Pretrained Transformers into RNNs
Successfully converts a pretrained transformer into its efficient linear-complexity recurrent counterpart with a learned feature map to improve the efficiency while retaining the accuracy.
arxiv.org/abs/2103.13076
Hey Everyone! @uwnlp is excited to see everyone starting at @NAACLHLT tomorrow. Our students, postdocs, faculty, and collaborators all published some great research so we hope you’ll come talk to use at our tutorials, talks, posters and workshops! #NLProc
