Just released our new XLM/mBERT pytorch model in 100 languages. Significantly outperforms the TensorFlow mBERT OSS model while trained on the same Wikipedia data. bit.ly/2KItiC4@GuillaumeLample@Thom_Wolf@PyTorch
DATASET RELEASE: "CC100", the CommonCrawl dataset of 2.5TB of clean unsupervised text from 100 languages (used to train XLM-R) is now publicly available.
You can find below the
Data: data.statmt.org/cc-100/
Script: bit.ly/3oC6aXy
By @VishravC et al.
👨🔬Life update: Happy to share that I recently joined @GoogleAI Language as a research scientist 👨🏫
I will continue my research on building neural networks that can learn with little to no supervision
@OpenAI#GPT4o#Audio
Extremely excited to share the results of what I've been working on for 2 years
GPT models now natively understand audio: you can talk to the Transformer itself!
The feeling is hard to describe so I can't wait for people to speak to it
#HearTheAGI 🧵1/N
Introducing GPT-4o, our new model which can reason across text, audio, and video in real time.
It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):
Happy to share our latest paper: "Self-training Improves Pretraining for Natural Language Understanding"
We show that self-training is complementary to strong unsupervised pretraining (RoBERTa) on a variety of tasks.
Paper: arxiv.org/abs/2010.02194
Code: github.com/facebookresear…
Excited to announce the creation of WaveForms AI (waveforms.ai) – an Audio LLM company aiming to solve the Speech Turing Test and bring Emotional Intelligence to AI @WaveFormsAI
Our new paper: Unsupervised Cross-lingual Representation Learning at Scale arxiv.org/pdf/1911.02116…
We release XLM-R, a Transformer MLM trained in 100 langs on 2.5 TB of text data.
Double digit gains on XLU benchmarks + strong per-language performance (~XLNet on GLUE). [1/6]
[XLSR-53: Multilingual Self-Supervised Speech Transformer]
We're happy to release XLSR-53: a wav2vec 2.0 model pre-trained on 56k hours of speech in 53 languages from MLS, CommonVoice and BABEL datasets!
Model: github.com/pytorch/fairse…
Updated paper: arxiv.org/abs/2006.13979
1/N
