alewkowycz (@alewkowycz) / X

alewkowycz

89 posts

alewkowycz

@alewkowycz

Research Scientist at Google DeepMind. Former @inflectionai, Microsoft AI. In a previous life, I did String Theory. Reasoning and Intelligence.

Joined January 2021

Pinned
alewkowycz
@alewkowycz
Jun 30, 2022
Very excited to present Minerva🦉: a language model capable of solving mathematical questions using step-by-step natural language reasoning. Combining scale, data and others dramatically improves performance on the STEM benchmarks MATH and MMLU-STEM. goo.gle/3yGpTN7
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
Evaluating our model in 2022 Poland’s National Math Exam it performed above the national average, it solved more than 80% GCSE Higher Mathematics problems and evaluated a variety of STEM undergraduate problems from MIT, it solved nearly a third of them.
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
Starting from PaLM🌴, Minerva was trained on a large dataset made of webpages with mathematical content and scientific papers. At inference time, we used chain-of-thought/scratchpad and majority voting to boost performance without the assistance of external tools.
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
Paper: bit.ly/3yuQtIU Great collaboration with Anders, @dmdohan, @ethansdyer, @hmichalewski, @vinayramasesh, @AmbroseSlone, @cem__anil, Imanol, Theo, @Yuhu_ai_, @bneyshabur, @guygr and @vedantmisra !
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
Models' mistakes are easily interpretable. Despite impressive arithmetic skills it still makes calculation mistakes. We estimate the false positive rate due to achieving the right answer from incorrect reasoning and find it relatively low. More samples: minerva-demo.github.io!
alewkowycz
@alewkowycz
Mar 19, 2024
Will be joining Microsoft AI to build the next generation of consumer AI!!! 🚀🚀
Satya Nadella
@satyanadella
Mar 19, 2024
Welcome to Microsoft, @mustafasuleyman. Thrilled to have you lead Microsoft AI as we build consumer AI, like Copilot, that is loved by and benefits people around the world.
42K
alewkowycz
@alewkowycz
Mar 24, 2021
New paper out arxiv.org/pdf/2103.12682…! We explore when learning rate schedules are beneficial. Main points: a) We present ABEL: a schedule which decays the learning rate automatically after the weight norm "bounces". It is as good and more robust than tuned schedules. 1/2
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
This was a collaboration between multiple teams at Google Research including Blueshift, Brain, and others. See research.google/teams/blueshif… for more info about our team.
alewkowycz
@alewkowycz
Jul 15, 2022
After almost three great years at Google, I decided to move on to my next adventure at @inflectionAI to work on conversational AI. Thanks to everyone at Blueshift and Brain for such great times!
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @alewkowycz
#minerva
alewkowycz
@alewkowycz
May 26, 2023
Replying to @Thom_Wolf
Where do the llama numbers come from? They seem quite different from the papers' numbers...
9.5K
alewkowycz
@alewkowycz
Jun 30, 2022
Replying to @ion_barrel
We haven't. For most of the evals we are using a fixed prompt with four examples, I would expect zero shot is a little worse? Extracting properly formatted solutions is probably harder too.
alewkowycz
@alewkowycz
Jul 1, 2022
Replying to @Tarwreck
Thanks! I think it is not unlikely to have the first AI paper in <5 years 🙃
alewkowycz
@alewkowycz
Jul 2, 2022
With current evaluations, the closest application seems education. Really curious what these models can do in terms of knowledge generation. Even if these models are interpolating in a large space, there are many holes to fill in the scientific knowledge graph!
David Dohan
@dmdohan
Jul 2, 2022
Replying to @dmdohan
The paper focuses on quantifiable problem solving, but 🦉does great at explaining technical concepts. It has read all of the arXiv after all. Curious about REINFORCE? Just prompt it to write a paper on it: `\section{A derivation of the score function gradient estimator}`