Michal Valko - AI Researcher | Reinforcement Learning, LLMs

, Founding Researcher at a startup, researcher at Inria, and a lecturer at MVA/ENS PS.

large language models, reasoning, fine-tuning, test-time computation, reinforcement learning with human feedback, world models

News

News: new New PhD student at University of Copenhagen (2025-2028) with Giorgio Racca, A. Sanyal!
News: new Master student (2025) with Andrej Špitalský, D. Tiapkin!
News: new Congrats to Daniil Tiapkin for defending his thesis in December 2025 and joining DeepMind! Daniil Tiapkin, thesis
News: new Congrats to Lisa Bedin for defending her thesis in 2025! Lisa Bedin, thesis
News: new Congrats to Côme Fiegel for defending his thesis in 2025! Côme Fiegel
News: Game theory paper accepted to ICML 2025!
News: Teaching at MVA/ENS Paris-Saclay for 2024-2025! Graphs in Machine Learning
News: I became the Founding Researcher at a stealth startup!
News: We released Llama 3, check it out! Llama 3

News: Four LLM alignment papers accepted to ICML 2024!
News: Paper on online RLHF accepted to NeurIPS 2024!
News: I served as a senior area chair for ICML 2024.
News: I gave an invited talk on LLM alignment at ICML 2024.
News: New postdoc joined the team in 2024! Pierre Ménard
News: I gave an invited talk on reinforcement learning from human feedback at DeepMind Paris in 2024.
News: Paper on test-time computation for LLMs accepted to ICLR 2024!
News: I served as an area chair for NeurIPS 2024.
News: I gave a keynote talk on alignment at the European AI Summit 2024.
News: Paper on scalable fine-tuning accepted to AISTATS 2024!
News: I co-organized the ICML 2024 workshop on LLM alignment.
News: I gave an invited talk on online RLHF at Google Research in 2024.
News: Paper on representation learning for world models accepted to NeurIPS 2023!
News: I served as a senior area chair for NeurIPS 2023.
News: I gave an invited talk on game-theoretic approaches to alignment at Stanford in 2023.
News: Paper on efficient exploration in RL accepted to ICML 2023!
News: I gave a tutorial on reinforcement learning from human feedback at ICML 2023.
News: I served as an area chair for ICML 2023.
News: Paper on self-supervised learning for video accepted to ICCV 2023!
News: I gave an invited talk on world models at MIT in 2023.
News: Paper on bandit algorithms with graph structure accepted to AISTATS 2023!
News: I co-organized the NeurIPS 2023 workshop on representation learning.
News: I gave an invited talk on BYOL and self-supervised learning at Facebook AI Research in 2023.
News: I became a Principal Llama Scientist of Meta's GenAI in Paris! Principal Llama Scientist
News: Teaching at MVA/ENS Paris-Saclay for 2023-2024! Graphs in Machine Learning
News: IPO paper accepted to IPO, AISTATS 2024!
News: One RLHF theory and one exploration bonus paper at ICLR 2024!
News: Fast-forward ⏩ alignment research with Nash learning from human feedback!
News: Another RLHF result, this time includes learning rates for RLHF! RLHF result
News: A new RLHF paper from our group! RLHF paper
News: Teaching at MVA/ENS Paris-Saclay for 2022-2023! Graphs in Machine Learning
News: An RL paper on learning rate randomization accepted to NeurIPS 2023!
News: BIG NEWS: Outstanding paper award at ICML 2023 for our game theory work! ICML 2023, game theory work
News: Nine papers including two orals accepted to ICML 2023!
News: Teaching at MVA/ENS Paris-Saclay for 2021-2022! Graphs in Machine Learning
News: BYOL-Hindsight led by Dan Jarret accepted to NeurIPS 2022 DeepRL Workshop!
News: Apply for DeepMind Paris internships for Summer 2023. DeepMind Paris internships
News: Two papers on RL accepted to NeurIPS 2022!
News: Congrats to Jean Tarbouriech for defending his thesis on July 6th, 2022! Jean Tarbouriech
News: BYOL-Explore that performs representation learning and exploration together is out! BYOL-Explore
News: Our paper From Dirichlet to Rubin received a long oral at ICML 2022! (< 2%) From Dirichlet to Rubin
News: Three papers accepted to ICML 2022!
News: Congrats to Omar D. Domingues for defending his thesis on March 18th, 2022! Omar D. Domingues
News: BGRL accepted to ICLR 2022!
News: Two RL papers accepted to AISTATS 2022!
News: Apply for DeepMind Paris internships for Summer 2022. DeepMind Paris internships
News: Five papers accepted to NeurIPS 2021, including 1 oral and 2 spotlights! NeurIPS 2021
News: Congrats to Xuedong Shang for defending his thesis on September 29th, 2021! Xuedong Shang
News: BraVe, self-supervised learning framework for video, accepted to ICCV 2021!
News: IXOMD was invited to be presented as RL Theory seminar!
News: Our minimax SSP work was invited to be presented as RL Theory seminar! minimax SSP
News: Six papers accepted to ICML 2021, including the long talk on UCBMQ! ICML 2021
News: KeRNS algorithm for non-stationary kernel RL accepted to AISTATS 2021!
News: Three papers on reinforcement learning theory accepted to ALT 2021!
News: After 4 years, the full Spectral Bandits paper with the lower bound is now online. now online
News: I gave 3 online talks on BYOL in November and December 2020. BYOL
News: Congrats to Julien Seznec for defending his thesis on December 15th, 2020! Julien Seznec
News: Congrats to Pierre Perrault for defending his thesis on November 30th, 2020! Pierre Perrault
News: The Graphs in ML MVA course started on January 5th, 2021 and was taught by Daniele Calandriello.
News: Five papers accepted to NeurIPS 2020 including two oral talks for BYOL and DISCO and 1 spotlight! NeurIPS 2020
News: I served as an area chair for ICLR 2021.
News: Yannic Kilcher made a YouTube video about our BYOL work! YouTube video
News: Three months of lockdown led to our three months intense self-supervised learning :-). BYOL is out! BYOL is out!
News: Eight papers accepted to ICML 2020. "See" you in Vienna! ICML 2020
News: Covariance-adapting semi-bandits paper accepted to COLT 2020. "See" you in Graz! COLT 2020
News: Congrats to Guillaume Gautier for defending his thesis on May 19th, 2020! Guillaume Gautier
News: I served as an area chair for NeurIPS 2020.
News: Four papers accepted to AISTATS 2020. "See" you in Palermo! AISTATS 2020
News: I gave an invited course on reinforcement learning at Math of Machine Learning Winter School during February 19th-22nd, 2020 in Sochi, Russia. Math of Machine Learning Winter School
News: Congrats to Omar and Guillaume for their NeurIPS 2019 travel awards!
News: Four papers accepted to NeurIPS 2019. See you in Vancouver and Whistler! NeurIPS 2019
News: I served as an area chair for NeurIPS 2019.
News: I gave an invited talk during October 16th-18th, 2019 at GIF 2019, Yerevan, Armenia. GIF 2019
News: I gave a talk during September 25-26, 2019, at Recent developments in kernel methods, UCL, London, UK. Recent developments in kernel methods
News: I gave an invited talk during September 26-27, 2019 at Lancaster and DeepMind Bandit Workshop, London, UK. Lancaster and DeepMind Bandit Workshop
News: I gave an invited talk on July 23rd, 2019 for Cisco in Kraków, Poland. Cisco
News: BOLD (ANR) project accepted for 2019 - 2023 (PI: V. Perchet)
News: I gave an invited talk at Yandex HQ on July 5th, 2019 in Moscow, Russia. Yandex
News: I gave an invited talk during July 3rd-8th, 2019 at RAAI Summer School 2019, Moscow Institute of Physics and Technology. RAAI Summer School 2019
News: We organized Reinforcement Learning Summer SCOOL on 1-12 July 2019 in Lille, France. Reinforcement Learning Summer SCOOL
News: I gave an invited talk on June 14th-15th, 2019 at ICML workshop on negative dependence.
News: On June 3rd-4th, 2019 we organized The power of graphs workshop. The power of graphs, Laura Toni
News: I gave an invited talk on May 28th, 2019 at Theoretical Computer Science seminar at CU in Bratislava. Theoretical Computer Science seminar
News: A GP-UCB sparsification paper accepted to COLT 2019. See you in Phoenix! COLT 2019
News: Two papers accepted to ICML 2019. See you in Long Beach! ICML 2019
News: We organized Optimizing Human Learning 2019 workshop. Optimizing Human Learning 2019
News: I gave an invited talk on February 20th, 2019 at Data Analytics Meetings at UPJŠ in Košice. Data Analytics Meetings
News: P. Ménard joined as a postdoc! P. Ménard
News: Three papers accepted to AISTATS 2019. See you in Okinawa! AISTATS 2019
News: I gave an invited talk on January 25th, 2019 at DPMMS, U. Cambridge, UK. DPMMS
News: Two papers on black-box optimization accepted to ALT 2019! ALT 2019
News: I gave two invited talks on January 7th and 8th, 2019 at Verimag, CNRS Grenoble, France. Verimag
News: Pierre Perrault gave an invited talk on Stochastic multi-arm bandit problem, November 23rd, 2018 at Lambda seminar at Université de Bordeaux.
News: DPPy: Sampling determinantal point processes with Python released! released
News: I was on the program committee for COLT 2019.
News: Brownian motion optimization accepted to NeurIPS 2018! See you in Montréal! NeurIPS 2018
News: I gave an invited talk on September 10th-13th, 2018 at International Workshop on Optimization and Machine Learning at CIMI, Toulouse. Optimization and Machine Learning
News: A paper on optimistic optimization accepted to EWRL 2018. EWRL 2018
News: A paper on scattering for deep learning accepted to ECCV 2018. ECCV 2018
News: Starting October 1st, 2018, I taught Graphs in Machine Learning in MVA Master at ENS Paris-Saclay! Graphs in Machine Learning, MVA Master, ENS Paris-Saclay
News: A paper on distributed graph sparsification accepted to ICML 2018. See you in Stockholm! ICML 2018
News: A bandit paper on best of both worlds accepted to COLT 2018. See you in Stockholm! COLT 2018
News: Received Inria award for scientific excellence for 2018 - 2021: Prime d'excellence scientifique
News: Congrats to Daniele Calandriello for winning the prize for the Best AI Thesis from France in 2018. Inria press release, Inria article, CNRS article
News: I served as an area chair for NeurIPS 2018.
News: We organized Optimizing Human Learning workshop. Optimizing Human Learning
News: I co-organized CNRS Summer school on Networks, Graphs, and Machine Learning (RESCOM 2018) in Porquerolles, June 18-22, 2018. RESCOM 2018
News: I gave an invited talk for Workshop on Graph Learning, LINCS, Paris, on May 14th, 2018. Workshop on Graph Learning
News: I gave an invited talk at Journée Big data, Polytech'Lille on March 22nd, 2018, in Lille. Journée Big data, Polytech'Lille
News: Adobe research highlights our work on online influence maximization presented at NIPS 2017. online influence maximization
News: I gave an invited talk for GDR ISIS on February 8th, 2018 at Télécom ParisTech.
News: I gave an invited talk on January 7th, 2018 at MIST 2018.
News: Daniele Calandriello defended his thesis on December 18th, 2017.
News: I gave a talk on November 9th, 2017 at Plateau Inria Euratechnologies.
News: Two papers accepted to NIPS 2017. See you in California! NIPS 2017
News: I gave a talk on September 19th, 2017 at DeepMind, London.
News: Starting October 2nd, 2017, I taught Graphs in Machine Learning in MVA Master at ENS Paris-Saclay! Graphs in Machine Learning, MVA Master, ENS Paris-Saclay
News: Équipe associée Nord-Européenne accepted to work with A. Carpentier on Adaptive allocation of resources for recommender systems.
News: I gave a talk on September 18th, 2017 at Decision Theory and Network Science, Lancaster, UK (STOR-i 2017) Decision Theory and Network Science
News: I gave a talk on July 11th, 2017 at ICML 2017 workshop on Picky Learners. ICML 2017 workshop on Picky Learners
News: Congrats to Guillaume and Daniele for their travel grants to ICML 2017. ICML 2017
News: Two papers accepted to ICML 2017. See you in Australia! ICML 2017
News: I gave a talk on June 28th, 2017 for L'Institut de Mathématiques de Toulouse.
News: I gave a talk on June 14th, 2017 at Journées Scientifiques Inria 2017 in Nice, France.
News: I gave a popularization talk on "Comment maximiser la détection des influenceurs sur les réseaux sociaux", May 30th, 2017 at 1:45 PM at Inria Lille.
News: I gave a talk on March 22nd, 2017 at Universität Potsdam at Amazon in Berlin.
News: Spectral Bandits accepted for publication to JMLR. JMLR
News: Congrats to Daniele for receiving travel grants to AISTATS 2017. AISTATS 2017
News: Two papers accepted to AISTATS 2017. See you in Florida! AISTATS 2017
News: I gave an invited talk on December 21st, 2016 at Textkernel talks series in Amsterdam. Textkernel talks series
News: Starting October 3rd, 2016, I taught Graphs in Machine Learning in MVA Master at ENS Paris-Saclay! Graphs in Machine Learning, MVA Master, ENS Paris-Saclay
News: I gave an invited talk on September 22nd, 2016 at Theoretical Computer Science seminar in Bratislava. Theoretical Computer Science seminar
News: My habilitation thesis, Bandits on graphs and structures, is now online. Bandits on graphs and structures
News: TrailBlazer paper on sample-efficient Monte-Carlo planning accepted as oral presentation to NeurIPS 2016! NeurIPS 2016
News: I gave an invited talk on September 15th-19th, 2016 at ITAT 2016 conference at High Tatras, Slovakia. ITAT 2016
News: I gave an invited talk on June 16th, 2016 at Graph-based Learning and Graph Mining in Lille. Graph-based Learning and Graph Mining
News: On June 15th, 3:30 PM at ENS Paris-Saclay, I defended my HdR thesis on Bandits on Graphs and Structures! Bandits on Graphs and Structures
News: One paper accepted to ICML 2016 and two to UAI 2016. See you in NYC! ICML 2016, UAI 2016
News: I gave an invited talk on May 13th, 2016 at Network Science Thematic Semester, at ENS Lyon. Network Science Thematic Semester
News: Bayesian policy gradient and actor-critic algorithms accepted to JMLR. JMLR
News: Two graph bandit papers accepted to AISTATS 2016! AISTATS 2016
News: I gave an invited talk at Multi-armed Bandit Workshop in Lancaster, UK. Multi-armed Bandit Workshop
News: Starting September 28th, 2015, I taught Graphs in Machine Learning in MVA Master at ENS Paris-Saclay! Graphs in Machine Learning, MVA Master, ENS Paris-Saclay
News: Polymatroid Bandits accepted for publication to JMLR. JMLR
News: Parallel Optimistic Optimization paper accepted to NIPS 2015! NIPS 2015
News: I became a reelected member of Inria Evaluation Committee for 2015-2019. Inria Evaluation Committee
News: Two bandit papers accepted to ICML 2015 in Lille, France. ICML 2015
News: Inria's press interview with N. Vayatis and myself about MVA's Graphs in ML. French, English
News: MESSI (MaxEnt SSIRL) paper accepted to IJCAI 2015 in Argentina. IJCAI 2015
News: Intel advertising face recognition. advertising
News: EduBand, associated team project with Carnegie Mellon, accepted for 2015 - 2018 (with A. Lazaric and E. Brunskill)
News: Starting in January 2015, I taught Graphs in Machine Learning in MVA Master at ENS Paris-Saclay! Graphs in Machine Learning, MVA Master, ENS Paris-Saclay
News: Three papers on practical bandit settings accepted to NeurIPS 2014! NeurIPS 2014
News: SequeL hosted ICML 2015 in Lille! ICML 2015
News: Extra-Learn (ANR) project accepted for 2014 - 2017 (PI: A. Lazaric)
News: Ford and Intel Mobii project using Face Recognition at Engadget. Engadget
News: Ford prototype using Face Recognition at Intel. Intel
News: I was elected to be a member of Inria Evaluation Committee for 2014-2015. Inria Evaluation Committee
News: Two Spectral Bandit papers accepted to ICML 2014 and AAAI 2014! papers
News: I organized a plenary Inria talk by: Jennifer Healey on Transportation Futures (50 attendees).
News: Received Inria award for scientific excellence for 2014 - 2017: Prime d'excellence scientifique
News: Bandits attack function optimization paper accepted to CEC 2014. CEC 2014
News: We established an Erasmus agreement between École Centrale, Lille and U. Comenius, Bratislava for Computer Science. École Centrale, Lille, U. Comenius, Bratislava
News: I became CR1, an experienced junior scientist, in 2014.
News: October 2013: INTEL/Inria collaboration: Signed INTEL funded project on Internet of Things research.
News: Recorded talk from ICML 2013 on StoSOO is online. online
News: Kernelised contextual bandits paper accepted to UAI 2013, see you in Bellevue, WA! UAI 2013
News: Inria publishes an article about our work at INTEL on Face Recognition. Face Recognition
News: StoSOO paper accepted to ICML 2013! ICML 2013
News: I was on the organizing committee of JFPDA 2013. JFPDA 2013
News: FG 2013 paper accepted. See you in April in Shanghai! FG 2013
News: Our article was accepted to the JMLR post-proceedings of EWRL 2012. EWRL 2012
News: I changed my office to "Bureau A05."
News: I gave a talk at Large-Scale Online Learning and Decision-Making Workshop in Windsor, UK.
News: An article accepted to JBI 2012. JBI 2012
News: I became a junior researcher at Inria Lille - Nord Europe with SequeL team.
News: I visited Inria Sophia-Antipolis in July 12th-18th, 2012.
News: I was at ICML and EWRL 2012 in Edinburgh in July 2012.
News: EWRL 2012 paper accepted for presentation. EWRL 2012
News: I was the opening speaker at Slovak Oxford Science 2012.
News: An article accepted to International Journal of Dentistry. article
News: ICDM 2011 paper accepted. paper
News: I received my PhD in Machine Learning from University of Pittsburgh.
News: I submitted the final version of my dissertation on August 18th, 2011. final version
News: I defended my dissertation on August 1st, 2011.
News: I gave a talk at Microsoft Research on July 6th, 10:30 AM in Research Room A.
News: I was at ICML from June 24th until July 2nd.
News: ICML 2011 - Global paper accepted: Conditional Anomaly Detection Using Soft Harmonic Functions. paper
News: I received the award for Runner-Up for Best Research Poster, Elevator Pitch and Scavenger Hunt Award on CS Day, March 24th, 2011. Runner-Up for Best Research Poster, Elevator Pitch
News: I won the Computer Science Research Competition 2011. Computer Science Research Competition 2011
News: Paper accepted to the Grad Expo Conference, February 7th, 2011. Paper
News: I passed my proposal defense on December 20th, 2010, 2:30 PM.
News: I received Academic Entrepreneurship Certificate. Certificate
News: highlight: Homer Warner Best Paper Award at AMIA 2010.
News: AMIA 2010 paper accepted. paper
News: highlight: Google Best Paper Award at OLCV - CVPR 2010. Award
News: UAI 2010 paper accepted. paper
News: I was in Texas, USA from May 2nd until May 10th on tour with Pitt Men's Glee Club.
News: Intel Labs Internship during Summer 2010.
News: I was an intern at Intel Research, Santa Clara, CA. Intel Research
News: 2008 awards updated. mellon, research competition, poster 1, poster 2

older news

Bio

Michal is the Founding Researcher at a stealth startup, tenured researcher at Inria, and a lecturer at MVA at ENS Paris-Saclay. Michal is primarily interested in designing algorithms that would require as little human supervision as possible. He works on methods and settings that are able to deal with minimal feedback, such as deep reinforcement learning, bandit algorithms, self-supervised learning, or self play. Michal has recently worked on representation learning, world models and deep (reinforcement) learning algorithms that have some theoretical underpinning. In the past he has also worked on sequential algorithms with structured decisions where exploiting the structure leads to provably faster learning. Michal is now working on a new generation of large language models (LLMs), in addition to providing algorithmic solutions for their scalable test-time inference, fine-tuning and alignment. He received his PhD in 2011 from the University of Pittsburgh, before getting a tenure at Inria in 2012 and co-creating Google DeepMind Paris with R. Munos. In 2024, he became a Principal Llama Scientist at Meta, building online reinforcement learning stack and research for Llama 3.

📋long CV

📄1-page resume