Harit Vishwakarma
Senior Postdoctoral Research Fellow
Dept. of Statistics, University of Oxford
[email protected]
About Me
I am a postdoc working with Prof. Yee Whye Teh at Oxford. My current focus is on studying the fundamentals and reliability of modern AI systems (GenAI / LLMs / Agents). Broadly, I am interested in ML foundations, uncertainty, reinforcement learning with focus on data efficiency and reliability.
I obtained my PhD in Computer Science from University of Wisconsin-Madison, where I was advised by Prof. Frederic Sala and Prof. Ramya Korlakai Vinayak.
Before coming to Madison I was at IBM Research, Bangalore and earned a master's in CS from
IISc, Bangalore, where I worked with Prof. Chiranjib Bhattacharyya.
Always open to chat and collaborate! so feel free to reach out :)
News
- [11/25] Joined Oxford to do a postdoc.
- [07/25] Interning at Snorkel AI studying RLVR, automating benchmark design and spatial reasoning.
- [07/25] Presented two papers at ICML. Thanks ICML for the travel award.
- On principled methods for confidence and thresholds estimation for pseudolabeling-based SSL.
- Improving language model inference using conformal prediction and test-time scaling.
- [05/25] Graduated with my PhD, yaay!!
- [01/25] Invited talk on Reliable AI: From Data to Deployment at Microsoft Research, Vancouver.
- [12/24] Excited to be at NeurIPS! Presenting works on improving confidence scores for auto-labeling and improving LLM's decision making with conformal prediction and test-time compute.
- [07/24] Papers on auto-labeling and pseudo-labeling based SSL in the DMLR workshop at ICML.
- [06/24] Excited to intern at JPMorgan AI Research, NYC with Alan Mishler and Nic Dalmasso.
- [04/24] In Valencia to present our work on OOD detection with FPR control in AISTATS. [Twitter]
- [04/24] New preprint showing how we improve confidence functions for auto-labeling.
- [01/24] Paper with Lin and Ramya on OOD Detection with FPR control got accepted in AISTATS 2024! 🎉.
- [11/23] Presented our work on auto-labeling in MLOPT Idea Seminar. [Slides].
- [10/23] Presented our work on Taming False Positives in Out of Distribution Detection at IFDS Seminar.
- [09/23] Two papers accepted in NeurIPS! 🎉 🎉.
- Checkout our work on fundamentals of auto-labeling got accepted as a spotlight .
- Checkout our work on foundations of parameter markets.
- [07/23] Presented our work on auto-labeling and OOD detection in DMLR and AI-HCI workshops @ ICML'23.
- [06/23] New blog post on aggregating LLM or foundation model objects using weak supervision.
Research Papers
F = Full/Conference Paper, S = Short/Workshop Paper
Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
J. Bauer, T. Walshe, D. Pham, H. Vishwakarma, A. Parchami, F. Sala
Under Review, 2025
Automating Benchmark Design
A. Dsouza, H. Vishwakarma, Z. Qi, J. Bauer, D. Pham, T. Walshe, A. Parchami, F. Sala, P. Varma
Under Review, 2025
Time To Impeach LLM-as-a-Judge: Programs are the Future of Evaluation
Tzu-Heng Huang, Harit Vishwakarma, Frederic Sala
ICML 2025 Workshop on Programmatic Representations for Agent Learning, 2025
Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection
Daisuke Yamada, Harit Vishwakarma, Ramya Korlakai Vinayak
Under Review, 2025
Prune 'n Predict: Optimizing LLM Decision-making with Conformal Prediction
Harit Vishwakarma, Alan Mishler, Thomas Cook, Niccolò Dalmasso, Natraj Raman, Sumitra Ganesh
International Conference on Machine Learning (ICML), 2025
Rethinking Confidence Scores and Thresholds in Pseudolabeling-based SSL
Harit Vishwakarma*, Reid (Yi) Chen*, Srinath Namburi, Sui Jiet Tay, Ramya K. Vinayak, Frederic Sala
International Conference on Machine Learning (ICML), 2025
Pearls from Pebbles: Improved Confidence Functions for Auto-labeling
Harit Vishwakarma, Reid (Yi) Chen, Sui Jiet Tay, Srinath Namburi, Frederic Sala, Ramya Korlakai Vinayak
Neural Information Processing Systems (NeurIPS), 2024
OTTER: Effortless Label Distribution Adaptation of Zero-shot Models
Changho Shin, Jitian Zhao, Sonia Cromp, Harit Vishwakarma, Frederic Sala
Neural Information Processing Systems (NeurIPS), 2024
Taming False Positives in Out-of-Distribution Detection with Human Feedback
Harit Vishwakarma, Heguang Lin, Ramya Korlakai Vinayak
International Conference on Artificial Intelligence and Statistics (AISTATS), 2024
Promises and Pitfalls of Threshold-based Auto-labeling
Harit Vishwakarma, Heguang Lin, Frederic Sala, Ramya Korlakai Vinayak
Neural Information Processing Systems (NeurIPS), 2023 (Spotlight)
ScriptoriumWS: A Code Generation Assistant for Weak Supervision
Tzu-Heng Huang, Catherine Cao, Spencer Schoenberg, Harit Vishwakarma, Nicholas Roberts, Fred Sala
Deep Learning for Code (DL4C) Workshop at ICLR, 2023
Train 'n Trade: Foundations of Parameter Markets
Tzu-Heng Huang, Harit Vishwakarma, Frederic Sala
Neural Information Processing Systems (NeurIPS), 2023
Lifting Weak Supervision To Structured Prediction
Harit Vishwakarma, Nicholas Roberts, Frederic Sala
Neural Information Processing Systems (NeurIPS), 2022
Universalizing Weak Supervision
Changho Shin, Winfred Li, Harit Vishwakarma, Nicholas Roberts, Frederic Sala
International Conference on Learning Representations (ICLR), 2022
Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient
Ankit Pensia, Shashank Rajput, Alliot Nagle, Harit Vishwakarma, Dimitris Papailiopoulos
Neural Information Processing Systems (NeurIPS), 2020 (Spotlight)
Attack of the Tails: Yes, You Really can Backdoor Federated Learning
H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, D. Papailiopoulos
Neural Information Processing Systems (NeurIPS), 2020
Quantum Embedding of Knowledge for Reasoning
D. Garg, S. Ikbal, S. K Srivastava, H. Vishwakarma, H. Karnam, L. V. Subramaniam
Neural Information Processing Systems (NeurIPS), 2019
Know Thy Neighbors, and More! Studying the Role of Context in Entity Recommendation
Sumit Bhatia, Harit Vishwakarma
ACM Conference on HyperText and Social Media (HT), 2018 (Best Paper Nominee)
An End-to-end Machine Learning Pipeline that Ensures Fairness Policies
S. Shaikh, H. Vishwakarma, S. Mehta, K. R. Varshney, K. N. Ramamurthy, D. Wei
Bloomberg Data for Good Exchange, 2017
Service
Served as a reviewer for,
- Neural Information Processing Systems (NeurIPS) 2021, 2022, 2023, 2024, 2025. Top Reviewer (~ top 8%) for NeurIPS 2022.
- International Conference on Machine Learning (ICML) 2022, 2023, 2024, 2025.
- International Conference on Learning Representations (ICLR) 2021, 2022, 2023, 2024, 2025.
- International Conference on Artificial Intelligence and Statistics (AISTATS) 2024, 2025.
- Association for the Advancement of Artificial Intelligence, Conference (AAAI) 2025.
- Transactions on Machine Learning Research Journal (TMLR) Since 2022.
Teaching
- I was TA for,
- 1. CS 761, advanced course on Mathematical Foundations of ML, Instructor Prof. Rob Nowak, [Spring, 22].
I helped with homeworks, exams, held office hours and gave a lecture on uniform convergence results for finite hypothesis class.
- 2. CS 760, grad level intro to ML course, Instructor Prof. Fred Sala, [Fall, 21 ]. I helped with homeworks, exams and held office hours.
- I have taught students at various levels in my village during my high school, partly to support my education and with the hope to get them interested.
Miscellaneous
- I enjoy sports/athletic activities. My current preference order is,
Badminton $\succ$ Tennis $\succ$ Swimming $\succ$ Racquetball/Squash $\succ$ Running $\succ$ Biking. - I like reading Indian philosophy and Hindi poetry (e.g. Kabir).
- Feel free to reach out for any of these activities :)