Log inSign up
Stella Biderman @ ICML
13.8K posts
user avatar
Stella Biderman @ ICML
@BlancheMinerva
Ensuring that tech companies don't have a monopoly on being able to do research on cutting edge AI @AiEleuther. She/her
stellabiderman.ai
Joined May 2019
873
Following
18.3K
Followers

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up
  • Pinned
    user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Jun 10
    In film, "we'll fix it in post" is what you say when something went wrong on set and you don't want to redo it. AI research has made it our entire methodology: train the model, then patch whatever comes out. Our new ICML oral argues this can't be the basis of a science of AI. 🧵
    54K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Jan 1, 2024
    Many people seem to think they can't do interesting LLM research outside a large lab, or are shoehorned into crowded topics. In reality, there are tons of wide-open high value questions. To prove it, I'll be tweeting one per week (every Monday) in 2024. Please steal my ideas!
    277K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Feb 7, 2025
    I had my first aha moment with OpenAI when it leaked that they had spent a year lying about that their API models being RLHF when they were really SFT. My second was when they sent anonymous legal threats to people in the OSS AI community who had GPT-4 details leaked to them.
    user avatar
    7oponaut
    @7oponaut
    Feb 7, 2025
    I had my first "aha" moment with OpenAI when they published a misleading article about being able to solve Rubik's cubes with a robot hand This was back in 2019, the same year they withheld GPT-2 for "safety" reasons. Another "aha" moment for me. When I see misleading outputs
    223K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Feb 20, 2023
    There's a lot of misinfo about this out there, so here's the definitive list of the largest open source LLMs in the world: 1. @yandexcom's YaLM-100B 2. @AiEleuther's GPT-NeoX-20B = @GoogleAI's UL2-20B = @NVIDIAAI's NeMo-Megatron-20B All are licensed under Apache 2.0
    257K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Oct 20, 2023
    The following were invented by non-profit researchers: - Alibi - Scaled RoPE - Flash Attention - Parallel attention and MLP layers - Latent diffusion - CLIP-Guidance - Training jointly on code and NL - GPT-NeoX tokenizer - 4-bit quant and qLoRA - RWKV - Instruction-tuning
    user avatar
    clem 🤗
    @ClementDelangue
    Oct 20, 2023
    Important to keep in mind that everyone in AI (including OAI) uses and benefits from open-source and wouldn’t be here without it. It’s the tide that lifts all boats!
    200K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Apr 5, 2023
    Have you ever wanted to do an experiment on LLMs and found that none of the existing model suites met your needs? At @AiEleuther we got tired of this happening and so designed a model suite that centers enabling scientific research as its primary goal
    arXiv logo
    arxiv.org
    Pythia: A Suite for Analyzing Large Language Models Across...
    How do large language models (LLMs) develop and evolve over the course of training? How do these patterns change as models scale? To answer these questions, we introduce \textit{Pythia}, a suite...
    234K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    May 3, 2022
    So… apparently @metaai is going to be releasing a 175B parameter language model for anyone to download
    arXiv logo
    arxiv.org
    OPT: Open Pre-trained Transformer Language Models
    Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. Given their computational cost, these...
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Dec 28, 2023
    Are you lost with all these new LLMs, trying to figure out what best practices are? Or even what practices different models follow? Do you wish documenting architectures didn't require delving into the code? I've done all that work so you don't have to!
    132K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Mar 29, 2022
    Every time someone asks me how to use AI to solve a social problem, I ask them why they're asking an expert in AI rather than an expert in the problem they want to use AI as a tool to solve. Image h/t @CFGeek
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Dec 28, 2023
    Replying to @BlancheMinerva
    In this spreadsheet I document a wide variety of h params on a per-model basis as well as provide my current recommendations for how to go about setting up a LLM architecture. Make sure to check out the notes tab for important caveats and warnings!
    docs.google.com
    Common LLM Settings
    51K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Jun 6, 2025
    Two years in the making, we finally have 8 TB of openly licensed data with document-level metadata for authorship attribution, licensing details, links to original copies, and more. Hugely proud of the entire team.
    user avatar
    EleutherAI
    @AiEleuther
    Jun 6, 2025
    Can you train a performant language models without using unlicensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1&2
    38K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Apr 24, 2023
    Not all memorization in LLMs is created equal. Some strings are bad to memorize, but most are quite benign. In our new paper we ask the questions "can we tell ahead of time which text will be memorized by a LLM?" and "can we change our mindset to make the answer yes?" A 🧵
    114K
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    May 1, 2022
    If you are an academic researcher interested in doing experimenting with #EleutherAI models like GPT-J-6B and GPT-NeoX-20B but lack the resources to do so, I will run the experiments for you for free. Support for reasonable finetuning is also available.
  • user avatar
    Stella Biderman @ ICML
    @BlancheMinerva
    Feb 23, 2025
    It's really disappointing to watch US media orgs consistently push out drivel about DeepSeek. The US is so obsessed with US-exceptionalism and Sinophobia that every story gets spun in the worst possible light. DeepSeek is the coolest AI start-up in the world. Write that story.
    user avatar
    The Atlantic
    @TheAtlantic
    Feb 5, 2025
    “The story of DeepSeek reveals just how much Chinese technological development continues to depend on the United States,” @MichaelSchuman writes: theatlantic.com/international/…
    45K