Post

Log inSign up

Post

Log inSign up

John Yang on X: "Simply connecting an LM to a vanilla bash terminal does not work well. Our key insight is that LMs require carefully designed agent-computer interfaces (similar to how humans like good UI design) E.g. When the LM messes up indentation, our editor prevents it and gives feedback"

Conversation

Log in or sign up for X

See what’s happening and join the conversation

Continue with phone
or
Log in with username or email

Relevant people

Avatar
John Yang@jyangballinFollow
CS PhD @Stanford. Creator of @SWEbench (multi-lingual/modal); SWE-agent; SWE-smith; SWE-sweep; InterCode; CodeClash; ProgramBench

What’s happening

Show more
Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
  • @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    SWE-agent is our new system for autonomously solving issues in GitHub repos. It gets similar accuracy to Devin on SWE-bench, takes 93 seconds on avg + it's open source! We designed a new agent-computer interface to make it easy for GPT-4 to edit+run code github.com/princeton-nlp/…
    44
  • @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    SWE-agent works by interacting with a specialized terminal, which allows it to: 🔍 Open, scroll and search through files ✍️ Edit specific lines w/ automatic syntax check 🧪 Write and execute tests This custom-built interface is critical for good performance! 2/N
    1
  • @jyangballin
    John Yang
    @jyangballin
    Simply connecting an LM to a vanilla bash terminal does not work well. Our key insight is that LMs require carefully designed agent-computer interfaces (similar to how humans like good UI design) E.g. When the LM messes up indentation, our editor prevents it and gives feedback
    10:53 AM · Apr 2, 2024·
    98.6K
    Views
    2
  • @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    Another example is that we discovered that for viewing files, letting SWE-agent only view 100 lines at a time was better than letting it view 200 or 300 lines and much better than letting it view the entire file. Good agent-computer design is important even when using GPT-4. 4/N
    2
    @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    SWE-agent can be easily configured and extended to improve future research on software engineering agents. Since SWE-agent is open source, anyone can experiment with and contribute new ways for agents to interact with computers. 5/N
    1
    @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    Check out some cool demonstrations of SWE-agent fixing real GitHub issues at swe-agent.com! 6/N
    1
    @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    SWE-agent is a Princeton NLP collaboration by @jyangballin*, @_carlosejimenez*, @_awettig, @ShunyuYao12, @karthik_r_n, and @OfirPress We’d love to hear your thoughts, comments and questions! Here or on our discord at discord.gg/AVEFbBn2rH 7/7
    discord.com
    Join the SWE-bench Discord Server!
    Check out the SWE-bench community on Discord - hang out with 1990 other members and enjoy free voice and text chat.
    3
    @jyangballin
    John Yang
    @jyangballin
    Apr 2, 2024
    Preprint coming next week!
    4
  • @Tomorrows_Gone_
    tomorrowsgone
    @Tomorrows_Gone_
    Apr 7, 2024
    @TA_c14y this is cool, gotta give the AI good ux