Howard Chen (@__howardchen) / X

Pinned

There are few moments in a lifetime to be part of something that gives you a sneak peek into the future, and I’ve been seeing sparks of something different firsthand @EngramLab since I joined earlier this year. Many exciting things to come soon!

Introducing Engram: Scaling compute on your context

We’re Engram. We’re building AI that learns from you and deeply understands your work. Today’s AI models don’t understand what you do. Not really. Everything models know comes from their training –...

Long context models are popular, but is it the final solution to long text reading? We introduce a fundamentally different method, MemWalker: 1. Build a data structure (memory tree) 2. Traverse it via LLM prompting Outperforms long context, retrieval, & recurrent baselines. (1/n)

Large language models are strong, but they’re not without limitations! It’s often hard to make them follow exactly *what to generate* and *what not to generate*. How do we study and improve it? Introducing the 🥃 Cognac task & benchmark! Preprint: arxiv.org/abs/2212.10466 (1/N)

arxiv.org

Controllable Text Generation with Language Constraints

We consider the task of text generation in language models with constraints specified in natural language. To this end, we first create a challenging benchmark Cognac that provides as input to the...

I'll be giving the oral presentation of our paper "Can Rationalization Improve Robustness?" at #NAACL2022 (Tue Session 4A; 10:45 – 12:15am PDT). Come check it out if you're interested in the intersection between interpretability and robustness! Paper: arxiv.org/abs/2204.11790

Howard Chen

@__howardchen

Oct 10, 2023

Replying to @__howardchen

Check out more from our the paper: arxiv.org/abs/2310.05029 Work done with the great collaborator & mentors @ramakanth1729, @jaseweston, @real_asli. (8/8)

Replying to @__howardchen

MemWalker is a two-stage process: 1) constructing the memory tree: the long text is first segmented and summarized into summary nodes. The summary nodes are further summarized into higher level nodes until it reaches the root. (2/n)

Replying to @__howardchen

2) navigation: upon seeing a query, the LLM navigates the tree to look for relevant information and crafts an appropriate response. The LLM achieves this by reasoning – whether to commit to an answer, choose to go further down the path, or retract it finds itself astray. (3/n)

Today marks my last day at ASAPP (@asapp) as an NLP Research Engineer after almost 3 wonderful years. It's been an amazing ride working with such a top-notch team! I'll be starting as a PhD student in CS at @Princeton this fall. Super excited to start the new chapter!

Howard Chen

@__howardchen

Oct 10, 2023

Replying to @__howardchen

The navigation can be achieved by zero-shot prompting and is readily applicable to any LLM of your choice. (4/n)

Replying to @__howardchen

And 2) the reasoning ability of the LLM – MemWalker is effective when the LLM passes a reasoning threshold. Error cascades too fast during navigation when the reasoning capability is below the threshold. (7/n)

Replying to @__howardchen

Two critical components enable the effectiveness of MemWalker: 1) working memory – When the LLM is allowed to carry information along the path it traverses, the LLM has a better global context (6/n)

Replying to @__howardchen

We show that through interactive reading on this model-constructed memory tree, MemWalker outperforms other long context baselines as well as retrieval and recurrent variants esp. with the longer examples. (5/n)

I’ll be at #NeurIPS2022 this week presenting our work on grounded language agent in web environment. Come chat with me if you’re interested! Also happy to set up 1-1 chat via DM.

Princeton NLP Group

@princeton_nlp

Nov 29, 2022

Replying to @princeton_nlp

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (Yao et al.) w/ @ShunyuYao12, @__howardchen, @jyangballin, @karthik_r_n openreview.net/forum?id=R9Knu… Poster at Hall J #1040 Thu 1 Dec 12 p.m. — 2 p.m. [5/7]

Howard Chen

@__howardchen

Apr 27, 2023

Replying to @arankomatsuzaki

Am I missing something? I don't see any quantitative evaluation.

947