alex peysakhovich (@alex

alex peysakhovich

2,525 posts

alex peysakhovich

@alex_peys

partner @shv, ex-facebook ai research. currently really into ai for biology. stealthily making something

Joined February 2013

alex peysakhovich
@alex_peys
Aug 18, 2023
really love when authors do things like explain equations in-line with colors etc... just makes papers so much easier to read.
455K
alex peysakhovich
@alex_peys
Aug 5, 2023
once did a screening interview with a “famous ml hedgefund”. got a leetcode-style problem to find connected components in a graph. they wanted bfs or dfs or whatever. i took the svd of the laplacian and counted the number of 0s. they didn’t pass me because “you can’t use numpy”
482K
alex peysakhovich
@alex_peys
Sep 11, 2024
Replying to @wwwojtekk
you mean "pole"-ing?
54K
alex peysakhovich
@alex_peys
Jun 23, 2024
claude3.5 is really good, but also still a transformer
168K
alex peysakhovich
@alex_peys
Jun 28, 2024
just released a paper with will berman on multimodal inputs for image generation main idea: describing things just in text is often hard. can you train a model that uses interleaved text/image prompts for image generation? the answer is yes. 🧵
53K
alex peysakhovich
@alex_peys
Mar 31, 2023
standard ml: oh no my model is memorizing the training set, better add some regularization to make that not happen llm ml: ugh it’s hallucinating, why can’t it just memorize some of the training set
85K
alex peysakhovich
@alex_peys
Sep 10, 2023
living the dream of the gpu upper middle class
37K
alex peysakhovich
@alex_peys
Jan 3, 2024
facebook didn't keep it secret, here is the paper that explains the early comment ranking system for exactly this issue arxiv.org/pdf/1702.06677…
Paul Novosad
@paulnovosad
Jan 3, 2024
Engineers have figured out to cut back on the toxicity of the internet — but firms who are good at it keep it secret as a competitive advantage. A fascinating question of private vs. public interests.
108K
alex peysakhovich
@alex_peys
Aug 25, 2024
Replying to @KennethCassel
that’s not a prisoner’s dilemma, the point of a pd is that the pie is maximized by cooperation, here the pie is maximized by the strategy your kids played
20K
alex peysakhovich
@alex_peys
Aug 5, 2023
Replying to @Apoorva__Lal
you can also estimate null space dimension by “uniformly” randomly sampling vectors, hitting them with the matrix, and seeing what % are 0. no numpy required
24K
alex peysakhovich
@alex_peys
Nov 8, 2023
ran gpt4 128k context on the "1 useful document + K distractors" from our "attention sorting" paper, seems like the very long (more than 32k) doesn't work that well. 32k is still extremely impressive though! also claude2 clearly has some nice trick behind the scenes
47K
alex peysakhovich
@alex_peys
Feb 4, 2024
the foundation of machine learning is that inside every big matrix lives a much smaller matrix and you should just use that
rohit
@krishnanrohit
Feb 3, 2024
Text embeddings are the unsung heroes of making ai work. Can't believe we found a way to make language into numbers, the matrix stuff backprop etc afterwards feels almost straightforward once it's done. All tricks are downstream of this. With numbers you can add them (merge),
20K
alex peysakhovich
@alex_peys
Jul 6, 2023
i worked at fb for 9 years, this is the first time i have ever seen people excited about an fb original product at launch time
24K
alex peysakhovich
@alex_peys
Oct 13, 2023
Replying to @infoxiao
this is literally the recipe for llms
207K