Armen Aghajanyan (@ArmenAgha) / X

Armen Aghajanyan

866 posts

Armen Aghajanyan

@ArmenAgha

Co-founder & CEO @perceptroninc; ex-RS FAIR/MSFT

Joined June 2013

Pinned
Armen Aghajanyan
@ArmenAgha
May 12
I'm excited to finally release the fruit of the research we've been doing at Perceptron for the last 16 months: Perceptron Mk1. We've been developing multi-modal recipes from the ground up to build models that perform best in the physical world, from video understanding to
Perceptron AI
@perceptroninc
May 12
Today we're releasing Perceptron Mk1: frontier video and embodied reasoning.
00:00
64K
Armen Aghajanyan
@ArmenAgha
Jan 26, 2025
There is an unprecedented level of cope around DeepSeek, and very little signal on X around R1. I recommend unfollowing anyone spreading conspiracy theories around R1/DeepSeek in general. (1/9)
1.1M
Armen Aghajanyan
@ArmenAgha
May 17, 2024
I’m excited to announce our latest paper, introducing a family of early-fusion token-in token-out (gpt4o….), models capable of interleaved text and image understanding and generation.
arxiv.org
Chameleon: Mixed-Modal Early-Fusion Foundation Models
We present Chameleon, a family of early-fusion token-based mixed-modal models capable of understanding and generating images and text in any arbitrary sequence. We outline a stable training...
492K
Armen Aghajanyan
@ArmenAgha
Aug 16, 2024
I've left FAIR/Meta, it's time to build.
265K
Armen Aghajanyan
@ArmenAgha
Dec 2, 2023
Big breakthrough last night. Really excited to share what we've been building with you guys soon.
390K
Armen Aghajanyan
@ArmenAgha
Jan 26, 2025
Replying to @ArmenAgha
First, the DeepSeek team is incredible and has been putting out absolutely fantastic work since their first model, especially around efficiency. MLA allows for ~10x memory efficiency from the KV cache. They got efficient MoE with >8 experts working with near-perfect
97K
Armen Aghajanyan
@ArmenAgha
May 14, 2024
Replying to @ArmenAgha
I firmly believe in ~2 months, there will be enough knowledge in the open-source for folks to start pre-training their own gpt4o-like models. We're working hard to make this happen.
65K
Armen Aghajanyan
@ArmenAgha
May 14, 2024
For the last two years, my team and I have been publicly working on laying the foundations of early-fusion, multi-modal (MM) token-in token-out approaches, from the original CM3 paper to MM-scaling laws to CM3Leon to half a dozen or so more papers all around space, to a couple
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)
@teortaxesTex
May 13, 2024
This is also a good indicator of the R&D gap between OpenAI and Meta. @ArmenAgha has reported the success with pretraining early fusion multi-modal models just a few days ago (though the core breakthrough has been achieved in Dec 2023). OAI are at cheap product stage already.
285K
Armen Aghajanyan
@ArmenAgha
Nov 21, 2024
Say hello to our new company Perceptron AI. Foundation models transformed the digital realm, now it’s time for the physical world. We’re building the first foundational models designed for real-time, multi-modal intelligence across the real world. perceptron.inc
106K
Armen Aghajanyan
@ArmenAgha
Jan 26, 2025
Replying to @ArmenAgha
Lastly, calculating the cost of DeepSeek is a basic exercise given that the architecture is public unless you think they are training more on multiple orders of magnitude. The conspiracy theories are just embarrassing. It's an incredibly bad look on the US to resort to excuses
75K
Armen Aghajanyan
@ArmenAgha
Feb 3, 2025
This is absolutely not true about what happened with Zetta. Do we really want to open up about what happened here?
Yann LeCun
@ylecun
Feb 2, 2025
Replying to @RawSucces and @DAcemogluMIT
You misread. There had been multiple LLM projects within FAIR for years. Some were open sourced as research prototypes (e.g. OPT175B, Galactica, BlenderBot...). In mid-2022, FAIR started a large LLM project called Zetta, which was still going in late 2022 when ChatGPT came out. A
145K
Armen Aghajanyan
@ArmenAgha
Oct 31, 2025
jackass leaking company IP at 2AM
Maciej Kilian
@kilian_maciej
Oct 31, 2025
remember: do not re-normalize MoE router scores post topk if k=1 weighing MoE outputs by the router scores is how the task loss can influence router params. when k=1 score=sum(score) so the update makes the scores=1 meaning it doesn't get gradients since its a constant!
250K
Armen Aghajanyan
@ArmenAgha
Jun 18, 2024
God will not forgive me for how we tortured this model to get it out. Things I recommend doing: - Further post-training with the amazing alignment datasets the OS community has created. - If you're using Chameleon for perception, fine-tune patches in (Fuyu style). You'll get
Armen Aghajanyan
@ArmenAgha
Jun 18, 2024
A restricted, safety aligned (no-image-out) version of Chameleon (7B/34B) is now open-weight! github.com/facebookresear… The team strongly believes in open-source. We had to do a lot of work to get this out to the public safely. Congrats to the Chameleon team!
155K
Armen Aghajanyan
@ArmenAgha
Jan 26, 2025
Replying to @ArmenAgha
So have the frontier model teams in the US misused capital scaling compute rather than focusing on efficiency? Partially yes, but it also depends on how you value opportunity cost, simplicity, first-to-market, etc. The calculus here is not simple. Compute, and researchers will
93K