Pinned
Intelligence should be defined by the people closest to the work. Intelligence should be owned by all of us.
Let’s build a many model future!
Baseten
2,407 posts
Baseten
@baseten
Inference is everything.
- The new AgentPerf benchmark by @ArtificialAnlys shows that @NVIDIAAI Blackwell delivers best performance for demanding agentic workloads. With NVIDIA, we're continuously investing in making your coding agents run fast, scale seamlessly, and cost less.
- We're thrilled to be working with the Harvey team to push open models to frontier-level performance for legal AI. Shout out to @gabepereyra for the great article. LAB was key to our joint work post-training open-weight models for legal agents.
- Congrats to the MiniMax team on the open-source launch of M3! There are very few <500bn parameter models that can tackle coding, agentic workloads, and multimodal all with a 1M-token context window but M3 does it all. Dig in here: baseten.co/library/minima…
- Most AI demos built for healthcare don't survive in real clinical or operational environments. The data is messy, the workflows are fragmented, margin for error is near zero. That's why I'm stoked to host a 1.5-day Healthcare x AI Hackathon with @HealthcareAIGuy in NYC: a
- We've heard from customers that they ship model updates >50% more often with rolling deploys than their previous solutions. No downtime, parallel GPU fleet, or off-hours babysitting. Rolling deploys are autoscaling-aware, and you can pause, inspect, or roll back at any step.
- Great to see @baseten’s own @oneill_c and @part_harry_ sitting down with @cursor_ai’s @sjwhitmore to talk about the many things their 128(!) agents are doing (and occasionally arguing about), compaction, and the future.We're trying a new experiment at @cursor_ai - interviewing devs we admire. I chatted with @oneill_c & @part_harry_ from @baseten about how they use coding agents. We discussed their current dev workflows & some predictions for the future. Check it out below!
00:00 - We are excited to announce that we have partnered with @_inception_ai to make Mercury 2 available on Baseten. This makes us the first inference platform to bring Inception’s diffusion LLM to production. Inception’s dLLM architecture fixes the bottlenecks of sequential token
- The longer the context, the more memory your LLM needs. We introduce research techniques to compress that memory 200x on the fly without changing the base model.1/ You can shrink a language model's KV cache by 200×, in a single forward pass, and it still answers correctly. At 256k context that's 36 GiB of cache down to ~360 MiB, with no change to the base model. Here's how we did it 👇
- Baseten reposted
Model selection isn't just a fancy term for "looking at benchmarks". If you're just auto-updating and going off twitter vibes, you're not really adding any value to your business or your customers. To do this well, it means you need to deeply understand your use cases, how muchWorking in the Training team at Baseten, I often see companies agonize over which model to use. So many people worry about how to keep up with benchmarks and new releases But with post-training and specialization, and as we see a rising tide in the intelligence of many
