Log inSign up
Ishaan
1,828 posts
user avatar
Ishaan
@ishaan_jaff
Co-Founder & CTO LiteLLM (YC W23) - Python SDK & LLM Gateway to Call 100+ LLMs in 1 format, set Budgets github.com/BerriAI/litell…
san francisco
Joined November 2017
2,320
Following
3,163
Followers

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

Terms·Privacy·Cookies·Accessibility·Ads Info·© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up
  • Pinned
    user avatar
    Ishaan
    @ishaan_jaff
    Nov 21, 2023
    🚅 LiteLLM v1.2.0 🚅 Use Azure, OpenAI, Cohere, Anyscale, Anthropic, Hugging Face 100+ LLMs as a drop in replacement for OpenAI ⚡️ Use LiteLLM to 20x your rate limits using load balancing + queuing github.com/BerriAI/litell… github.com/BerriAI/litellm
    GitHub - BerriAI/litellm: Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or...
    From github.com
    21K
  • user avatar
    Ishaan
    @ishaan_jaff
    Jun 21, 2023
    We got 99% reliability with this GPT error handling strategy. So we're open-sourcing it for everybody else. 💪github.com/BerriAI/reliab…
    61K
  • user avatar
    Ishaan
    @ishaan_jaff
    Feb 2, 2023
    Tired of long dev cycles for deploying LLM Agents from Google Colab to production? 🎉Introducing Berri AI 🍇- a Python package that makes it easy for developers to deploy @langchain LLM Agents with just one click! 💥 github.com/ClerkieAI/berr…
    00:00
    40K
  • user avatar
    Ishaan
    @ishaan_jaff
    Nov 23, 2023
    💥Thrilled to launch LiteLLM Load Balancer - 20x your throughput on vLLM, @OpenAI , @cohere, @anyscalecompute, @AnthropicAI, Open source LLMs Try it here: docs.litellm.ai/docs/routing 👉 Want to use the Proxy server for Load Balancing? Try it here: docs.litellm.ai/docs/simple_pr… 🔑 Support
    21K
  • user avatar
    Ishaan
    @ishaan_jaff
    Nov 4, 2023
    🚅 LiteLLM now powers ParlAI by Meta, use LiteLLM to easily evaluate 100+ LLMS github.com/BerriAI/litellm 🦬 Improvements to VertexAI Streaming h/t to Vijay Ramesh docs.litellm.ai/docs/providers… github.com/facebookresear…
    GitHub - BerriAI/litellm: Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or...
    From github.com
    11K
  • user avatar
    Ishaan
    @ishaan_jaff
    Jan 20, 2023
    Introducing ChatLangChain (@langchain): answer questions about the LangChain codebase and generate code 1️⃣ Simplifies using new libraries/frameworks for devs 2️⃣ Assists in answering support questions for Library maintainers. Sign up here: clerkie.co
    00:00
    17K
  • user avatar
    Ishaan
    @ishaan_jaff
    Dec 3, 2023
    🚅LiteLLM Proxy Server v1.10.2 - Call @openai, @cohere , @anyscalecompute , @huggingface LLMs in one interface, Load Balance Throughput, Track Costs per user docs.litellm.ai/docs/simple_pr… 🐳 New @Docker Image for LiteLLM Proxy: docs.litellm.ai/docs/simple_pr… h/t @TheAyushSomani for the
    11K
  • user avatar
    Ishaan
    @ishaan_jaff
    Oct 28, 2023
    🚨 LiteLLM Server: 1 server, call 100+ LLMs, 96.6% reliability github.com/BerriAI/litell… 🚨 LiteLLM Server auto logs exceptions to @getsentry 💥 Define Fallback LLMs on exceptions for LiteLLM Server (ex - if OpenAI fails use Llama2) ✂️ Reduced bloat on LiteLLM - removed deprecated
    9.6K
  • user avatar
    Ishaan
    @ishaan_jaff
    Nov 29, 2023
    💥LiteLLM Proxy - Now handles 500+ requests/second, Load Balance Azure + OpenAI deployments, Track spend per user 💥 Try it here: docs.litellm.ai/docs/simple_pr… 🔑 Support for `AZURE_OPENAI_API_KEY` on Azure docs.litellm.ai/docs/providers… h/t @solyarisoftware ⚡️ LiteLLM Router can now
    7.5K
  • user avatar
    Ishaan
    @ishaan_jaff
    Oct 15, 2023
    LiteLLM v0.8.4 📞 STOP Spending time converting your functions to JSONs for LLM Function calling. LiteLLM Converts your functions to JSON inputs for OpenAI ⭐️ Docstrings for LiteLLM util functions docs.litellm.ai/docs/completio… h/t to Hugues-Yanis Amanieu docs.litellm.ai/docs/completio…
    Completion Token Usage & Cost | liteLLM
    From docs.litellm.ai
    17K
  • user avatar
    Ishaan
    @ishaan_jaff
    Jan 13, 2023
    Introducing Clerkie-LangChain, our bot that answers questions about LangChain codebase 1️⃣ Simplifies using new libraries/frameworks for devs 2️⃣ Assists in answering support questions for Library maintainers React ❤️ / comment 👋 to get access to our bot or for your library
    7K
  • user avatar
    Ishaan
    @ishaan_jaff
    Oct 31, 2023
    💥 LiteLLM now powers github.com/homanp/nagato by @pelaseyed and github.com/safevideo/auto… by @safevideo_ai 🔎 Updated Tutorial on how to use fine tuned gpt-3.5-turbo with litellm h/t to @dpaleka: docs.litellm.ai/docs/tutorials…
    GitHub - homanp/nagato: 🌸 The open framework for question answering fine-tuning LLMs on private...
    From github.com
    4.3K
  • user avatar
    Ishaan
    @ishaan_jaff
    Jun 3, 2023
    we're open-sourcing our automated LLM app evaluation framework that helped us 10x the speed of our dev loops: github.com/ClerkieAI/bett… Instantly get an accuracy score on your LLM generated answers and evaluations on the quality of your response
    7.9K
  • user avatar
    Ishaan
    @ishaan_jaff
    Aug 13, 2023
    liteLLM change log v0.1.386: github.com/BerriAI/litell… 💥🦙 Ollama x liteLLM integration: now interact with llama2 using chatGPT Input/Outputs Here’s our notebook for trying it with liteLLM: github.com/BerriAI/litell…
    4.7K