Log inSign up
Confident AI
186 posts
user avatar
Confident AI
@confident_ai
AI Observability & Evaluation Platform. Creator of @deepeval, the open-source LLM evaluation framework: github.com/confident-ai/d…
San Francisco
confident-ai.com
Joined October 2024
31
Following
436
Followers

New to X?

Sign up now to get your own personalized timeline!

Create account

By signing up, you agree to the Terms of Service and Privacy Policy, including Cookie Use.

TermsΒ·PrivacyΒ·CookiesΒ·AccessibilityΒ·Ads InfoΒ·Β© 2026 X Corp.
Don't miss what's happening
People on X are the first to know.
Log inSign up
  • Pinned
    user avatar
    Confident AI
    @confident_ai
    Oct 17, 2024
    πŸ’₯ After reaching nearly 200k downloads a month🀯 and 3.3K⭐GitHub stars on DeepEval (github.com/confident-ai/d……), here are the top takeaways and learnings we have to share for evaluating LLM applications... #deepeval #llmevaluation #llms #opensource A thread πŸ§΅πŸ‘‡(1/10) :
    GitHub - confident-ai/deepeval: The LLM Evaluation Framework
    From github.com
    10K
  • user avatar
    Confident AI
    @confident_ai
    Oct 19, 2024
    Who's building RAG based LLM applications πŸ™Œ? If you are, this thread is for you 🫡 Building a retrieval augmented generation application (RAG) application isn't rocket science, but it's definitely difficult to get it performing as expected. A thread on RAG evaluation🧡(1/7):
    154
  • user avatar
    Confident AI
    @confident_ai
    Jul 8, 2025
    πŸš€ Shipped Integration You can now trace your @crewAIInc apps on the @confident_ai platform. By just adding 2 lines of code in your app, you can get the entire execution steps of your agent in the form of a single trace. Leverage your LLM application performance using
    214
  • user avatar
    Confident AI
    @confident_ai
    Oct 29, 2024
    Replying to @svpino
    We've also built a platform dedicated for automated LLM application evaluation that became very popular recently, check it out at app.confident-ai.com
    00:00
    132
  • user avatar
    Confident AI
    @confident_ai
    Oct 24, 2024
    Replying to @svpino
    πŸ”₯ You can also use DeepEval. Open-source, more feature rich, great documentation and also has a platform for auto-LLM safety (πŸ₯³) detection, LLM testing, and monitoring in production: github.com/confident-ai/d… Here's a quick demo of the platform (app.confident-ai.com):
    00:00
    125
  • user avatar
    Confident AI
    @confident_ai
    Oct 18, 2024
    🚨 ⚠️ Did you know that anyone with the ability to type can exploit your LLM application to hack your systems 😰? LLMs nowadays are granted free access your data, and a single failure in your safety defense can be nightmare to deal with πŸš‘πŸŒ‹ A thread on LLM security
    123
  • user avatar
    Confident AI
    @confident_ai
    Apr 3, 2025
    Somehow deepteam got picked up github.com/confident-ai/d…
    user avatar
    Transformer Lab
    @transformerlab
    Mar 26, 2025
    🚨 AI Safety Evolved: We just added the Red Teaming Plugin, powered by @Confident_AI's DeepTeam framework. Comprehensive vulnerability testing across 10+ critical domainsβ€”from bias detection and misinformation to PII protection and unauthorized access checks. πŸ”
    199
  • user avatar
    Confident AI
    @confident_ai
    Oct 15, 2024
    πŸ’₯ ANNOUNCEMENT: DeepEval v1.3.8 is released! πŸ’₯ DeepEval is an open source LLM evaluation framework for: πŸ”₯ Implementing custom LLM metrics πŸͺ² Safety testing LLM apps πŸ’― Scoring LLM responses And so more more. Checkout repo below.
    GitHub - confident-ai/deepeval: The LLM Evaluation Framework
    From github.com
    66
  • user avatar
    Confident AI
    @confident_ai
    Oct 26, 2024
    Community write-ups like these make DeepEval alive.
    Building Confidence in LLM Evaluation: My Experience Testing DeepEval on an Open Dataset
    From serj-smor.medium.com
    111
  • user avatar
    Confident AI
    @confident_ai
    Jul 27, 2025
    Thanks for the feedback and feature @ycombinator!
    user avatar
    Y Combinator
    @ycombinator
    Jul 25, 2025
    The best product websites do more than pitch. They explain, guide, and reflect the product itself, and often serve as a company’s first impression. For Design Review, YC’s @aaron_epstein is joined by Karri Saarinen (@karrisaarinen), co-founder & CEO of Linear, to break down how
    00:00
    157
  • user avatar
    Confident AI
    @confident_ai
    Jul 15, 2025
    πŸš€ Shipped Integration You can now trace your @llama_index application on the @confident_ai platform, using evals from @deepeval. We are planning to introduce easy-to-execute evaluations for your agents. Stay tuned!
    138
  • user avatar
    Confident AI
    @confident_ai
    Oct 17, 2024
    Just setup a X account for Confident AI πŸ₯³
    62
  • user avatar
    Confident AI
    @confident_ai
    Jul 15, 2025
    πŸš€ Big update to our @langchain integration with @deepeval ! βœ… Human-AI message roles βœ… Tool call output now visible in LangChain βœ… Tool attachments in inputs βœ… Span naming, fallback + metadata βœ… Generation models + token tracking (cost coming soon!)
    00:00
    122
  • user avatar
    Confident AI
    @confident_ai
    Oct 24, 2024
    Replying to @EcZachly
    πŸ”₯ How about adding a section on LLM evaluation? We have DeepEval, open-source LLM evaluation framework for LLM functional and safety testing: github.com/confident-ai/d… (PS. Also comes with a platform, app.confident-ai.com, quick demo down thereπŸ‘‡)
    00:00
    295