Owen Colegrove (@ocolegro) / X

Owen Colegrove

2,107 posts

Owen Colegrove

@ocolegro

Physicist | Quant | Founder

San Francisco

Joined January 2021

Pinned
Owen Colegrove
@ocolegro
Jul 19, 2024
Today SciPhi is open-sourcing Triplex, a SOTA LLM for knowledge graph construction. Triplex is so small that it can be used with SciPhi's R2R to build knowledge graphs directly from your laptop. Triplex outperforms few-shot prompted gpt-4o at 1/60th the inference cost.
00:00
43K
Owen Colegrove
@ocolegro
Dec 21, 2023
Today, I'm sharing AgentSearch, an open-core effort to make humanity's knowledge accessible for LLM agents. To start, I have embedded all of Wikipedia, Arxiv, filtered common crawl, and more. The result is over 1 billion embedding vectors, try it at search.sciphi.ai.
239K
Owen Colegrove
@ocolegro
Jan 9, 2024
This result is so fascinating: WikiChat: Stopping LLM Hallucination - Achieves 97.9% factual accuracy in conversations with human users about recent topics, 55.0% better than GPT-4! Anyone interested in replicating + extending w/ AgentSearch?
126K
Owen Colegrove
@ocolegro
Sep 29, 2023
We now have a pipeline that goes: MIT OCW -> Syllabus -> Table of Contents -> Textbook. The final step done w/ RAG over all of wikipedia. I see why Anthropic thinks they can auto research within in a few years. First book is here: [github.com/emrgnt-cmplxty…]. More coming soon.
109K
Owen Colegrove
@ocolegro
Jan 22, 2022
1.) As a builder on #ALGO, the recent price action does not worry me. In fact it excites me 👇
Owen Colegrove
@ocolegro
Oct 12, 2023
Our textbook generation is really cooking now. Now putting out ~500m tokens/day of RAG-grounded synthetic textbooks with a fine-tuned Mistral 7b. That's a good fraction of a public library. Thx in part to the help from @winglian
100K
Owen Colegrove
@ocolegro
Sep 27, 2023
If you are GPU-Poor, then you should probably be paying attention to small LLMs & synthetic data. Below is a list of the most interesting papers relating to this that I have found from the last year (ordered by release date), what am I missing? Self-Instruct
39K
Owen Colegrove
@ocolegro
Sep 28, 2023
With LLMs we can create a fully open-source Library of Alexandria. As a first attempt, I have generated 650,000 unique textbook samples from a diverse span of courses, kindergarten through graduate school. They are all available here: [huggingface.co/datasets/emrgn…]
60K
Owen Colegrove
@ocolegro
Oct 31, 2023
We encountered issues with accuracy in synthetic data generation. Fortunately, a great approach called self-rag was introduced by @AkariAsai et al. Introducing SciPhi-Self-RAG-Mistral-7B-32k I fine-tuned the SciPhi model with this and more. The result is powerful + efficient.
69K
Owen Colegrove
@ocolegro
Nov 16, 2023
After the release of Phi-2, I felt it was a good time write-up my learnings from early efforts at replicating Phi-1.5. The number one insight is: Generating 20 billion unique, high-quality synthetic tokens is challenging. I explain why in depth here 👇
72K
Owen Colegrove
@ocolegro
Aug 27, 2023
WizardCoder-34B appeared to be a big step forward in open-source AI models, but I find it falls far short of GPT-4! After reproducing their HumanEval and assessing on ~400 OOS LeetCode problem, I see that it is more on par w/ Claude-2 or GPT-3.5. See results in img below 👇
57K
Owen Colegrove
@ocolegro
Feb 6, 2024
Recently, engaging with developers during my time at YCW24 revealed a valuable insight: the demand for the advanced Retrieval Augmented Generation (RAG) solutions I envisioned is overshadowed by the need for simpler, more intuitive tooling. I'm grateful that YC's culture pushes
30K
Owen Colegrove
@ocolegro
Mar 5, 2024
I've been programming w/ Claude-3 Opus today, it feels like a step up from gpt-4. It is often providing better code and w/out abbreviation so that I can immediately copy + paste the output.
15K
Owen Colegrove
@ocolegro
Nov 29, 2023
DeepSeek team doing some truly outstanding work. Their latest model is close to GPT-4 across a variety of high quality evals, and looks by far the best open option. It seems like there might be some advantage to training on both Chinese and English. github.com/deepseek-ai/de…
64K