Are you a PhD student or an independent researcher who is working on creating a dataset?
@scale_AI will sponsor promising dataset creation projects that can widely benefit a community. Please DM me for details.
So far (2 days gone), this is my favorite #NeurIPS18 paper. I just love the paper that tells me why it works. It also showed that the popular previous belief "internal covariate shift" has little to do with batch norm's success. Ah! 😍
3-min video: youtu.be/ZOabsYbmBRM
Training Data. The neglected in academia, but an important topic in industry.
I learned the hard way- errors in training data can easily wipe out accuracy gain from a model.
I'm writing codes to monitor data fed to ML.
Things to check (can you add more?):
1. skewness between
I still remember being surprised 8 yrs ago when I first reduced Word2vec/Glove embeddings to a lower dimensional space. To my surprise, antonyms were placed very close to the words themselves and models sometimes failed to understand negation. At the time, I couldn't find a way
You might have heard that word2vec, test of time award winner at #NeurIPS2023, was rejected from #ICLR2013 back in the day. Interestingly, one reviewer felt so strongly that they recommended "Strong Reject" four times.
My team spoke very highly about this blog (and they're also wondering if self-supervised learning could eliminate the need for labeling entirely) so I gave it a read. It was a very well-written, thorough overview of SSL. What stands out the most was it was written by Dr. LeCun,
How can we build machines with human-level intelligence? There’s a limit to how far the field of AI can go with supervised learning alone. @ylecun and @imisra_ discuss why self-supervised learning is a promising way to make significant progress in AI. ai.facebook.com/blog/self-supe…
What @bryan_johnson does is pretty common for South Korean women..
As a woman who was born and raised in Korea, I've been doing basically the same, like religiously avoiding the sun, regularly getting facials, calorie restriction, etc. It's the norm there.
My new boss 🕶️🚬 got me a new desktop from @LambdaAPI.
4x @Nvidia RTX 8000 (48G VRAM in a single GPU. Total 192 GB)
I was able to wrap them in a beautiful box. I certainly cannot wrap 8 Titans in a box.
Which model should I try first on my new machine? ☺️
Before I forget, I'd like to summarize some interesting papers that I found at #CVPR2022.
Dual-key multimodal backdoors for visual question answering
arxiv.org/abs/2112.07668
1. This paper proposes an interesting Trojan attack method.
To start, what exactly is a Trojan attack?
One thing that stood out to me at #NeurIPS2023 was there were a lot more application papers than before.
This was not the case with my last neurips (2018).
I'd like to encourage this wave by highlighting one such paper.
LayoutGPT: Compositional Visual Planning and Generation
How can we tame GPTs?
Tame as, how do we make LLM give us more of what we asked for, instead of random answers?
Also, how do we discover what GPTs are capable of?
Can we use LLM as, e.g. copilot or translator?
We'll discuss I-GPT, which all OpenAI GPT production APIs use.
How can you ensure your AI language models are aligning with human intent? 🤖
Join Long, OpenAI Research Scientist, and Aerin, Scale AI Engineering Manager, for a technical deepdive followed by discussion around Long's work with InstructGPT, and live audience Q&A.
"Engineers over 35 aren't sharp/fast anymore."
-> Not true.
Attended @Replit hackathon last week and teamed with a former sr dir of eng at xxx, 35+. He was one of the fastest and most effective I've ever worked with.
I don't think coding skill is something that fades with age.
Introducing a new paper: Natural Adversarial Object
published @AndrewYNg's Data-Centric AI workshop in #NeurIPS2021.
What is a Natural Adversarial Object?🤷
You might have already heard of adversarial samples - the inputs to ML models that we intentionally created to confuse
🦈Natural Adversarial Objects: a new dataset to evaluate the robustness of object detection models. NAO contains 7,934 images and 9,943 objects that are unmodified and representative of real-world scenarios, but cause SoTA detection models to misclassify.
paperswithcode.com/dataset/nao
I wrote about the Beta distribution during the holidays.
1. Why does the PDF of Beta dist look the way it does?
2. Why do we want to use Beta dist?
3. Intuition for its shapes
Feedback welcome! I'm still looking for an explanation of U-shaped Beta.
medium.com/@aerinykim/bet…
When linear bmm made LSTM disappear, I thought some other overly elaborated architectures will fade away too.
But I didn’t expect CNN belongs to that category..
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
When pre-trained and transferred to CV tasks, Vision Transformer attains excellent results compared to SOTA CNNs while requiring much fewer computational resources to train.
openreview.net/forum?id=YicbF…
