Published November 20, 2025
| Version v1
Dataset
Open
RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs
Authors/Creators
-
1.
University of Trento
-
2.
University of Amsterdam
-
3.
Ludwig-Maximilians-Universität München
- 4. IT University of Copenhagen
Description
RAcQUEt
Dataset accompanying the EMNLP 2025 paper “RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs”.
It contains 740 image–question pairs specifically designed to test how Vision–Language Models handle referential ambiguity. The resource includes two subsets:
- RAcQUEt-GENERAL – ambiguous questions about real-world MSCOCO images, targeting cases where multiple valid referents exist.
- RAcQUEt-BIAS – controlled, synthetically generated image–question pairs focusing on how unresolved ambiguity can lead to socially biased or stereotypical outputs (gender, ethnicity, disability).
Files
README.md
Files
(117.2 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:bf0c344078e2b0efcffd06d886a5fe90
|
39.2 kB | Download |
|
md5:7ab235a700d9c0adb0a75f5adf2f9aa4
|
75.7 kB | Download |
|
md5:efefa7c22e5b16f2992ad18ffe3a03d3
|
2.3 kB | Preview Download |