NeuBAROCO Reasoning Datasets

Datasets and scripts for the following paper(s). See the respective links for more details.

• "Evaluation of Deontic Conditional Reasoning in Large Language Models: The Case of Wason's Selection Task" (accepted to EACL2026 Main)
  • Dataset
  • Paper (the link will be updated upon publication)
• "Normative Reasoning in Large Language Models: A Comparative Benchmark from Logical and Modal Perspectives" (BlackboxNLP 2025)
  • Datasets and scripts
  • Paper
• "Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset" (ACL2024 Findings)
  • Datasets and scripts
  • Paper
• "Evaluating Large Language Models with NeuBAROCO: Syllogistic Reasoning Ability and Human-like Biases" (NALOMA 2023)
  • Datasets and scripts
  • Paper

License

The datasets are licensed under Creative Commons Attribution 4.0 International.