Skip to content

About

Link-checked catalogue of Indonesian datasets for language and multimodal models — Bahasa Indonesia plus 10 local languages.

Topics

awesome-listdataset-cataloguedatasetsindonesianindonesian-nlpllmlow-resource-languagesmachine-learningnlp

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository
BranchesTags

Repository files navigation

Indonesian LLM Datasets

Link check

A catalogue of Indonesian-language datasets for training and evaluating language and multimodal models: Bahasa Indonesia plus the low-resource local languages (Acehnese, Balinese, Banjarese, Buginese, Javanese, Madurese, Minangkabau, Ngaju, Sundanese, Toba Batak).

Maintained derivative of irfanfadhullah/awesome-indonesian-llm-dataset. Provenance and licensing are documented in ATTRIBUTION.md.
Categories 14 task-oriented files under data/
Link targets 261 unique URLs
Dead links 0, as of the 2026-10-28 audit
Languages Indonesian and 10 local languages
Source of truth Markdown only; nothing is generated
For agents SKILL.md — how to search, verify, and cite entries
Web version https://h0c0de.github.io/indonesian-llm-datasets/

Link health

Every URL here is checked automatically, and a dead link fails the build rather than sitting there for years:

python3 scripts/check_links.py                        # exits 1 when a link is dead
python3 scripts/check_links.py --markdown-out report.md

.github/workflows/link-check.yml runs weekly (Mondays 06:00 UTC), writes the report to the job summary, pushes it to Telegram when the secrets exist, and opens or updates an issue labelled link-rot.

Classification is deliberately conservative: dead means a 4xx/5xx response or a host that no longer resolves. Bot walls (403/429) and hosts unreachable from the runner are reported separately and do not fail the check.

Curation policy

An entry is never deleted because its download link rotted. These corpora are named artefacts — Kompas/Tempo news, the Kaskus crawl, Indonesian CC-News — that outlive their hosting. Dead links are handled in three ways:

  1. the entry keeps a live alternative if one exists;
  2. otherwise the link points at a verified Wayback snapshot;
  3. otherwise it becomes plain text marked dead link, Oct 2026.

Categories

Task File
Natural language understanding 01
Token classification 02
Knowledge graphs 03
Web crawl and text corpora 04
Local languages 05
Multimodal and vision-language 06
Paraphrase and text similarity 07
Question answering 08
Speech and audio 09
Text summarization 10
Machine translation 11
Dictionary and vocabulary 12
Pre-trained models 13
Tools and libraries 14

Related

  • NusaX — parallel sentiment and MT corpus for Indonesian and 10 local languages (EACL 2023).
  • SEACrowd — the Southeast Asian data hub several entries now point at.
  • IndoNLU — benchmark suite for Indonesian NLU.

Sections below

Quick start · Contributing · Key papers · License · Citations

📊 Dataset Categories

📄 Go to overview file

🧠 Natural Language Understanding

📄 Go to detailed file

IndoNLI: Natural Language Inference Dataset

  • Description: Human-annotated NLI data for Indonesian with train/val/test splits
  • Size: Thousands of premise-hypothesis pairs
  • Tasks: Natural language inference, textual entailment
  • Links: GitHub Repository | HuggingFace
  • Paper: EMNLP 2021

Key Features:

  • Expert and lay annotator splits
  • Diagnostic subset with linguistic phenomena
  • Translated MNLI dataset included

Indonesian NLP Resources

Comprehensive collection of datasets for various NLP tasks:

Hate Speech & Abusive Language Detection

  • ID Multi-label Hate Speech
    • Description: 13,169 Indonesian tweets labeled for multi-label hate speech and abusive language
    • Labels: Multiple categories of hate speech and abusive content
    • Links: GitHub Repository
    • Paper: ACL 2019
  • ID Abusive Language Detection
  • ID Hate Speech Detection
    • Description: 713 Indonesian tweets labeled hate speech/non-hate speech
    • Links: GitHub Repository
    • Paper: IEEE

Translated Dataset (Our Works)

QQPR-Triplets-ID

🔡 Token Classification

📄 Go to detailed file

Named Entity Recognition (NER)

  • Product NER
  • NER-GRIT
    • Description: Indonesian NER corpus for research and development
    • Links: HuggingFace
  • Singgalang NER Dataset
    • Description: Automatically generated NER corpus
    • Size: 48K sentences
    • Links: GitHub Repository
  • nlp-experiments NER Dataset
    • Description: NER dataset for research purposes
    • Size: 2,125 sentences
    • Links: GitHub Repository

POS Tagging & Parsing

📚 Knowledge Graphs

📄 Go to detailed file

IndoWiki: Indonesian Knowledge Graph

  • Description: Knowledge graph from WikiData aligned with Indonesian Wikipedia
  • Size: 533K+ entities, 939 relations, 2.6M+ triplets
  • Format: Both transductive and inductive splits available
  • Links: GitHub Repository | Google Drive

Data Structure:

  • Transductive setting: Shared entities/relations across splits
  • Inductive setting: Disjoint entities between train/test
  • Complete pipeline for data processing included

🌐 Web Crawl & Text Corpora

📄 Go to detailed file

Large-Scale Text Collections

OSCAR Corpus

CC-100

  • Size: ~4.8B sentences, 6B sentence piece tokens
  • Source: FAIR's CommonCrawl extraction
  • Links: StatMT CC-100

CulturaX

Indonesian News Corpus

Leipzig Corpora Collection

IndoNLU Benchmark

Other Large Corpora

  • WiLI-2018: Zenodo Link - 235000 paragraphs of 235 languages
  • WikiANN: DropBox Link - Supports 176 languages
  • Tatoeba: HuggingFace Link - 32G translation units in 2,539 bitexts
  • QCRI Educational Domain Corpus: Link - 225 languages, 271,558 files

Specialized Crawl Datasets

Kaskus WebText

  • Size: 7K URLs from Indonesian forum with quality filtering
  • Links: Download (dead link, Oct 2026) | Source List (dead link, Oct 2026)

Wikipedia Links

  • Size: 58K URLs from Indonesian Wikipedia external links
  • Links: Download (dead link, Oct 2026)

Indonesian CommonCrawl News

  • Description: Processed CommonCrawl News filtered for Indonesian
  • Links: HuggingFace

Twitter Collections

  • Twitter Puisi: 80K+ poems from @PelangiPuisi
  • Twitter Crawl: Download (dead link, Oct 2026)

"Clean" mC4

Javanese Text

  • Sources: CC100, data.statmt.org, Wikipedia
  • Links: Download (dead link, Oct 2026)

Historical News Corpora

  • Kompas Online Collection (2001–2002)
    • Links: Download (dead link, Oct 2026)
  • Tempo Online Collection (2000–2002)
    • Links: Download (dead link, Oct 2026)

HuggingFace Datasets Collection

Additional datasets available on HuggingFace:

🗣️ Local Languages

📄 Go to detailed file

NusaX-MT: Multilingual Translation Dataset

  • Languages: 12 languages including Indonesian + 10 local languages
  • Local Languages: Acehnese, Balinese, Banjarese, Buginese, Madurese, Minangkabau, Javanese, Ngaju, Sundanese, Toba Batak
  • Size: Parallel corpus across all language pairs
  • Links: GitHub Repository | Paper

Regional Language Resources

MinangNLP

  • Description: Minangkabau language resources
  • Content:
    • Bilingual dictionary: 11,905 Minangkabau-Indonesian pairs
    • Sentiment analysis: 5,000 parallel texts
    • Machine translation: 16,371 sentence pairs
  • Paper: PACLIC 2020

MadureseSet

Javanese Datasets

NusaWrites Benchmark

  • NusaTranslation: 72,444 texts across 11 local languages
  • NusaParagraph: 57,409 paragraphs across 10 local languages
  • Tasks: Sentiment analysis, emotion classification, machine translation
  • Links: GitHub Repository | Paper

HuggingFace Access for NusaWrites:

# NusaTranslation datasets
datasets.load_dataset('indonlp/nusatranslation_emot')
datasets.load_dataset('indonlp/nusatranslation_senti') 
datasets.load_dataset('indonlp/nusatranslation_mt')

# NusaParagraph datasets
datasets.load_dataset('indonlp/nusaparagraph_emot')
datasets.load_dataset('indonlp/nusaparagraph_rhetoric')
datasets.load_dataset('indonlp/nusaparagraph_topic')

Sundanese Resources

Indonesian Speech with Accents

  • Coverage: 5 ethnic groups (Batak, Malay, Javanese, Sundanese, Papuan)
  • Links: Kaggle Dataset

Story Cloze in Local Languages

🖼️ Multimodal & Vision-Language

📄 Go to detailed file

Vision-Language Datasets

Conceptual Captions (Indonesian)

  • CC3M: 3M image-caption pairs translated to Indonesian
  • CC12M: 12M image-caption pairs for vision-language pre-training

YFCC100M OpenAI Subset

  • Size: 14.8M images with natural language descriptions
  • Links: Original

MSVD-Indonesian

KTP VLM Instruct Dataset

Educational & Professional Assessment

IndoMMLU

  • Description: Multi-task language understanding benchmark
  • Size: 14,906 questions across 63 tasks
  • Levels: Primary school to university entrance exams
  • Coverage: 46% Indonesian language + 9 local languages/cultures
  • Links: HuggingFace

IndoCareer

  • Description: Professional certification exam questions
  • Size: 8,834 multiple-choice questions
  • Sectors: Healthcare, finance, design, tourism, education, law
  • Links: HuggingFace | Paper

IndoCulture

  • Description: Cultural knowledge questions about Indonesia
  • Size: 2,430 questions with multiple choice answers
  • Links: HuggingFace

IndoCloze

  • Description: Commonsense story understanding through cloze evaluation
  • Size: 2,325 Indonesian stories (4-sentence premise + endings)
  • Split: 1,000 train / 200 dev / 1,135 test
  • Award: Best Paper Award at CSRR (ACL 2022)

🔄 Paraphrase & Text Similarity

📄 Go to detailed file

Paraphrase Collections

PAWS (Indonesian)

  • Size: 100K human-labeled pairs
  • Source: Translated from Google's PAWS dataset
  • Focus: Word order and structure importance
  • Links: GitHub Repository | Original

Quora Paraphrasing ID

  • Description: Indonesian adaptation of Quora paraphrase pairs
  • Links: GitHub Repository

Indonesian PAWS

  • Description: Translated PAWS dataset with 100K human paraphrase pairs
  • Links: GitHub Repository

ParaNMT-50M (Indonesian)

  • Size: Subset of 50M+ English paraphrase pairs
  • Method: Neural machine translation approach
  • Links: Original

ParaBank

  • Description: Diverse paraphrastic bitexts via sampling and clustering
  • Links: Original

Indonesian MultiNLI & SNLI

SBERT Paraphrase Data

  • Description: Various paraphrase datasets compiled by SBERT, translated to Indonesian
  • Links: Download (dead link, Oct 2026)

❓ Question Answering

📄 Go to detailed file

QA Datasets

SQuAD (Indonesian)

  • Description: Reading comprehension dataset
  • Source: Translated Stanford Question Answering Dataset
  • Links: Original

Mathematics Dataset (Indonesian)

  • Size: 1,000 questions per module
  • Content: School-level mathematical reasoning problems
  • Tasks: Algebraic reasoning, problem solving
  • Links: GitHub Repository | Original

FacQA

  • Description: Factoid question answering from news articles
  • Source: Find answers from provided short passages
  • Links: GitHub | HuggingFace

TyDi QA

  • Description: Question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs
  • Source: GitHub
  • Links: HuggingFace

mLAMA

  • Description: Multilingual version of Language Model Analysis dataset
  • Links: HuggingFace

QED (Question-Answer pairs multilingual)

  • Description: Educational domain corpus with Indonesian data
  • Links: Opus

🎙️ Speech & Audio

📄 Go to detailed file

Speech Recognition

TITML-IDN Speech Corpus

  • Size: 20 speakers (11M, 9F), 343 utterances each
  • Quality: Phonetically balanced
  • Access: Academic/non-commercial use
  • Links: NII Research

Mozilla Common Voice (Indonesian)

  • Description: Crowdsourced speech recognition dataset
  • Links: HuggingFace

CoVoST2

  • Description: Speech translation dataset including Indonesian
  • Links: HuggingFace

Large Javanese ASR Dataset

  • Description: Transcribed audio data for Javanese
  • Source: Google collaboration with universities
  • Links: OpenSLR

Indonesian Speech with Accents

  • Coverage: 5 ethnic groups (Batak, Malay, Javanese, Sundanese, Papuan)
  • Content: 320-word standardized Indonesian text per speaker
  • Links: Kaggle Dataset

Indonesian Speech Recognition (Small)

  • Size: 50 utterances by single male speaker
  • Links: GitHub Repository
  • Note: School project - not for production use

CMU Wilderness Multilingual Speech

  • Coverage: 700+ languages including Indonesian
  • Source: Bible recordings from bible.is
  • Links: GitHub Repository

Google TTS Dataset

  • Description: Automatically generated speech using Google Translate TTS
  • Source: Indonesian newspaper titles
  • Links: Download (dead link, Oct 2026)

Indonesian Unsupervised Speech Dataset

  • Content: Podcast (170GB) + YouTube (90GB)
  • Access: Contact [email protected]
  • Note: Research use only

VoxLingua107 Dataset

LUMINA (Linguistic Unified Multimodal Indonesian Natural Audio-Visual)

  • Description: Constrained audio-visual dataset for speech perception research- Size: 14 native speakers (9 male, 5 female), ~1,000 sentences each- Total: Approximately 14,000 utterances
  • Quality: High-quality facial recordings with controlled environment- Access: Open access via Creative Commons Attribution 4.0 International- Links: Mendeley Data- DOI: 10.17632/8fw93k4rny.4

📝 Text Summarization

📄 Go to detailed file

Summarization Corpora

Liputan6

  • Size: 215K+ article-summary pairs (193,883 train, 10,972 dev/test)
  • Period: 2000-2010 news articles
  • Types: Both abstractive and extractive summaries
  • Links: GitHub Repository | Access Form
  • Paper: AACL-IJCNLP 2020

IndoSum

  • Size: 20K online news article-summary pairs
  • Categories: 6 categories, 10 sources
  • Features: Abstractive summaries + extractive labels
  • Links: GitHub Repository | Data Download
  • Paper: IALP 2018

XLSum

  • Size: 45 languages ranging from low to high-resource
  • Split: Train:38242 Dev:4780 Test:4780 Total:47802
  • Features: Highly abstractive, concise, and high quality
  • Links: HuggingFace

WikiLingua

  • Size: ~770k article and summary pairs in 18 languages from WikiHow
  • Categories: 17 languages
  • Features: Gold-standard article-summary alignments across languages
  • Links: GitHub Repository | HuggingFace

Translated Summarization Datasets

Reddit TLDR

  • Size: ~4M content-summary pairs
  • Source: Social media domain summaries
  • Links: Download (dead link, Oct 2026)

WikiHow (Indonesian)

  • Description: Headline and procedural text pairs
  • Source: WikiHow Indonesia
  • Links: Original

GigaWord

  • Description: Article headline generation dataset
  • Size: ~4M article pairs
  • Links: Download (dead link, Oct 2026)

Indonesian Simple Summaries (Our Works)

  • Description: Indonesian version of Simple Summaries
  • Size: ~10k pair
  • Links: Download

🌍 Machine Translation

📄 Go to detailed file

Translation Pairs

OPUS Parallel Corpus

  • Description: Indonesian paired with 21+ European languages
  • Source: European Parliament proceedings, OpenSubtitles
  • Links: OPUS Collection

ParaCrawl v7.1

  • Description: Web-scale parallel corpora (41 language pairs)
  • Method: Bitextor parallel data crawling
  • Links: HuggingFace IndoParaCrawl

Europarl

  • Description: European Parliament proceedings in 21 languages
  • Links: Original

EuroPat v2

  • Description: Patent translations from USPTO and EPO
  • Links: Original

Specialized Translation Datasets

IWSLT2017

  • Description: TEDtalk subtitles (ID-EN)
  • Size: ~100K sentences
  • Links: WIT3 IWSLT2017

Asian Language Treebank

  • Languages: Indonesian, English, and Asian languages
  • Size: 20K sentences
  • Links: ALT Corpus

IDENTICv1.0

  • Languages: Indonesian-English
  • Size: 45K sentences, ~1M tokens
  • Links: LINDAT Repository

Additional Parallel Corpora

📖 Dictionary & Vocabulary

📄 Go to detailed file

Lexical Resources

Indonesia Wordlist

  • Size: 105,226 words from Kamus Besar Bahasa Indonesia
  • Coverage: Most common Indonesian vocabulary
  • Links: GitHub Repository

Colloquial Indonesian Lexicon

  • Size: 3,592 colloquial tokens → 1,742 lemmas
  • Purpose: Text normalization for informal language
  • Links: GitHub Repository
  • Paper: IEEE 2018

Synonym and Semantic Relations

Tesaurus

  • Description: Indonesian synonym, antonym, and word relationships
  • Links: GitHub Repository

WordNet Bahasa

  • Description: Semantic dictionary inspired by Princeton WordNet for Malay and Indonesian
  • Links: SourceForge

id-wordnet npm package

Word Analogy

KAWAT

  • Description: Word analogy dataset with semantic and syntactic Indonesian word relations
  • Links: GitHub Repository

Formal-Informal Mapping

STIF-Indonesia

  • Description: Dataset for Indonesian formal-informal style conversion
  • Links: GitHub Repository

IndoCollex

  • Description: Comprehensive lexical collection for formal-informal mapping
  • Links: GitHub Repository

Indonesian Slang Mapping

  • Description: Large dictionary mapping slang/alay words to formal Indonesian
  • Links: CSV File

Sentiment Lexicons

Comprehensive Sentiment Word Collections

Root Words

Slang Words

Stopwords

Emoticon & Emoji

Acronyms

Indonesia Regions & Administrative Data

🤖 Pre-trained Models

📄 Go to detailed file

Transformer-based Models

Indo-BERT

  • Description: Indonesian BERT model for various NLP tasks
  • Repository: GitHub
  • Models: HuggingFace

Indo-BERTweet

  • Description: BERT model specifically trained on Indonesian tweets
  • Repository: GitHub
  • Model: HuggingFace

Other Transformer Models

  • Description: Collection of Indonesian RoBERTa, DistilBERT, and other transformer models
  • Links: GitHub Repository

Word Embeddings

Various pre-trained word embeddings are available including:

  • FastText: Indonesian word vectors
  • Word2Vec: Pre-trained Indonesian embeddings
  • Polyglot: Multilingual embeddings including Indonesian

🛠️ Tools & Libraries

📄 Go to detailed file

NLP Processing Tools

Pujangga REST API

Sastrawi Stemmer

NLP-ID

MorphInd

  • Description: Indonesian morphological analyzer
  • Links: Website

INDRA

Indonesian Typo Checker

Framework Integration

Flair NLP

spaCy Bahasa Indonesia

Miscellaneous Utilities

Poetry & Text Generation

Indonesian Poetry & Pantun Generator

  • Description: Collection and generation tools for Indonesian poetry
  • Links: GitHub Repository

Spelling Correction

Norvig's Spell Corrector (Indonesian Adaptation)

  • Description: Adaptation of Norvig's spell corrector for Bahasa Indonesia
  • Original: Norvig's Algorithm

Social Media Scraping

GetOldTweets3

Tweepy

🚀 Quick Start

Loading for Several Datasets

From HuggingFace:

import datasets

# Load IndoNLI
indonli = datasets.load_dataset('indonli')

# Load NusaX datasets
nusax_mt = datasets.load_dataset('indonlp/nusatranslation_mt')
nusax_sentiment = datasets.load_dataset('indonlp/nusatranslation_senti')

# Load OSCAR Indonesian
oscar_id = datasets.load_dataset('oscar-corpus/OSCAR-2301', language='id')

# Load IndoMMLU
indommlu = datasets.load_dataset('indolem/IndoMMLU')

# Load IndoCareer
indocareer = datasets.load_dataset('indolem/IndoCareer')

From GitHub Repositories:

# Clone specific dataset repositories
git clone https://github.com/ir-nlp-csui/indonli
git clone https://github.com/kata-ai/indosum
git clone https://github.com/IndoNLP/nusax
git clone https://github.com/indolem/sum_liputan6
git clone https://github.com/IgoRamli/IndoWiki

Example Usage

Training a Sentiment Classifier:

from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load Indonesian sentiment data
dataset = load_dataset('indonlp/nusatranslation_senti')

# Use Indonesian BERT
tokenizer = AutoTokenizer.from_pretrained('indobenchmark/indobert-base-p1')
model = AutoModelForSequenceClassification.from_pretrained('indobenchmark/indobert-base-p1')

# Training code here...

🤝 Contributing

We welcome contributions to expand this collection! Please:

  1. Fork this repository
  2. Add new datasets with proper documentation
  3. Include dataset descriptions, sizes, and access links
  4. Provide citation information when available
  5. Run python3 scripts/check_links.py — a pull request that adds a dead link fails CI
  6. Submit a pull request with your additions

Dataset Submission Guidelines

When adding new datasets, please include:

  • Dataset name and description
  • Size and format information
  • Download links or access instructions
  • License and usage restrictions
  • Citation information if available
  • Example usage code when possible

📚 Key Papers

  • IndoNLI: Mahendra et al., EMNLP 2021
  • NusaX: Winata et al., EACL 2022
  • IndoSum: Kurniawan & Louvan, IALP 2018
  • Liputan6: Koto et al., AACL-IJCNLP 2020
  • IndoMMLU: Koto et al., EMNLP 2023
  • MSVD-Indonesian: Hendria, arXiv 2023
  • NusaWrites: Cahyawijaya et al., AACL-IJCNLP 2023
  • IndoCareer: Koto, NAACL HLT 2025
  • IndoCloze: Koto et al., CSRR at ACL 2022 (Best Paper Award)
  • POS Tagging Bahasa Indonesia with Flair NLP: Medium Article
  • Indonesian POS Tagset Design: PDF
  • Sentiment Analysis Indonesian Twitter Dataset: ResearchGate
  • Hate Speech Multi-label Classification: ACL 2019
  • Indonesian Treebank Workshop: NTU Workshop Slides

⭐ Star This Repository

🌟 Star this repository if you find it useful for your Indonesian NLP research!

📧 Contact

📧 Contact: Open an issue for questions, suggestions, or dataset additions.

📄 License

This repository serves as a collection of links and information about publicly available datasets. Each dataset has its own license terms:

  • Most academic datasets: Non-commercial research use only
  • News-based datasets: Often restricted by copyright law
  • Government/public domain: More permissive licensing
  • Translated datasets: Subject to original dataset licenses

⚠️ Important: Always check individual dataset licenses before use, especially for commercial applications.

📖 Citations

If you use datasets from this collection, please cite the relevant papers and acknowledge the original dataset creators. See individual dataset sections for specific citation information.

📖 Citations

Repository Citation

When using datasets from this collection, please cite this repository:

@misc{awesome-indonesian-llm-dataset,
  title={Awesome Indonesian LLM Dataset: A Curated Collection for Indonesian AI},
  author={Muhamad Irfan Fadhullah},
  year={2025},
  url={https://github.com/irfanfadhullah/awesome-indonesian-llm-dataset}
}

Individual Dataset Citations

IndoNLI:

@inproceedings{mahendra-etal-2021-indonli,
    title = "{I}ndo{NLI}: A Natural Language Inference Dataset for {I}ndonesian",
    author = "Mahendra, Rahmad and Aji, Alham Fikri and Louvan, Samuel and Rahman, Fahrurrozi and Vania, Clara",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.emnlp-main.821",
    pages = "10511--10527",
}

NusaX:

@misc{winata2022nusax,
      title={NusaX: Multilingual Parallel Sentiment Dataset for 10 Indonesian Local Languages},
      author={Winata, Genta Indra and Aji, Alham Fikri and Cahyawijaya,
      Samuel and Mahendra, Rahmad and Koto, Fajri and Romadhony,
      Ade and Kurniawan, Kemal and Moeljadi, David and Prasojo,
      Radityo Eko and Fung, Pascale and Baldwin, Timothy and Lau,
      Jey Han and Sennrich, Rico and Ruder, Sebastian},
      year={2022},
      eprint={2205.15960},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

IndoSum:

@inproceedings{kurniawan2018,
  place={Bandung, Indonesia},
  title={IndoSum: A New Benchmark Dataset for Indonesian Text Summarization},
  url={https://ieeexplore.ieee.org/document/8629109},
  DOI={10.1109/IALP.2018.8629109},
  booktitle={2018 International Conference on Asian Language Processing (IALP)},
  publisher={IEEE},
  author={Kurniawan, Kemal and Louvan, Samuel},
  year={2018},
  month={Nov},
  pages={215-220}
}

Liputan6:

@inproceedings{koto-etal-2020-liputan6,
    title = "Liputan6: A Large-scale Indonesian Dataset for Text Summarization",
    author = "Koto, Fajri and Lau, Jey Han and Baldwin, Timothy",
    booktitle = "Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing",
    year = "2020"
}

IndoMMLU:

@inproceedings{koto-etal-2023-indommlu,
    title = "Large Language Models Only Pass Primary School Exams in {I}ndonesia: A Comprehensive Test on {I}ndo{MMLU}",
    author = "Fajri Koto and Nurul Aisyah and Haonan Li and Timothy Baldwin",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
    month = December,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
}

MSVD-Indonesian:

{% raw %}
@article{Hendria2023MSVDID,
  title={{MSVD}-{I}ndonesian: A Benchmark for Multimodal Video-Text Tasks in Indonesian},
  author={Willy Fitra Hendria},
  journal={arXiv preprint arXiv:2306.11341},
  year={2023}
}
{% endraw %}

NusaWrites:

@inproceedings{cahyawijaya-etal-2023-nusawrites,
    title = "{N}usa{W}rites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages",
    author = "Cahyawijaya, Samuel  and  Lovenia, Holy  and Koto, Fajri  and  Adhista, Dea  and  Dave, Emmanuel  and  Oktavianti, Sarah  and  Akbar, Salsabil  and  Lee, Jhonson  and  Shadieq, Nuur  and  Cenggoro, Tjeng Wawan  and  Linuwih, Hanung  and  Wilie, Bryan  and  Muridan, Galih  and  Winata, Genta  and  Moeljadi, David  and  Aji, Alham Fikri  and  Purwarianti, Ayu  and  Fung, Pascale",
    booktitle = "Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = nov,
    year = "2023",
    address = "Nusa Dua, Bali",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.ijcnlp-main.60",
    pages = "921--945",
}

MinangNLP:

@inproceedings{koto-koto-2020-towards,
    title = "Towards Computational Linguistics in Minangkabau Language: Studies on Sentiment Analysis and Machine Translation",
    author = "Koto, Fajri and Koto, Ikhwan",
    booktitle = "Proceedings of the 34th Pacific Asia Conference on Language, Information and Computation",
    year = "2020"
}

MadureseSet:

@article{ifada2023madureseset,
  title={MadureseSet: Madurese-Indonesian Dataset},
  author={Ifada, N. and Rachman, F.H. and Syauqy, M.W.M.A. and Wahyuni, S. and Pawitra, A.},
  journal={Data in Brief},
  volume={48},
  pages={109035},
  year={2023},
  doi={https://doi.org/10.1016/j.dib.2023.109035}
}

IndoCareer:

@inproceedings{koto2025cracking,
  title={Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesia},
  author={"Fajri Koto"},
  booktitle={Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2025), Industry Track},
  year={2025}
}

PAWS (Indonesian):

@misc{zhang2019paws,
      title={PAWS: Paraphrase Adversaries from Word Scrambling}, 
      author={Yuan Zhang and Jason Baldridge and Luheng He},
      year={2019},
      eprint={1904.01130},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

CC-100:

@inproceedings{wenzek-etal-2020-ccnet,
    title = "{CCN}et: Extracting High Quality Monolingual Datasets from Web Crawl Data",
    author = "Wenzek, Guillaume  and
      Lachaux, Marie-Anne  and
      Conneau, Alexis  and
      Chaudhary, Vishrav  and
      Guzm{\'a}n, Francisco  and
      Joulin, Armand  and
      Grave, {\'E}douard",
    booktitle = "Proceedings of The 12th Language Resources and Evaluation Conference",
    month = may,
    year = "2020",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2020.lrec-1.494",
    pages = "4003--4012",
}

OSCAR:

@inproceedings{abadji-etal-2022-towards,
    title = "Towards a Cleaner Document-Oriented Multilingual Crawled Corpus",
    author = "Abadji, Julien  and
      Ortiz Suarez, Pedro  and
      Romary, Laurent  and
      Sagot, Benoit",
    booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
    month = jun,
    year = "2022",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2022.lrec-1.463",
    pages = "4344--4355",
}

Conceptual Captions:

@inproceedings{sharma2018conceptual,
  title = {Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning},
  author = {Sharma, Piyush and Ding, Nan and Goodman, Sebastian and Soricut, Radu},
  booktitle = {Proceedings of ACL},
  year = {2018},
}

Large Javanese ASR Dataset:

@inproceedings{kjartansson-etal-sltu2018,
    title = {{Crowd-Sourced Speech Corpora for Javanese, Sundanese,  Sinhala, Nepali, and Bangladeshi Bengali}},
    author = {Oddur Kjartansson and Supheakmungkol Sarin and Knot Pipatsrisawat and Martin Jansche and Linne Ha},
    booktitle = {Proc. The 6th Intl. Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU)},
    year  = {2018},
    address = {Gurugram, India},
    month = aug,
    pages = {52--55},
    URL   = {http://dx.doi.org/10.21437/SLTU.2018-11},
}

OSCAR Corpus:

@inproceedings{abadji-etal-2022-towards,
    title = "Towards a Cleaner Document-Oriented Multilingual Crawled Corpus",
    author = "Abadji, Julien  and
      Ortiz Suarez, Pedro  and
      Romary, Laurent  and
      Sagot, Benoit",
    booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation Conference",
    month = jun,
    year = "2022",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2022.lrec-1.463",
    pages = "4344--4355",
}

CCNet (CC-100):

@inproceedings{wenzek-etal-2020-ccnet,
    title = "{CCN}et: Extracting High Quality Monolingual Datasets from Web Crawl Data",
    author = "Wenzek, Guillaume  and
      Lachaux, Marie-Anne  and
      Conneau, Alexis  and
      Chaudhary, Vishrav  and
      Guzm{\'a}n, Francisco  and
      Joulin, Armand  and
      Grave, {\'E}douard",
    booktitle = "Proceedings of The 12th Language Resources and Evaluation Conference",
    month = may,
    year = "2020",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2020.lrec-1.494",
    pages = "4003--4012",
}

Conceptual Captions:

@inproceedings{sharma2018conceptual,
  title = {Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning},
  author = {Sharma, Piyush and Ding, Nan and Goodman, Sebastian and Soricut, Radu},
  booktitle = {Proceedings of ACL},
  year = {2018},
}

Conceptual 12M:

@inproceedings{changpinyo2021cc12m,
  title = {{Conceptual 12M}: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts},
  author = {Changpinyo, Soravit and Sharma, Piyush and Ding, Nan and Soricut, Radu},
  booktitle = {CVPR},
  year = {2021},
}

YFCC100M:

@article{Thomee_2016,
   title={YFCC100M},
   volume={59},
   ISSN={1557-7317},
   url={http://dx.doi.org/10.1145/2812802},
   DOI={10.1145/2812802},
   number={2},
   journal={Communications of the ACM},
   publisher={Association for Computing Machinery (ACM)},
   author={Thomee, Bart and Shamma, David A. and Friedland, Gerald and Elizalde, Benjamin and Ni, Karl and Poland, Douglas and Borth, Damian and Li, Li-Jia},
   year={2016},
   month={Jan},
   pages={64–73}
}

MultiNLI:

@misc{williams2018broadcoverage,
      title={A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference}, 
      author={Adina Williams and Nikita Nangia and Samuel R. Bowman},
      year={2018},
      eprint={1704.05426},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

ParaBank:

@inproceedings{hu-etal-2019-large,
    title = "Large-Scale, Diverse, Paraphrastic Bitexts via Sampling and Clustering",
    author = "Hu, J. Edward  and
      Singh, Abhinav  and
      Holzenberger, Nils  and
      Post, Matt  and
      Van Durme, Benjamin",
    booktitle = "Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)",
    month = nov,
    year = "2019",
    address = "Hong Kong, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/K19-1005",
    doi = "10.18653/v1/K19-1005",
    pages = "44--54",
}

ParaNMT-50M:

@inproceedings{wieting-gimpel-2018-paranmt,
    title = "{P}ara{NMT}-50{M}: Pushing the Limits of Paraphrastic Sentence Embeddings with Millions of Machine Translations",
    author = "Wieting, John  and
      Gimpel, Kevin",
    booktitle = "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2018",
    address = "Melbourne, Australia",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/P18-1042",
    doi = "10.18653/v1/P18-1042",
    pages = "451--462",
}

SNLI:

@inproceedings{snli:emnlp2015,
	Author = {Bowman, Samuel R. and Angeli, Gabor and Potts, Christopher, and Manning, Christopher D.},
	Booktitle = {Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
	Publisher = {Association for Computational Linguistics},
	Title = {A large annotated corpus for learning natural language inference},
	Year = {2015}
}

Stanford Question Answering Dataset (SQuAD):

@misc{rajpurkar2018know,
      title={Know What You Don't Know: Unanswerable Questions for SQuAD}, 
      author={Pranav Rajpurkar and Robin Jia and Percy Liang},
      year={2018},
      eprint={1806.03822},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Mathematics Dataset:

@inproceedings{saxton2019analysing,
  title={Analysing Mathematical Reasoning Abilities of Neural Models},
  author={Saxton, David and Grefenstette, Edward and Hill, Felix and Kohli, Pushmeet},
  booktitle={International Conference on Learning Representations},
  year={2019}
}

Reddit TLDR:

@inproceedings{volske-etal-2017-tl,
    title = "{TL};{DR}: Mining {R}eddit to Learn Automatic Summarization",
    author = {V{\"o}lske, Michael  and
      Potthast, Martin  and
      Syed, Shahbaz  and
      Stein, Benno},
    booktitle = "Proceedings of the Workshop on New Frontiers in Summarization",
    month = sep,
    year = "2017",
    address = "Copenhagen, Denmark",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/W17-4508",
    doi = "10.18653/v1/W17-4508",
    pages = "59--63",
}

WikiHow:

@misc{koupaee2018wikihow,
      title={WikiHow: A Large Scale Text Summarization Dataset}, 
      author={Mahnaz Koupaee and William Yang Wang},
      year={2018},
      eprint={1810.09305},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

GigaWord:

@article{graff2003english,
  title={English gigaword},
  author={Graff, David and Kong, Junbo and Chen, Ke and Maeda, Kazuaki},
  journal={Linguistic Data Consortium, Philadelphia},
  volume={4},
  number={1},
  pages={34},
  year={2003}
}

Europarl:

@inproceedings{koehn-2005-europarl,
    title = "{E}uroparl: A Parallel Corpus for Statistical Machine Translation",
    author = "Koehn, Philipp",
    booktitle = "Proceedings of Machine Translation Summit X: Papers",
    month = sep # " 13-15",
    year = "2005",
    address = "Phuket, Thailand",
    url = "https://aclanthology.org/2005.mtsummit-papers.11",
    pages = "79--86",
}

ParaCrawl:

@inproceedings{espla-etal-2019-paracrawl,
    title = "{P}ara{C}rawl: Web-scale parallel corpora for the languages of the {EU}",
    author = "Espl{\`a}, Miquel  and
      Forcada, Mikel  and
      Ram{\'\i}rez-S{\'a}nchez, Gema  and
      Hoang, Hieu",
    booktitle = "Proceedings of Machine Translation Summit XVII Volume 2: Translator, Project and User Tracks",
    month = aug,
    year = "2019",
    address = "Dublin, Ireland",
    publisher = "European Association for Machine Translation",
    url = "https://www.aclweb.org/anthology/W19-6721",
    pages = "118--119",
}

AMSunda:

@dataset{maesya_2025_15494944,
  author       = {Maesya, Aries and
                  Arifin, Yulyani and
                  Budiharto, Widodo and
                  Amalia, Zahra},
  title        = {AMSunda: A Novel Dataset for Sundanese Information
                   Retrieval
                  },
  month        = may,
  year         = 2025,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.15494944},
  url          = {https://doi.org/10.5281/zenodo.15494944},
}

SU-CSQA:

@inproceedings{putri-etal-2024-llm,
    title = "Can {LLM} Generate Culturally Relevant Commonsense {QA} Data? Case Study in {I}ndonesian and {S}undanese",
    author = "Putri, Rifki Afina  and
      Haznitrama, Faiz Ghifari  and
      Adhista, Dea  and
      Oh, Alice",
    editor = "Al-Onaizan, Yaser  and
      Bansal, Mohit  and
      Chen, Yun-Nung",
    booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2024",
    address = "Miami, Florida, USA",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.emnlp-main.1145",
    pages = "20571--20590",
}

Javanese Sundanese Story Cloze:

@misc{pranida2025syntheticdatagenerationculturally,
      title={Synthetic Data Generation for Culturally Nuanced Commonsense Reasoning in Low-Resource Languages}, 
      author={Salsabila Zahirah Pranida and Rifo Ahmad Genadi and Fajri Koto},
      year={2025},
      eprint={2502.12932},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.12932}, 
}

IndoCloze:

@inproceedings{koto-etal-2022-cloze,
    title = "Cloze Evaluation for Deeper Understanding of Commonsense Stories in Indonesian",
    author = "Koto, Fajri  and
      Baldwin, Timothy  and
      Lau, Jey Han",
    booktitle = "Proceedings of the First Workshop on Commonsense Representation and Reasoning (CSRR 2022)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.csrr-1.2",
    pages = "9--25",
}

Kompas Online Collection:

@misc{kompas2002,
  title={Kompas Online Collection},
  author={University of Amsterdam},
  year={2002},
  url={https://web.archive.org/web/20201030140022/https://ilps.science.uva.nl/resources/bahasa/}
}

Tempo Online Collection:

@misc{tempo2002,
  title={Tempo Online Collection},
  author={University of Amsterdam},
  year={2002},
  url={https://web.archive.org/web/20201030140022/https://ilps.science.uva.nl/resources/bahasa/}
}

Leipzig Corpora Collection:

@misc{leipzig2013,
  title={Indonesian Mixed Corpus},
  author={Leipzig University},
  year={2013},
  url={https://corpora.uni-leipzig.de/en?corpusId=ind_mixed_2013}
}

Indonesian News Corpus:

@dataset{indonesian_news_2015,
  title={Indonesian News Corpus},
  author={Mendeley Data},
  year={2015},
  url={https://data.mendeley.com/datasets/2zpbjs22k3/1}
}

IDN Tagged Corpus:

@misc{rashel2018idn,
  title={IDN Tagged Corpus},
  author={Rashel, Fam},
  year={2018},
  url={https://github.com/famrashel/idn-tagged-corpus}
}

Aspect and Opinion Terms Extraction:

@article{jordhy2019aspect,
  title={Aspect and Opinion Terms Extraction for Hotel Reviews},
  author={Jordhy, Jordhy},
  journal={arXiv preprint arXiv:1908.04899},
  year={2019}
}

Indonesian Treebank:

@misc{rashel2018treebank,
  title={Indonesian Treebank},
  author={Rashel, Fam},
  year={2018},
  url={https://github.com/famrashel/idn-treebank}
}

Universal Dependencies Indonesian:

@misc{mcdonald2013universal,
  title={Universal Dependencies Indonesian-GSD},
  author={McDonald, Ryan and Nivre, Joakim and Yvonne, Quirmbach-Brundage and others},
  year={2013},
  url={https://github.com/UniversalDependencies/UD_Indonesian-GSD}
}

IDENTICv1.0:

@inproceedings{larasati2012identic,
  title={IDENTICv1.0: Indonesian-English Parallel Corpus},
  author={Larasati, Septina Dian and Kubon, Vladislav and Zeman, Daniel},
  booktitle={Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)},
  year={2012},
  pages={644}
}

CMU Wilderness Multilingual Speech Dataset:

@inproceedings{black2019cmu,
  title={CMU Wilderness Multilingual Speech Dataset},
  author={Black, Alan W},
  booktitle={ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={5971--5975},
  year={2019},
  organization={IEEE}
}

Colloquial Indonesian Lexicon:

@inproceedings{nasalsabila2018,
  title={Colloquial Indonesian Lexicon},
  author={Nasalsabila, Nasal and others},
  booktitle={2018 International Conference on Asian Language Processing (IALP)},
  year={2018},
  organization={IEEE}
}

TITML-IDN Speech Corpus:

@misc{titml2018,
  title={TITML-IDN Speech Corpus},
  author={National Institute of Informatics},
  year={2018},
  url={http://research.nii.ac.jp/src/en/TITML-IDN.html}
}

SMS Spam Dataset:

@misc{wibisono2018sms,
  title={Indonesian SMS Spam Dataset},
  author={Wibisono, Yudi},
  year={2018}
}

Hate Speech Detection:

@misc{alfina2018hate,
  title={Indonesian Hate Speech Detection},
  author={Alfina, Ika and others},
  year={2018},
  url={https://github.com/ialfina/id-hatespeech-detection}
}

Abusive Language Detection:

@misc{ibrohim2018abusive,
  title={Indonesian Abusive Language Detection},
  author={Ibrohim, Okky and others},
  year={2018},
  url={https://github.com/okkyibrohim/id-abusive-language-detection}
}

EuroPat v2:

@misc{europat2019,
  title={EuroPat: Parallel Patent Corpus},
  author={European Patent Organisation},
  year={2019},
  url={https://europat.net/}
}

Rush et al. 2015 (Neural Attention Model for GigaWord):

@article{Rush_2015,
   title={A Neural Attention Model for Abstractive Sentence Summarization},
   url={http://dx.doi.org/10.18653/v1/D15-1044},
   DOI={10.18653/v1/d15-1044},
   journal={Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing},
   publisher={Association for Computational Linguistics},
   author={Rush, Alexander M. and Chopra, Sumit and Weston, Jason},
   year={2015}
}

Radford et al. 2021 (CLIP - Learning Transferable Visual Models):

@misc{radford2021learning,
      title={Learning Transferable Visual Models From Natural Language Supervision}, 
      author={Alec Radford and Jong Wook Kim and Chris Hallacy and Aditya Ramesh and Gabriel Goh and Sandhini Agarwal and Girish Sastry and Amanda Askell and Pamela Mishkin and Jack Clark and Gretchen Krueger and Ilya Sutskever},
      year={2021},
      eprint={2103.00020},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Saxton et al. 2019 (Mathematics Dataset):

@inproceedings{saxton2019analysing,
  title={Analysing Mathematical Reasoning Abilities of Neural Models},
  author={Saxton, David and Grefenstette, Edward and Hill, Felix and Kohli, Pushmeet},
  booktitle={International Conference on Learning Representations},
  year={2019}
}

Raffel et al. 2020 (T5 - Exploring the Limits of Transfer Learning):

@article{Raffel2020ExploringTL,
  title={Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
  author={Colin Raffel and Noam M. Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and W. Li and Peter J. Liu},
  journal={ArXiv},
  year={2020},
  volume={abs/1910.10683}
}

Cheng & Lapata 2016 (Neural Summarization referenced in IndoSum):

@inproceedings{cheng-lapata-2016-neural,
    title = "Neural Summarization by Extracting Sentences and Words",
    author = "Cheng, Jianpeng  and
      Lapata, Mirella",
    booktitle = "Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = aug,
    year = "2016",
    address = "Berlin, Germany",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/P16-1046",
    doi = "10.18653/v1/P16-1046",
    pages = "484--494",
}

VoxLingua107 Dataset

@inproceedings{valk2021slt,
  title={{VoxLingua107}: a Dataset for Spoken Language Recognition},
  author={J{\"o}rgen Valk and Tanel Alum{\"a}e},
  booktitle={Proc. IEEE SLT Workshop},
  year={2021},
}

WiLI-18 Dataset

@dataset{thoma_2018_841984,
  author       = {Thoma, Martin},
  title        = {WiLI-2018 - Wikipedia Language Identification
                   database
                  },
  month        = jan,
  year         = 2018,
  publisher    = {Zenodo},
  version      = {1.0.0},
  doi          = {10.5281/zenodo.841984},
  url          = {https://doi.org/10.5281/zenodo.841984},
}

WikiANN Dataset

@inproceedings{rahimi-etal-2019-massively,
    title = "Massively Multilingual Transfer for {NER}",
    author = "Rahimi, Afshin  and
      Li, Yuan  and
      Cohn, Trevor",
    booktitle = "Proceedings of the 57th Annual Meeting of the Association \
    for Computational Linguistics",
    month = jul,
    year = "2019",
    address = "Florence, Italy",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/P19-1015",
    pages = "151--164",
}

Tatoeba

@inproceedings{tiedemann-2020-tatoeba,
    title = "The Tatoeba Translation Challenge {--} Realistic Data Sets for Low Resource and Multilingual {MT}",
    author = {Tiedemann, J{\"o}rg},
    booktitle = "Proceedings of the Fifth Conference on Machine Translation",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.wmt-1.139",
    pages = "1174--1182",
}

The QCRI Educational Domain Corpus (formerly QCRI AMARA Corpus)

@inproceedings{abdelali-etal-2014-amara,
    title = "The {AMARA} Corpus: Building Parallel Language Resources for the Educational Domain",
    author = "Abdelali, Ahmed  and
      Guzman, Francisco  and
      Sajjad, Hassan  and
      Vogel, Stephan",
    editor = "Calzolari, Nicoletta  and
      Choukri, Khalid  and
      Declerck, Thierry  and
      Loftsson, Hrafn  and
      Maegaard, Bente  and
      Mariani, Joseph  and
      Moreno, Asuncion  and
      Odijk, Jan  and
      Piperidis, Stelios",
    booktitle = "Proceedings of the Ninth International Conference on Language Resources and Evaluation ({LREC}'14)",
    month = may,
    year = "2014",
    address = "Reykjavik, Iceland",
    publisher = "European Language Resources Association (ELRA)",
    url = "https://aclanthology.org/L14-1675/",
    pages = "1856--1862",
    abstract = "This paper presents the AMARA corpus of on-line educational content: a new parallel corpus of educational video subtitles, multilingually aligned for 20 languages, i.e. 20 monolingual corpora and 190 parallel corpora. This corpus includes both resource-rich languages such as English and Arabic, and resource-poor languages such as Hindi and Thai. In this paper, we describe the gathering, validation, and preprocessing of a large collection of parallel, community-generated subtitles. Furthermore, we describe the methodology used to prepare the data for Machine Translation tasks. Additionally, we provide a document-level, jointly aligned development and test sets for 14 language pairs, designed for tuning and testing Machine Translation systems. We provide baseline results for these tasks, and highlight some of the challenges we face when building machine translation systems for educational content."
}

HOASA Dataset

@inproceedings{azhar2019multi,
  title={Multi-label Aspect Categorization with Convolutional Neural Networks and Extreme Gradient Boosting},
  author={A. N. Azhar, M. L. Khodra, and A. P. Sutiono}
  booktitle={Proceedings of the 2019 International Conference on Electrical Engineering and Informatics (ICEEI)},
  pages={35--40},
  year={2019}
}

CASA Dataset

@INPROCEEDINGS{8629181,
    author={Ilmania, Arfinda and Abdurrahman and Cahyawijaya, Samuel and Purwarianti, Ayu},
    booktitle={2018 International Conference on Asian Language Processing (IALP)},
    title={Aspect Detection and Sentiment Classification Using Deep Neural Network for Indonesian Aspect-Based Sentiment Analysis},
    year={2018},
    volume={},
    number={},
    pages={62-67},
    doi={10.1109/IALP.2018.8629181}
}

Indonesian CLickbait Headlines Dataset

@article{WILLIAM2020106231,
title = {CLICK-ID: A novel dataset for Indonesian clickbait headlines},
journal = {Data in Brief},
volume = {32},
pages = {106231},
year = {2020},
issn = {2352-3409},
doi = {https://doi.org/10.1016/j.dib.2020.106231},
url = {https://www.sciencedirect.com/science/article/pii/S2352340920311252},
author = {Andika William and Yunita Sari},
keywords = {Indonesian, Natural Language Processing, News articles, Clickbait, Text-classification},
abstract = {News analysis is a popular task in Natural Language Processing (NLP). In particular, the problem of clickbait in news analysis has gained attention in recent years [1, 2]. However, the majority of the tasks has been focused on English news, in which there is already a rich representative resource. For other languages, such as Indonesian, there is still a lack of resource for clickbait tasks. Therefore, we introduce the CLICK-ID dataset of Indonesian news headlines extracted from 12 Indonesian online news publishers. It is comprised of 15,000 annotated headlines with clickbait and non-clickbait labels. Using the CLICK-ID dataset, we then developed an Indonesian clickbait classification model achieving favourable performance. We believe that this corpus will be useful for replicable experiments in clickbait detection or other experiments in NLP areas.}
}

SmSA Dataset

@INPROCEEDINGS{8904199,
    author={Purwarianti, Ayu and Crisdayanti, Ida Ayu Putu Ari},
    booktitle={2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)},
    title={Improving Bi-LSTM Performance for Indonesian Sentiment Analysis Using Paragraph Vector},
    year={2019},
    pages={1-5},
    doi={10.1109/ICAICTA.2019.8904199}
}

Dataloader for CASA HOASA Dataset

@article{lovenia2024seacrowd,
    title={SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages}, 
    author={Holy Lovenia and Rahmad Mahendra and Salsabil Maulana Akbar and Lester James V. Miranda and Jennifer Santoso and Elyanah Aco and Akhdan Fadhilah and Jonibek Mansurov and Joseph Marvin Imperial and Onno P. Kampman and Joel Ruben Antony Moniz and Muhammad Ravi Shulthan Habibi and Frederikus Hudi and Railey Montalan and Ryan Ignatius and Joanito Agili Lopo and William Nixon and Börje F. Karlsson and James Jaya and Ryandito Diandaru and Yuze Gao and Patrick Amadeus and Bin Wang and Jan Christian Blaise Cruz and Chenxi Whitehouse and Ivan Halim Parmonangan and Maria Khelli and Wenyu Zhang and Lucky Susanto and Reynard Adha Ryanda and Sonny Lazuardi Hermawan and Dan John Velasco and Muhammad Dehan Al Kautsar and Willy Fitra Hendria and Yasmin Moslem and Noah Flynn and Muhammad Farid Adilazuarda and Haochen Li and Johanes Lee and R. Damanhuri and Shuo Sun and Muhammad Reza Qorib and Amirbek Djanibekov and Wei Qi Leong and Quyet V. Do and Niklas Muennighoff and Tanrada Pansuwan and Ilham Firdausi Putra and Yan Xu and Ngee Chia Tai and Ayu Purwarianti and Sebastian Ruder and William Tjhi and Peerat Limkonchotiwat and Alham Fikri Aji and Sedrick Keh and Genta Indra Winata and Ruochen Zhang and Fajri Koto and Zheng-Xin Yong and Samuel Cahyawijaya},
    year={2024},
    eprint={2406.10118},
    journal={arXiv preprint arXiv: 2406.10118}
}

WReTE Dataset

@inproceedings{10.1007/978-3-031-23793-5_34,
author = {Setya, Ken Nabila and Mahendra, Rahmad},
title = {Semi-supervised Textual Entailment on Indonesian Wikipedia Data},
year = {2018},
isbn = {978-3-031-23792-8},
publisher = {Springer-Verlag},
address = {Berlin, Heidelberg},
url = {https://doi.org/10.1007/978-3-031-23793-5_34},
doi = {10.1007/978-3-031-23793-5_34},
abstract = {Recognizing Textual Entailment (RTE) is a research in Natural Language Processing that aims to identify whether there is an entailment relation between two texts. Textual Entailment has been studied in a variety of languages, but it is rare for the Indonesian language. The purpose of the work presented in this paper is to conduct the RTE experiment on Indonesian language dataset. Since manual data creation is costly and time consuming, we choose semi-supervised machine learning approach. We apply co-training algorithm to enlarge small amounts of annotated data, called seeds. With this method, the human effort only needed to annotate the seeds. The data resource used is all from Wikipedia and the entailment pairs are extracted from its revision history. Evaluation of 1,857 sentence pairs labelled with entailment information using our approach achieve accuracy 76\%.},
booktitle = {Computational Linguistics and Intelligent Text Processing: 19th International Conference, CICLing 2018, Hanoi, Vietnam, March 18–24, 2018, Revised Selected Papers, Part I},
pages = {416–427},
numpages = {12},
keywords = {Textual entailment, Co-training, Wikipedia revision history, Indonesian language, Semi-supervised, LSTM},
location = {Hanoi, Vietnam}
}

EMoT Dataset

@INPROCEEDINGS{8629262,
  author={Saputri, Mei Silviana and Mahendra, Rahmad and Adriani, Mirna},
  booktitle={2018 International Conference on Asian Language Processing (IALP)}, 
  title={Emotion Classification on Indonesian Twitter Dataset}, 
  year={2018},
  volume={},
  number={},
  pages={90-95},
  keywords={Twitter;Feature extraction;Task analysis;Standards;Government;Logistics;natural language processing;emotion classification;indonesian tweet;feature engineering},
  doi={10.1109/IALP.2018.8629262}}

SentiWS

@inproceedings{chen-skiena-2014-building,
    title = "Building Sentiment Lexicons for All Major Languages",
    author = "Chen, Yanqing  and
      Skiena, Steven",
    editor = "Toutanova, Kristina  and
      Wu, Hua",
    booktitle = "Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jun,
    year = "2014",
    address = "Baltimore, Maryland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/P14-2063/",
    doi = "10.3115/v1/P14-2063",
    pages = "383--389"
}

POSP Dataset

@inproceedings{hoesen2018investigating,
  title={Investigating Bi-LSTM and CRF with POS Tag Embedding for Indonesian Named Entity Tagger},
  author={Devin Hoesen and Ayu Purwarianti},
  booktitle={Proceedings of the 2018 International Conference on Asian Language Processing (IALP)},
  pages={35--38},
  year={2018},
  organization={IEEE}
}

BaPOS Dataset

@inproceedings{dinakaramani2014designing,
  title={Designing an Indonesian Part of Speech Tagset and Manually Tagged Indonesian Corpus},
  author={Arawinda Dinakaramani, Fam Rashel, Andry Luthfi, and Ruli Manurung},
  booktitle={Proceedings of the 2014 International Conference on Asian Language Processing (IALP)},
  pages={66--69},
  year={2014},
  organization={IEEE}
}
@inproceedings{kurniawan2018toward,
  title={Toward a Standardized and More Accurate Indonesian Part-of-Speech Tagging},
  author={Kemal Kurniawan and Alham Fikri Aji},
  booktitle={Proceedings of the 2018 International Conference on Asian Language Processing (IALP)},
  pages={303--307},
  year={2018},
  organization={IEEE}
}

TermA Dataset

@article{winatmoko2019aspect,
  title={Aspect and Opinion Term Extraction for Hotel Reviews Using Transfer Learning and Auxiliary Labels},
  author={Yosef Ardhito Winatmoko, Ali Akbar Septiandri, Arie Pratama Sutiono},
  journal={arXiv preprint arXiv:1909.11879},
  year={2019}
}
@article{fernando2019aspect,
  title={Aspect and Opinion Terms Extraction Using Double Embeddings and Attention Mechanism for Indonesian Hotel Reviews},
  author={Jordhy Fernando, Masayu Leylia Khodra, Ali Akbar Septiandri},
  journal={arXiv preprint arXiv:1908.04899},
  year={2019}
}

KEPS Dataset

@inproceedings{mahfuzh2019improving,
  title={Improving Joint Layer RNN based Keyphrase Extraction by Using Syntactical Features},
  author={Miftahul Mahfuzh, Sidik Soleman, and Ayu Purwarianti},
  booktitle={Proceedings of the 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA)},
  pages={1--6},
  year={2019},
  organization={IEEE}
}

NERGrit Dataset

@online{nergrit2019,
  title={NERGrit Corpus},
  author={NERGrit Developers},
  year={2019},
  url={https://huggingface.co/datasets/grit-id/id_nergrit_corpus}
}

NERP Dataset

@inproceedings{hoesen2018investigating,
  title={Investigating Bi-LSTM and CRF with POS Tag Embedding for Indonesian Named Entity Tagger},
  author={Devin Hoesen and Ayu Purwarianti},
  booktitle={Proceedings of the 2018 International Conference on Asian Language Processing (IALP)},
  pages={35--38},
  year={2018},
  organization={IEEE}
}

FacQA

@inproceedings{purwarianti2007machine,
  title={A Machine Learning Approach for Indonesian Question Answering System},
  author={Ayu Purwarianti, Masatoshi Tsuchiya, and Seiichi Nakagawa},
  booktitle={Proceedings of Artificial Intelligence and Applications },
  pages={573--578},
  year={2007}
}

TyDi QA

@article{clark-etal-2020-tydi,
    title = "{T}y{D}i {QA}: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages",
    author = "Clark, Jonathan H.  and
      Choi, Eunsol  and
      Collins, Michael  and
      Garrette, Dan  and
      Kwiatkowski, Tom  and
      Nikolaev, Vitaly  and
      Palomaki, Jennimaria",
    editor = "Johnson, Mark  and
      Roark, Brian  and
      Nenkova, Ani",
    journal = "Transactions of the Association for Computational Linguistics",
    volume = "8",
    year = "2020",
    address = "Cambridge, MA",
    publisher = "MIT Press",
    url = "https://aclanthology.org/2020.tacl-1.30",
    doi = "10.1162/tacl_a_00317",
    pages = "454--470",
    abstract = "Confidently making progress on multilingual modeling requires challenging, trustworthy evaluations.
    We present TyDi QA{---}a question answering dataset covering 11 typologically diverse languages with 204K
    question-answer pairs. The languages of TyDi QA are diverse with regard to their typology{---}the set of
    linguistic features each language expresses{---}such that we expect models performing well on this set to
    generalize across a large number of the world{'}s languages. We present a quantitative analysis of the data
    quality and example-level qualitative linguistic analyses of observed language phenomena that would not be found
    in English-only corpora. To provide a realistic information-seeking task and avoid priming effects, questions are
    written by people who want to know the answer, but don{'}t know the answer yet, and the data is collected directly
    in each language without the use of translation.",
}

mLAMA

@article{kassner2021multilingual,
  author    = {Nora Kassner and
               Philipp Dufter and
               Hinrich Sch{\"{u}}tze},
  title     = {Multilingual {LAMA:} Investigating Knowledge in Multilingual Pretrained
               Language Models},
  journal   = {CoRR},
  volume    = {abs/2102.00894},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.00894},
  archivePrefix = {arXiv},
  eprint    = {2102.00894},
  timestamp = {Tue, 09 Feb 2021 13:35:56 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2102-00894.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
  note      = {to appear in EACL2021}
}

XLSum Dataset

@inproceedings{hasan-etal-2021-xl,
    title = "{XL}-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages",
    author = "Hasan, Tahmid  and
      Bhattacharjee, Abhik  and
      Islam, Md. Saiful  and
      Mubasshir, Kazi  and
      Li, Yuan-Fang  and
      Kang, Yong-Bin  and
      Rahman, M. Sohel  and
      Shahriyar, Rifat",
    booktitle = "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.findings-acl.413",
    pages = "4693--4703",
}

WikiLingua Dataset

@inproceedings{ladhak-wiki-2020,
    title={WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization},
    author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
    booktitle={Findings of EMNLP, 2020},
    year={2020}
}

LUMINA Dataset

@article{SETYANINGSIH2024110279,
title = {LUMINA: Linguistic unified multimodal Indonesian natural audio-visual dataset},
journal = {Data in Brief},
volume = {54},
pages = {110279},
year = {2024},
issn = {2352-3409},
doi = {https://doi.org/10.1016/j.dib.2024.110279},
url = {https://www.sciencedirect.com/science/article/pii/S2352340924002488},
author = {Eka Rahayu Setyaningsih and Anik Nur Handayani and Wahyu Sakti Gunawan Irianto and Yosi Kristian and Christian Trisno Sen Long Chen},
keywords = {Constrained audio-visual dataset, Lips reading, Speech synthesis, Face processing, Computer vision},
abstract = {The LUMINA (Linguistic Unified Multimodal Indonesian Natural Audio-Visual) Dataset is a carefully curated constrained audio-visual dataset designed to support research in the field of speech perception. Spoken exclusively in Indonesian, LUMINA contains high-quality audio-visual recordings featuring 14 native speakers, including 9 males and 5 females. Each speaker contributes approximately 1,000 sentences, producing a rich and diverse data collection. The recorded videos focus on facial recordings, capturing essential visual cues and expressions that accompany speech. This extensive dataset provides a valuable resource for understanding how humans perceive and process spoken language, paving the way for speech recognition and synthesis technology advancements.}
}

About

Link-checked catalogue of Indonesian datasets for language and multimodal models — Bahasa Indonesia plus 10 local languages.

Topics

awesome-listdataset-cataloguedatasetsindonesianindonesian-nlpllmlow-resource-languagesmachine-learningnlp

Resources

Readme
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Used by

Contributors

Languages