IntentGPT: Few-Shot Intent Discovery with Large Language Models

Juan A. Rodriguez, Nicholas Botzer, David Vazquez, Christopher Pal, Marco Pedersoli, Issam Laradji

IntentGPT is a training-free framework that leverages Large Language Models to efficiently discover and classify user intents with minimal labeled data. By combining informative prompt engineering, semantic feedback mechanisms, and in-context learning, IntentGPT adapts to emerging intents without requiring domain-specific fine-tuning.

Key Features

• 🚀 Training-Free: No model fine-tuning required - works out of the box with OpenAI models
• 🎯 Few-Shot Learning: Discovers new intents with just a few examples per class
• 🔄 Semantic Feedback: Iteratively improves intent discovery using embedding-based similarity
• 📊 Multi-Dataset Support: Tested on Banking, CLINC, SNIPS, StackOverflow, and mTOP datasets
• 💪 Robust Performance: Handles ambiguous utterances and maintains consistency across domains

Installation

1. Clone the repository

   git clone joanrod/intentgpt
   cd intentgpt

2. Create and activate a virtual environment

   python -m venv venv
   source venv/bin/activate 

3. Install dependencies

   pip install -r requirements.txt

4. Set up environment variables

   export OPENAI_API_KEY="your_openai_api_key_here"
   export HF_HOME="path_to_hf"

Usage

Run IntentGPT with a configuration file:

python main.py config=configs/banking.yaml

Additional Parameters

You can override config parameters via command line:

python main.py config=configs/banking.yaml attack=true use_wandb=false temperature=0.2

Available Datasets

• banking - Banking intent classification
• clinc - CLINC150 intent dataset
• snips - SNIPS voice assistant dataset
• stackoverflow - StackOverflow question classification
• mtop-{lang} - Multilingual Task-Oriented Parsing (de, en, es, fr, hi)

Key Parameters

• attack: Enable adversarial testing (default: false)
• scramble_known_intents: Randomize known intent order (default: false)
• scramble_test_samples: Randomize test sample order (default: false)
• use_wandb: Enable Weights & Biases logging (default: false)
• temperature: LLM sampling temperature (default: 0.15)
• gpt_model: OpenAI model to use (default: gpt-3.5-turbo)

Results

Results are saved in results_v3/{dataset}/ with experiment metadata and performance metrics.

Project Structure

intentgpt/
├── intentgpt/                   # Main package
│   ├── core/                    # Core functionality
│   │   ├── intent_gpt.py        # Main IntentGPT class
│   │   └── embeddings.py        # Embedding utilities
│   ├── data/                    # Data handling
│   │   ├── dataloaders.py       # Data loading
│   │   └── processors.py        # Data processing
│   └── utils/                   # Utility functions
│       ├── metrics.py           # Evaluation metrics
│       ├── prompt_utils.py      # Prompt handling
│       └── experiment_utils.py  # Experiment tools
├── scripts/                     # Analysis scripts
├── configs/                     # Configuration files
├── prompts/                     # Prompt templates
└── data/                        # Dataset files
    ├── banking/
    │   ├── train.tsv
    │   ├── test.tsv
    │   └── eval.tsv
    └── ...

Citation

@article{rodriguez2024intentgpt,
  title={Intentgpt: Few-shot intent discovery with large language models},
  author={Rodriguez, Juan A and Botzer, Nicholas and Vazquez, David and Pal, Christopher and Pedersoli, Marco and Laradji, Issam},
  journal={arXiv preprint arXiv:2411.10670},
  year={2024}
}

License

This project is licensed under the Apache License, Version 2.0 - see the LICENSE.txt file for details.