Data Cleaner

A comprehensive data cleaning application with support for both structured and unstructured data processing.

Features

Unstructured Data

• Text extraction and processing
• Automatic schema inference
• Data structure extraction

Structured Data

• CSV processing
• Data validation and cleaning
• Error detection and repair

Techniques

• Data cleaning
• Validation
• Multi-source resolution
• Intelligent error handling
• Robust decision-making under uncertainty
• LLM-powered semantic cleaning

LLM API Configuration

The application supports two LLM providers:

Anthropic Claude (Default)

Set your Anthropic API key in the environment:

export ANTHROPIC_API_KEY=your_anthropic_api_key_here

Cerebras API (Alternative)

To use Cerebras API instead of Anthropic:

export USE_CEREBRAS_API=true
export CEREBRAS_API_KEY=your_cerebras_api_key_here

When USE_CEREBRAS_API=true, the application will use Cerebras API for all LLM operations, regardless of whether an Anthropic API key is present.

Running the Application

Backend

uv run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Frontend

cd frontend
npm run dev

Testing

Test the LLM integration:

python test_cerebras_integration.py