OpenLearn Backend

A comprehensive TypeScript backend for cohort-based educational platforms

Modern educational platform featuring role-based learning paths, specializations, progress tracking, and gamification systems

Technology Stack

Live Services

Frontend: openlearn.org.in - Main platform interface
API: api.openlearn.org.in - REST API backend
Status: api.openlearn.org.in/status-page - System status dashboard
Monitoring: ops.openlearn.org.in - Grafana metrics dashboard
Health: api.openlearn.org.in/health - Health check endpoint

Documentation

Quick Access

CHANGELOG.md - Version history and updates
API Documentation - Complete API reference and guides
Architecture - System design and technical architecture
Development - Development guides and tutorials
Deployment - Production deployment and CI/CD
Monitoring - Prometheus & Grafana setup and usage

Core API References

Authentication API - Complete authentication system
Course Management - Course and cohort administration
Progress Tracking - Learning progress and analytics
Assignment System - Assignment submission and grading

Architecture & Operations

Complete Backend Guide - Full system overview
Authentication System - Security and authentication architecture
Email Service - Email system design and flow
Prometheus & Grafana Setup - Monitoring infrastructure guide

Platform Features

Educational Structure

Cohort-based learning with specialized tracks
Multi-league specializations (AI/ML, Finance, Creative, etc.)
Week-by-week structured curriculum delivery
Section-based content organization with progress tracking

User Management

Role-based access control (Pioneers, Pathfinders, Grand Pathfinders)
Hierarchical permission system with league-specific scoping
Social profiles and peer interaction features
Comprehensive user progress analytics

Content Management

Multi-format resources (blogs, videos, articles, external links)
Assignment submission and evaluation system
Personal notes and revision marking
Badge and achievement system

Technical Features

RESTful API design with comprehensive documentation
JWT-based authentication with refresh token rotation
Real-time health monitoring and status pages
Email service integration (Resend, Amazon SES, Mailtrap)
Production-grade observability with Prometheus & Grafana
Docker containerization for consistent deployment
Automated CI/CD with GitHub Actions

Technical Architecture

System Overview

graph TB
    subgraph "Client Layer"
        WEB[Web Dashboard]
        OPS[Observability Dashboard]
        STATUS[Status Page]
    end
    
    subgraph "Load Balancing & SSL"
        NGINX[Nginx Reverse Proxy]
        SSL[SSL/TLS Termination]
    end
    
    subgraph "Application Services"
        API[Express.js API]
        AUTH[Auth Middleware]
        RBAC[RBAC System]
        METRICS_MW[Metrics Middleware]
    end
    
    subgraph "Business Logic"
        USER_SVC[User Management]
        COHORT_SVC[Cohort Service]
        CONTENT_SVC[Content Management]
        PROGRESS_SVC[Progress Tracking]
        BADGE_SVC[Badge System]
        EMAIL_SVC[Email Service]
    end
    
    subgraph "Data Layer"
        POSTGRES[(PostgreSQL 15)]
        REDIS[(Redis Cache)]
    end
    
    subgraph "Monitoring Stack"
        PROMETHEUS[Prometheus]
        GRAFANA[Grafana Dashboard]
        METRICS[Metrics Endpoint]
    end
    
    subgraph "External Services"
        RESEND[Resend API]
        AWS_SES[Amazon SES]
        MAILTRAP[Mailtrap]
    end
    
    WEB --> NGINX
    OPS --> NGINX
    STATUS --> NGINX
    
    NGINX --> SSL
    SSL --> API
    API --> METRICS_MW
    METRICS_MW --> AUTH
    AUTH --> RBAC
    RBAC --> USER_SVC
    RBAC --> COHORT_SVC
    RBAC --> CONTENT_SVC
    RBAC --> PROGRESS_SVC
    RBAC --> BADGE_SVC
    
    USER_SVC --> POSTGRES
    COHORT_SVC --> POSTGRES
    CONTENT_SVC --> POSTGRES
    PROGRESS_SVC --> POSTGRES
    BADGE_SVC --> POSTGRES
    EMAIL_SVC --> POSTGRES
    
    API --> REDIS
    EMAIL_SVC --> REDIS
    
    EMAIL_SVC --> RESEND
    EMAIL_SVC --> AWS_SES
    EMAIL_SVC --> MAILTRAP
    
    API --> METRICS
    PROMETHEUS --> METRICS
    GRAFANA --> PROMETHEUS
    
    style POSTGRES fill:#336791,stroke:#fff,color:#fff
    style REDIS fill:#DC382D,stroke:#fff,color:#fff
    style API fill:#000,stroke:#fff,color:#fff
    style NGINX fill:#009639,stroke:#fff,color:#fff
    style PROMETHEUS fill:#E6522C,stroke:#fff,color:#fff
    style GRAFANA fill:#F46800,stroke:#fff,color:#fff

Database Schema (Core Models)

erDiagram
    User ||--o{ Enrollment : enrolls
    User ||--o{ SectionProgress : tracks
    User ||--o{ ResourceProgress : completes
    User ||--o{ UserBadge : earns
    User ||--o{ AssignmentSubmission : submits
    User ||--o{ AuditLog : generates
    
    Cohort ||--o{ Enrollment : contains
    Cohort ||--o{ Specialization : offers
    
    Specialization ||--o{ SpecializationLeague : connects
    League ||--o{ SpecializationLeague : belongs
    League ||--o{ Week : organizes
    League ||--o{ Badge : offers
    League ||--o{ Assignment : contains
    
    Week ||--o{ Section : includes
    Section ||--o{ SectionResource : contains
    Section ||--o{ SectionProgress : tracked_by
    
    SectionResource ||--o{ ResourceProgress : measured_by
    Assignment ||--o{ AssignmentSubmission : receives
    Badge ||--o{ UserBadge : awarded_as
    
    User {
        string id PK
        string email UK
        string olid UK
        string name
        UserRole role
        UserStatus status
        datetime createdAt
    }
    
    Cohort {
        string id PK
        string name
        string description
        boolean isActive
        datetime createdAt
    }
    
    League {
        string id PK
        string name
        string description
        string iconUrl
        datetime createdAt
    }
    
    Week {
        string id PK
        string leagueId FK
        string name
        int orderIndex
        datetime createdAt
    }
    
    Section {
        string id PK
        string weekId FK
        string name
        int orderIndex
        datetime createdAt
    }

Monitoring & Observability

OpenLearn includes production-grade monitoring with Prometheus and Grafana for real-time metrics, performance tracking, and system health.

Monitoring Architecture

graph LR
    subgraph "Application"
        APP[Express.js API]
        METRICS_MW[Metrics Middleware]
        METRICS_EP[Metrics Endpoint]
    end
    
    subgraph "Metrics Collection"
        PROM[Prometheus<br/>:9090]
        SCRAPE[Scrapes every 15s]
    end
    
    subgraph "Visualization"
        GRAFANA[Grafana Dashboard<br/>ops.openlearn.org.in]
        DASHBOARDS[Custom Dashboards]
    end
    
    subgraph "Metrics Types"
        HTTP[HTTP Metrics]
        DB[Database Metrics]
        AUTH[Auth Metrics]
        RATE[Rate Limit Metrics]
        NODE[Node.js Metrics]
    end
    
    APP --> METRICS_MW
    METRICS_MW --> HTTP
    METRICS_MW --> DB
    METRICS_MW --> AUTH
    METRICS_MW --> RATE
    METRICS_MW --> NODE
    
    HTTP --> METRICS_EP
    DB --> METRICS_EP
    AUTH --> METRICS_EP
    RATE --> METRICS_EP
    NODE --> METRICS_EP
    
    METRICS_EP --> SCRAPE
    SCRAPE --> PROM
    PROM --> GRAFANA
    GRAFANA --> DASHBOARDS
    
    style PROM fill:#E6522C,stroke:#fff,color:#fff
    style GRAFANA fill:#F46800,stroke:#fff,color:#fff
    style APP fill:#000,stroke:#fff,color:#fff

Metrics Collected

HTTP Metrics

Request duration (histogram)
Request count by method/route/status
In-flight requests (gauge)
Request/response sizes
Error rates by type

Database Metrics

Query execution time (histogram)
Query count by operation/model
Active/idle connections
Transaction duration
Error rates by operation

Authentication Metrics

Login attempts (success/failure)
Token validations
JWT errors
Active authenticated users
Authorization failures

Rate Limiting Metrics

Rate limit hits (allowed/blocked)
Exceeded limits by endpoint type

Node.js Metrics (Default)

Heap memory usage
Event loop lag
CPU usage
Garbage collection stats

Access Monitoring

Grafana Dashboard: ops.openlearn.org.in
Prometheus UI: Internal only (port 9090, not publicly exposed)
Metrics Endpoint: /metrics (protected by X-API-Secret for external access)

Deployment Architecture

Production Infrastructure

graph TB
    subgraph "External Layer"
        USER[Users]
        DOMAIN[openlearn.org.in]
        API_DOMAIN[api.openlearn.org.in]
        OPS_DOMAIN[ops.openlearn.org.in]
    end
    
    subgraph "Security & Load Balancing"
        CF[CloudFlare]
        SSL[SSL/TLS]
        NGINX[Nginx Reverse Proxy]
    end
    
    subgraph "Docker Environment"
        APP_CONTAINER[App Container<br/>:3000]
        DB_CONTAINER[PostgreSQL<br/>:5432]
        REDIS_CONTAINER[Redis<br/>:6379]
        PROM_CONTAINER[Prometheus<br/>:9090]
        GRAFANA_CONTAINER[Grafana<br/>:3001]
    end
    
    subgraph "Monitoring & Health"
        HEALTH[Health Checks]
        STATUS_PAGE[Status Dashboard]
        METRICS[Metrics Collection]
    end
    
    subgraph "External Services"
        RESEND[Resend API]
        AWS_SES[Amazon SES]
    end
    
    USER --> DOMAIN
    USER --> API_DOMAIN
    USER --> OPS_DOMAIN
    DOMAIN --> CF
    API_DOMAIN --> CF
    OPS_DOMAIN --> CF
    CF --> SSL
    SSL --> NGINX
    
    NGINX --> APP_CONTAINER
    NGINX --> GRAFANA_CONTAINER
    
    APP_CONTAINER --> DB_CONTAINER
    APP_CONTAINER --> REDIS_CONTAINER
    APP_CONTAINER --> METRICS
    
    PROM_CONTAINER --> METRICS
    GRAFANA_CONTAINER --> PROM_CONTAINER
    
    APP_CONTAINER --> HEALTH
    HEALTH --> STATUS_PAGE
    
    APP_CONTAINER --> RESEND
    APP_CONTAINER --> AWS_SES
    
    style APP_CONTAINER fill:#4CAF50,stroke:#fff,color:#fff
    style DB_CONTAINER fill:#336791,stroke:#fff,color:#fff
    style REDIS_CONTAINER fill:#DC382D,stroke:#fff,color:#fff
    style PROM_CONTAINER fill:#E6522C,stroke:#fff,color:#fff
    style GRAFANA_CONTAINER fill:#F46800,stroke:#fff,color:#fff
    style NGINX fill:#009639,stroke:#fff,color:#fff

CI/CD Pipeline

graph TB
    subgraph "Source Control"
        PUSH[Code Push to main]
    end
    
    subgraph "GitHub Actions"
        CHECKOUT[Checkout Code]
        DEPS[Install Dependencies]
        BUILD[TypeScript Build]
        PRISMA[Generate Prisma Client]
        PACKAGE[Create Tarball]
        UPLOAD[Upload Artifact]
    end
    
    subgraph "Deployment"
        SCP[SCP to EC2]
        EXTRACT[Extract Tarball]
        REBUILD[Rebuild Native Modules]
        MIGRATE[Run Migrations]
        RESTART[Restart Containers]
    end
    
    subgraph "Verification"
        HEALTH_CHECK[Health Check]
        SMOKE_TEST[Smoke Tests]
        NOTIFY[Notify Status]
    end
    
    PUSH --> CHECKOUT
    CHECKOUT --> DEPS
    DEPS --> BUILD
    BUILD --> PRISMA
    PRISMA --> PACKAGE
    PACKAGE --> UPLOAD
    UPLOAD --> SCP
    SCP --> EXTRACT
    EXTRACT --> REBUILD
    REBUILD --> MIGRATE
    MIGRATE --> RESTART
    RESTART --> HEALTH_CHECK
    HEALTH_CHECK --> SMOKE_TEST
    SMOKE_TEST --> NOTIFY
    
    style BUILD fill:#2196F3,stroke:#fff,color:#fff
    style RESTART fill:#4CAF50,stroke:#fff,color:#fff
    style HEALTH_CHECK fill:#FF9800,stroke:#fff,color:#fff

Deployment Features

Build & Deployment

Builds on GitHub Actions (not on EC2 micro instance)
Artifact-based deployment via SCP
Native module rebuild for Alpine Linux compatibility (bcrypt)
Automated database migrations with Prisma
Zero-downtime deployment with health checks

Container Strategy

Multi-stage Docker builds for optimized images
Separate containers for app, database, cache, and monitoring
Persistent volumes for data and metrics
Health checks for all services
Automatic restart policies

Monitoring Persistence

Prometheus and Grafana remain running during app deployments
Automatic reconnection after app container restarts
No metrics data loss during deployments
Independent update cycles for monitoring stack

Security

Security Stack

SSL/TLS: HTTPS enforcement with CloudFlare
JWT Authentication: Secure token-based auth with refresh rotation
RBAC: Role-based access control with granular permissions
Helmet.js: Security headers
Input Validation: express-validator for all inputs
Rate Limiting: Configurable limits per endpoint type
Audit Logging: Complete user activity tracking
Secrets Management: Environment-based configuration
Monitoring Auth: X-API-Secret protection for /metrics endpoint

User Roles & Permissions

Role Hierarchy

GRAND_PATHFINDER: System administrator with full access
CHIEF_PATHFINDER: Administrative role with management capabilities
PATHFINDER: Educator/mentor with content creation rights
PIONEER: Student/learner with progress tracking
LUMINARY: Distinguished achievement role

Permission Matrix

Resource               | PIONEER | PATHFINDER | CHIEF_PATHFINDER | GRAND_PATHFINDER
--------------------- |---------|------------|------------------|------------------
View Content          |    ✓    |     ✓      |        ✓         |        ✓
Submit Assignments    |    ✓    |     ✓      |        ✓         |        ✓
Create Content        |    ✗    |     ✓      |        ✓         |        ✓
Manage Users          |    ✗    |     ✗      |        ✓         |        ✓
System Administration |    ✗    |     ✗      |        ✗         |        ✓
View Metrics          |    ✗    |     ✗      |        ✓         |        ✓

API Overview

Core Endpoints

# System Health & Status
GET  /health                         # Application health check
GET  /api/status/public              # Public system status
GET  /api/status/components          # Component status details
GET  /metrics                        # Prometheus metrics (protected)

# Authentication & User Management
POST /api/auth/register              # User registration
POST /api/auth/login                 # User authentication
POST /api/auth/refresh               # Token refresh
POST /api/auth/logout                # User logout
POST /api/auth/password-reset        # Password reset request
POST /api/auth/password-reset/verify # Verify OTP and reset password

# Learning Content Management
GET  /api/cohorts                    # List all cohorts
GET  /api/leagues                    # List specialization leagues
GET  /api/weeks                      # List weekly content
GET  /api/sections                   # List section content
GET  /api/resources                  # List learning resources

# Progress Tracking & Analytics
GET  /api/progress                   # User progress overview
POST /api/progress/section           # Mark section completed
POST /api/progress/resource          # Mark resource completed
GET  /api/analytics/counts           # Platform analytics
GET  /api/leaderboard               # Competition leaderboards

# Assignment System
GET  /api/assignments                # List assignments
POST /api/assignments/submit         # Submit assignment solution
GET  /api/assignments/:id/submissions # View submissions (educators)

# Gamification Features
GET  /api/badges                     # Available badges
GET  /api/badges/user                # User's earned badges
POST /api/social/share               # Share achievement

For complete API documentation, see docs/api/

Quick Start

Prerequisites

Node.js 18.0.0+ and npm 9.0.0+
PostgreSQL 15.0+ database
Redis 7.0+ server (optional for local development)
Docker 24.0+ & Docker Compose 2.20+ (recommended)

Local Development

# Clone the repository
git clone https://github.com/openlearnnitj/openlearn-backend.git
cd openlearn-backend

# Install dependencies
npm install

# Set up environment variables
cp .env.example .env
# Edit .env with your configuration

# Start PostgreSQL and Redis with Docker
docker compose up -d postgres redis

# Run database migrations
npx prisma migrate dev

# Seed the database (optional)
npm run db:seed:dev

# Start development server
npm run dev

# Server will be running at http://localhost:3000

Production Deployment

Live Infrastructure

Frontend: openlearn.org.in
API: api.openlearn.org.in
Monitoring: ops.openlearn.org.in - (Restricted)
Status: api.openlearn.org.in/status-page
Health: api.openlearn.org.in/health

Deployment Process

Automated via GitHub Actions:

Push to main branch triggers CI/CD pipeline
Build runs on GitHub Actions runner
Creates deployment artifact (tarball)
Transfers artifact to EC2 via SCP
Extracts and rebuilds native modules (bcrypt for Alpine)
Runs database migrations
Restarts app container only
Prometheus/Grafana continue running
Health checks verify deployment

Manual deployment (if needed):

# SSH to EC2 server
ssh ubuntu@your-ec2-ip

# Navigate to project directory
cd /home/ubuntu/openlearn-backend

# Pull latest changes (or use artifact)
# Already done by GitHub Actions

# Run migrations
docker compose exec app npx prisma migrate deploy

# Restart app container
docker compose restart app

# Check health
curl http://localhost:3000/health

Post-Deployment

# Verify all containers
docker compose ps

# Check logs
docker compose logs -f app

# View metrics
# Visit https://ops.openlearn.org.in

# Check Prometheus targets
# Internal only: curl http://localhost:9090/targets

Contributing

We welcome contributions! Please read our Contributing Guidelines and Code of Conduct before getting started.

Here's how to contribute:

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes following our coding standards
Test thoroughly
Commit (git commit -m 'Add amazing feature')
Push (git push origin feature/amazing-feature)
Open a Pull Request

Support & Resources

Getting Help

Documentation: Comprehensive guides in /docs
Issues: GitHub Issues

Project Links

Repository: github.com/openlearnnitj/openlearn-backend
Changelog: CHANGELOG.md
Architecture: docs/architecture/
API Docs: docs/api/
Monitoring Docs: docs/monitoring/

Built with TypeScript, Express.js, Prisma, and PostgreSQL

Monitored with Prometheus & Grafana

Empowering education through technology

OpenLearn Platform • Production Ready • Open Source

License

This project is licensed under the MIT License - see the LICENSE file for details.

Name		Name	Last commit message	Last commit date
Latest commit History 182 Commits
.github		.github
deployment		deployment
docs		docs
environments		environments
grafana/provisioning/datasources		grafana/provisioning/datasources
prisma		prisma
public		public
scripts		scripts
src		src
.dockerignore		.dockerignore
.env.docker		.env.docker
.env.example		.env.example
.env.micro		.env.micro
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile.production		Dockerfile.production
LICENSE		LICENSE
README.md		README.md
debug-email-flow.sh		debug-email-flow.sh
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.yml		docker-compose.yml
docker-entrypoint.sh		docker-entrypoint.sh
package-lock.json		package-lock.json
package.json		package.json
prometheus.yml		prometheus.yml
tsconfig.json		tsconfig.json

License

openlearnnitj/openlearn-backend

Folders and files

Latest commit

History

Repository files navigation

OpenLearn Backend

Technology Stack

Live Services

Documentation

Quick Access

Core API References

Architecture & Operations

Platform Features

Technical Architecture

System Overview

Database Schema (Core Models)

Monitoring & Observability

Monitoring Architecture

Metrics Collected

Access Monitoring

Deployment Architecture

Production Infrastructure

CI/CD Pipeline

Deployment Features

Security

Security Stack

User Roles & Permissions

API Overview

Core Endpoints

Quick Start

Prerequisites

Local Development

Production Deployment

Live Infrastructure

Deployment Process

Post-Deployment

Contributing

Support & Resources

Getting Help

Project Links

License

About

Resources

License

Code of conduct

Contributing

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 2

Languages

Packages