HyperDX NestJS POC

A proof-of-concept project demonstrating how to integrate HyperDX observability platform with a NestJS application. This project showcases distributed tracing and centralized logging using OpenTelemetry and Pino.

Features

• 🔍 Distributed Tracing - Automatic trace collection using OpenTelemetry
• 📊 Centralized Logging - Logs sent to HyperDX with automatic trace correlation
• 🎯 Trace Context Injection - Automatic injection of trace IDs, span IDs, and trace flags into logs
• 🚀 Dual Output - Logs appear both in console (pretty-printed) and HyperDX
• 📦 Reusable Logger Module - Clean, injectable logger service with intuitive API
• 🏷️ Tagging Support - Optional tags for log categorization and filtering

Prerequisites

• Node.js (v18 or higher)
• npm or yarn
• HyperDX account and API key (optional - works without it for local development)

Installation

# Install dependencies
npm install

Configuration

Create a .env file in the root directory with the following variables:

# HyperDX Configuration
HYPERDX_API_KEY=your-hyperdx-api-key-here
HYPERDX_SERVICE_NAME=your-service-name
# Optional: For self-hosted HyperDX instances
# HYPERDX_URL=http://your-hyperdx-instance:8080

# Application Configuration
PORT=3000

Note:

• If HYPERDX_API_KEY is not provided, the application will still run but logs will only be sent to the console (not to HyperDX).
• For self-hosted HyperDX, set HYPERDX_URL to point to your self-hosted instance (e.g., http://localhost:8080).

Running the Application

# Development mode with hot-reload
npm run start:dev

# Production mode
npm run build
npm run start:prod

# Debug mode
npm run start:debug

The application will start on http://localhost:3000 (or the port specified in PORT env variable).

Usage

Using the Logger Service

The AppLoggerService is available for injection throughout your application. The API uses a clean, intuitive pattern where the message is the first positional argument:

import { Injectable } from '@nestjs/common';
import { AppLoggerService } from './shared/app-logger/app-logger.service';

@Injectable()
export class YourService {
  constructor(private readonly logger: AppLoggerService) {}

  someMethod() {
    // Simple log with just a message
    this.logger.info('Something happened');

    // Log with additional data
    this.logger.info('User action', {
      data: { userId: '123', action: 'login' },
    });

    // Log with a tag for categorization
    this.logger.info('Processing payment', {
      tag: 'payment',
      data: { amount: 100, currency: 'USD' },
    });

    // Error logging
    this.logger.error('An error occurred', {
      data: { error: 'Something went wrong', code: 500 },
    });

    // Different log levels
    this.logger.debug('Debug information', { data: { step: 1 } });
    this.logger.warn('Warning message', { tag: 'deprecated' });
  }
}

API Reference

All logger methods follow the same pattern:

logger.info(message: string, options?: { data?: Record<string, any>; tag?: string })
logger.warn(message: string, options?: { data?: Record<string, any>; tag?: string })
logger.error(message: string, options?: { data?: Record<string, any>; tag?: string })
logger.debug(message: string, options?: { data?: Record<string, any>; tag?: string })

Parameters:

• message (required): The log message as a string
• options (optional): An object containing:
  • data (optional): Additional structured data to include in the log
  • tag (optional): A single tag string for categorizing/filtering logs

Example Endpoints

The project includes example endpoints in AppController:

• POST /create-post - Demonstrates logging throughout an async operation (creating a post via external API)
• POST /error - Demonstrates error-level logging

Test them with:

curl -X POST http://localhost:3000/create-post
curl -X POST http://localhost:3000/error

How It Works

Architecture

1. OpenTelemetry Integration: HyperDX SDK initializes OpenTelemetry instrumentation for automatic trace collection
2. Pino Logger: High-performance structured logger with multiple transport targets
3. Dual Transport:
   • pino-pretty - Pretty-printed console output for local development
   • @hyperdx/node-logger - HyperDX transport for remote log aggregation
4. Trace Context Mixin: Automatically injects OpenTelemetry trace context (trace_id, span_id, trace_flags) into every log entry

Log Flow

Application Code
    ↓
AppLoggerService.info/warn/error/debug()
    ↓
Pino Logger
    ↓
    ├─→ pino-pretty (Console)
    └─→ HyperDX Transport → HyperDX Platform

Trace Correlation

Every log entry automatically includes:

• trace_id - Unique identifier for the entire trace
• span_id - Unique identifier for the current span
• trace_flags - Trace sampling flags

This allows HyperDX to correlate logs with distributed traces automatically, making it easy to debug issues across microservices.

Tagging

Tags are useful for:

• Categorization: Group related logs (e.g., payment, auth, api)
• Filtering: Quickly find logs in HyperDX by tag
• Organization: Structure logs by feature or module

Tags are included as a top-level field in the log entry, making them easily searchable in HyperDX.

Project Structure

src/
├── app.controller.ts          # Example controller with logging
├── app.module.ts              # Root application module
├── main.ts                    # Application entry point
└── shared/
    └── app-logger/
        ├── app-logger.module.ts    # Logger module
        └── app-logger.service.ts   # Logger service implementation

Available Scripts

# Development
npm run start:dev      # Start in watch mode
npm run start:debug    # Start in debug mode

# Production
npm run build          # Build the project
npm run start:prod     # Start production server

# Code Quality
npm run lint           # Run ESLint
npm run format         # Format code with Prettier

# Testing
npm run test           # Run unit tests
npm run test:watch     # Run tests in watch mode
npm run test:cov       # Run tests with coverage
npm run test:e2e       # Run end-to-end tests

Dependencies

Core

• @nestjs/common - NestJS core framework
• @nestjs/config - Configuration management
• pino - High-performance logger
• pino-pretty - Pretty console output

Observability

• @hyperdx/node-opentelemetry - HyperDX OpenTelemetry SDK
• @hyperdx/node-logger - HyperDX Pino transport
• @opentelemetry/api - OpenTelemetry API

Troubleshooting

Logs not appearing in HyperDX

1. Verify HYPERDX_API_KEY is set correctly in your .env file
2. Check that HYPERDX_SERVICE_NAME is configured
3. Ensure network connectivity to HyperDX endpoints
4. Check console for initialization errors (look for "Failed to initialize HyperDX" messages)

Trace context not appearing in logs

• Ensure you're within an active OpenTelemetry span
• Verify HyperDX SDK initialization succeeded (check console on startup)
• Check that the mixin function is working correctly

Logs not showing in console

• Verify pino-pretty is installed
• Check that the logger is being called (add a simple console.log to verify)
• Ensure the log level matches your configuration

Best Practices

1. Use descriptive messages: Make log messages clear and actionable

   // Good
   this.logger.info('User login successful', { data: { userId: '123' } });
   
   // Avoid
   this.logger.info('OK', { data: { id: '123' } });

2. Include relevant context: Use the data field for structured information

   this.logger.error('Payment processing failed', {
     data: { 
       userId: '123', 
       amount: 100, 
       errorCode: 'INSUFFICIENT_FUNDS' 
     },
   });

3. Use tags consistently: Establish a tagging convention for your team

   // Examples: 'payment', 'auth', 'api', 'database', 'cache'
   this.logger.info('Processing payment', { tag: 'payment' });

4. Log at appropriate levels: Use debug for development, info for normal flow, warn for potential issues, error for failures

Plain Node.js Integration with Pino

For a basic Node.js application (without NestJS), here's how to integrate Pino:

Installation

npm install pino pino-pretty

Basic Setup

const pino = require('pino');

// Simple logger
const logger = pino({
  level: 'info',
  transport: {
    target: 'pino-pretty',
    options: {
      colorize: true,
    },
  },
});

// Usage
logger.info('Hello world');
logger.error({ err: new Error('Something went wrong') }, 'Error occurred');
logger.info({ userId: '123' }, 'User logged in');

With HyperDX

Installation:

npm install pino pino-pretty @hyperdx/node-opentelemetry @hyperdx/node-logger

Setup:

const pino = require('pino');
const HyperDX = require('@hyperdx/node-opentelemetry');

// Initialize HyperDX
HyperDX.init({
  apiKey: process.env.HYPERDX_API_KEY,
  service: process.env.HYPERDX_SERVICE_NAME,
});

// Logger with HyperDX transport
const logger = pino({
  level: 'info',
  transport: {
    targets: [
      {
        target: 'pino-pretty',
        options: { colorize: true },
      },
      {
        target: '@hyperdx/node-logger/build/src/pino',
        options: {
          apiKey: process.env.HYPERDX_API_KEY,
          service: process.env.HYPERDX_SERVICE_NAME,
        },
      },
    ],
  },
});

// Usage
logger.info('Application started');
logger.error({ error: 'Failed to connect' }, 'Database connection failed');

Self-Hosted HyperDX

For self-hosted HyperDX instances, you need to configure a custom endpoint URL.

Installation:

npm install pino pino-pretty @hyperdx/node-opentelemetry @hyperdx/node-logger

Setup:

const pino = require('pino');
const HyperDX = require('@hyperdx/node-opentelemetry');

// Initialize HyperDX with custom endpoint
HyperDX.init({
  apiKey: process.env.HYPERDX_API_KEY,
  service: process.env.HYPERDX_SERVICE_NAME,
  // Self-hosted endpoint configuration
  url: process.env.HYPERDX_URL || 'http://localhost:8080', // Your self-hosted HyperDX URL
});

// Logger with self-hosted HyperDX transport
const logger = pino({
  level: 'info',
  transport: {
    targets: [
      {
        target: 'pino-pretty',
        options: { colorize: true },
      },
      {
        target: '@hyperdx/node-logger/build/src/pino',
        options: {
          apiKey: process.env.HYPERDX_API_KEY,
          service: process.env.HYPERDX_SERVICE_NAME,
          url: process.env.HYPERDX_URL || 'http://localhost:8080', // Self-hosted endpoint
        },
      },
    ],
  },
});

Environment Variables for Self-Hosted:

HYPERDX_API_KEY=your-api-key
HYPERDX_SERVICE_NAME=your-service-name
HYPERDX_URL=http://your-hyperdx-instance:8080  # Self-hosted HyperDX URL

Note: For self-hosted HyperDX, you typically run it via Docker:

docker run -p 8080:8080 -p 8123:8123 -p 4318:4318 -p 4317:4317 \
  docker.hyperdx.io/hyperdx/hyperdx-local:2-beta

Then point your application to http://localhost:8080 (or your self-hosted domain).