Skip to content

About

Production-grade Kafka transport for NestJS microservices built on @platformatic/kafka: drop-in replacement for the kafkajs transport with request-reply, retries, dead-letter topics and batches.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

3 stars

Watchers

0 watching

Forks

Repository files navigation

nestjs-kafka-transport: Kafka for NestJS microservices, without kafkajs

nestjs-kafka-transport

Kafka for NestJS microservices, without kafkajs.

npm CI coverage node license

A production-grade Kafka transport for @nestjs/microservices, built on @platformatic/kafka (a modern, pure-JavaScript client maintained by Node.js core contributors) instead of the unmaintained kafkajs.

It is a drop-in replacement for the built-in Kafka transport: same decorators, same KafkaContext, same request-reply headers and reply topics. A service on this transport can talk to a service still on the kafkajs transport, so you can migrate one service at a time.

Why

  • kafkajs has had no maintainer activity for years (nestjs/nest#13223); NestJS core keeps it for backwards compatibility.
  • The alternatives either wrap a client without being a real transport (no @MessagePattern request-reply) or are copy-paste snippets without reconnection, typing or tests.
  • Teams running Kafka in production keep re-implementing the same things: retries with backoff, dead-letter topics, batch consumption, manual commits, health checks, metrics.

Features

built-in kafkajs transport nestjs-kafka-transport
@MessagePattern request-reply, @EventPattern events ✅ ✅ wire-compatible
KafkaContext (getMessage, getTopic, getPartition, getConsumer, getProducer) ✅ ✅
KafkaRetriableException redelivery ✅ ✅ with exponential backoff
Interoperates with services on the kafkajs transport (tested both directions) – ✅
Numbers, booleans and null keep their type end to end ❌ (strings) ✅
Lost-reply detection (KafkaReplyLostError) and operation ids for idempotent retries ❌ (timeouts only) ✅
Custom serializers / deserializers, Schema Registry passthrough ✅ ✅
Maintained client, Node 22/24, Kafka 3.x and 4.x (KRaft) ❌ ✅
Typed options, no any in the public API partial ✅
Retries with backoff, dead-letter topics (kafka_dlt-* headers) ❌ ✅
Per-record or manual commits (ctx.commit()), at-least-once semantics partial ✅
Terminus health indicator (transport status, broker probe) ❌ ✅
Batch consumption with per-partition ordering ❌ v0.3
Prometheus metrics, OpenTelemetry spans ❌ v0.3

Install

npm install nestjs-kafka-transport @platformatic/kafka
# peer dependencies: @nestjs/common @nestjs/core @nestjs/microservices rxjs

Requires Node.js ≥ 22.22 (or ≥ 24.6), the minimum of @platformatic/kafka.

Quick start

Server (a microservice or a hybrid HTTP app):

import { NestFactory } from '@nestjs/core';
import { KafkaTransportServer } from 'nestjs-kafka-transport';

const app = await NestFactory.createMicroservice(AppModule, {
  strategy: new KafkaTransportServer({
    client: { clientId: 'orders', bootstrapBrokers: ['localhost:9092'] },
    consumer: { groupId: 'orders-consumer' },
  }),
});
await app.listen();

Handlers are the ones you already have:

@Controller()
export class OrdersController {
  @MessagePattern('order.total') // request-reply
  total(@Payload() order: Order, @Ctx() ctx: KafkaContext) {
    return order.items.reduce((sum, i) => sum + i.price, 0);
  }

  @EventPattern('order.created') // fire-and-forget
  created(@Payload() order: Order) {
    /* ... */
  }
}

Client:

import { KafkaTransportClient } from 'nestjs-kafka-transport';

const client = new KafkaTransportClient({
  client: { clientId: 'gateway', bootstrapBrokers: ['localhost:9092'] },
  consumer: { groupId: 'gateway-consumer' },
});
client.subscribeToResponseOf('order.total');
await client.connect();

const total = await firstValueFrom(client.send<number>('order.total', order));
client.emit('order.created', order);

Migrating from the built-in transport

Replace transport: Transport.KAFKA, options: {...} with strategy: new KafkaTransportServer({...}) and ClientKafka with KafkaTransportClient. Options map one-to-one; the migration guide lists every kafkajs option and its equivalent. Because reply topics (<pattern>.reply) and headers (kafka_correlationId, kafka_replyTopic, kafka_replyPartition, kafka_nest-err) are unchanged, mixed deployments work during the migration.

Documentation

Status

Pre-1.0. The v0.1 line covers the built-in transport's feature set; see the roadmap. Feedback from production users is what drives priorities: open an issue or a discussion.

Contributing

See CONTRIBUTING.md. Every change ships with a test; e2e tests run against a real broker in Docker.

License

MIT © Fadi Romdhan

About

Production-grade Kafka transport for NestJS microservices built on @platformatic/kafka: drop-in replacement for the kafkajs transport with request-reply, retries, dead-letter topics and batches.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages