### Streaming in Node.
js
Streams in Node.js provide a powerful way to handle continuous data flow,
especially when working with large files or real-time data.
They are used for reading or writing sequential data in chunks, making them memory
efficient.
#### Types of Streams
1. **Readable Streams**
- Data flows from a source (e.g., file, network) to the application.
- Example: `fs.createReadStream`
2. **Writable Streams**
- Data flows from the application to a destination (e.g., file, network).
- Example: `fs.createWriteStream`
3. **Duplex Streams**
- Allows both reading and writing (e.g., a TCP socket).
- Example: `net.Socket`
4. **Transform Streams**
- A type of duplex stream that can modify data as it is read or written (e.g.,
`zlib` for compression).
- Example: `zlib.createGzip`
#### Key Methods and Events
- **Methods**:
- `read()`: Reads data from a readable stream.
- `write(chunk)`: Writes data to a writable stream.
- `pipe(dest)`: Pipes data from one stream to another.
- **Events**:
- `data`: Emitted when a chunk of data is available.
- `end`: Emitted when no more data is available.
- `error`: Emitted when an error occurs.
- `finish`: Emitted when all data is flushed to the writable stream.
#### Example 1: Reading a File Stream
```javascript
const fs = require('fs');
const readStream = fs.createReadStream('largeFile.txt', { encoding: 'utf8' });
readStream.on('data', chunk => {
console.log('Received chunk:', chunk);
});
readStream.on('end', () => {
console.log('No more data.');
});
readStream.on('error', err => {
console.error('Error:', err);
});
```
#### Example 2: Writing to a File Stream
```javascript
const fs = require('fs');
const writeStream = fs.createWriteStream('output.txt');
writeStream.write('Hello, World!\n');
writeStream.end('Finished writing.');
writeStream.on('finish', () => console.log('Write complete.'));
```
#### Example 3: Piping Streams
```javascript
const fs = require('fs');
const readStream = fs.createReadStream('input.txt');
const writeStream = fs.createWriteStream('output.txt');
readStream.pipe(writeStream);
readStream.on('end', () => console.log('File copied successfully.'));
```
#### Example 4: Transform Stream (Compression)
```javascript
const fs = require('fs');
const zlib = require('zlib');
const readStream = fs.createReadStream('input.txt');
const gzipStream = zlib.createGzip();
const writeStream = fs.createWriteStream('output.txt.gz');
readStream.pipe(gzipStream).pipe(writeStream);
```
---
### Cluster in Node.js
The **Cluster** module allows Node.js to create child processes (workers) that
share the same server port, enabling you to utilize multi-core CPUs for better
scalability.
#### Key Concepts
1. **Master Process**: The main process that controls worker processes.
2. **Worker Processes**: Child processes created by the master to handle tasks.
3. **IPC (Inter-Process Communication)**: Workers communicate with the master using
messages.
#### How Clustering Works
- The master process distributes incoming connections or tasks to worker processes.
- Each worker runs on its own thread and handles incoming requests independently.
#### Basic Cluster Example
```javascript
const cluster = require('cluster');
const http = require('http');
const os = require('os');
if (cluster.isMaster) {
// Fork workers for each CPU core
const numCPUs = os.cpus().length;
console.log(`Master process is running (PID: ${process.pid})`);
for (let i = 0; i < numCPUs; i++) {
cluster.fork();
}
// Handle worker exit
cluster.on('exit', (worker, code, signal) => {
console.log(`Worker ${worker.process.pid} died`);
cluster.fork(); // Restart a new worker
});
} else {
// Workers can share a TCP connection
http.createServer((req, res) => {
res.writeHead(200);
res.end(`Hello from worker ${process.pid}`);
}).listen(8000);
console.log(`Worker process started (PID: ${process.pid})`);
}
```
#### Message Passing Between Master and Workers
Workers can send and receive messages from the master process using `process.send`
and `cluster.on`.
```javascript
// Master Process
if (cluster.isMaster) {
const worker = cluster.fork();
worker.on('message', message => {
console.log('Message from worker:', message);
});
worker.send('Hello Worker!');
} else {
// Worker Process
process.on('message', message => {
console.log('Message from master:', message);
process.send('Hello Master!');
});
}
```
#### Advantages of Clustering
1. **Utilizes Multi-Core CPUs**: Distributes the load across multiple cores.
2. **Improves Performance**: Each worker handles separate tasks.
3. **Resilience**: Restart workers on failure.
#### Limitations
- Workers do not share memory directly. Use shared databases, caches, or message
brokers for state sharing.
- Clustering doesn't inherently solve CPU-intensive task bottlenecks.
---
These concepts of **streaming** and **clustering** are crucial for building high-
performance, scalable Node.js applications.
