WebRTC Basic implementation, advance discussion 3 of 9
Why WebRTC?
WebRTC is the core/only protocol that lets you do real time media
communication from inside a browser.
We already did this fairly well in a live stream
[Link]
[Link]
You use WebRTC for applications that require sub second latency.
Examples include
1. Zoom/Google meet (Multi party call)
2. Omegle, teaching (1:1 call)
3. 30FPS games (WebRTC can also send data)
WebRTC Basic implementation, advance discussion 3 of 9
WebRTC
Architecture/jargon
P2P
WebRTC is a peer to peer protocol. This means the you directly
send your media over to the other person without the need of a
central server
💡You do need a central server for signalling and sometimes
for sending media as well (turn). We’ll be discussing this later
Signalling server
Both theWebRTC Basic
browsers implementation,
need to exchange advance
their discussion 3 of 9
address before they can
start talking to each other. A signaling server is used for that.
It is usually a websocket server but can be anything (http)
Stun (Session Traversal Utilities for NAT)
It gives you back your publically accessable IPs. It shows you how
the world sees you
Check
[Link]
kle-ice/
WebRTC Basic implementation, advance discussion 3 of 9
Ice candidates
ICE (Interactive Connectivity Establishment) candidates are
potential networking endpoints that WebRTC uses to establish a
connection between peers. Each candidate represents a possible
method for two devices (peers) to communicate, usually in the
context of real-time applications like video calls, voice calls, or
peer-to-peer data sharing.
If two friends are trying to connect to each other in a hostel wifi ,
then they can connect
WebRTC via their private
Basic implementation, router
advance ice candidates.
discussion 3 of 9
If two people from different countries are trying to connect to each
other, then they would connect via their public IPs.
Turn server
A lot of times, your network doesn’t allow media to come in from
browser 2 . This depends on how restrictive your network is
Since the ice candidate is discovered by the stun server , your
network might block incoming data from browser 2 and only allow
it from the stun server
Offer
The process of the first browser (the one initiating connection)
sending their ice candidates to the other side.
Answer
The other side returning their ice candidates is called the answer.
SDP - Session description protocol
A single file that contains all your
1. ice candidates
2. what WebRTC Basicwant
media you implementation, advance
to send, what discussion
protocols 3 of 9
you’ve used to
encode the media
This is the file that is sent in the offer and received in the answer
Example
v=0
o=- 423904492236154649 2 IN IP4 [Link]
s=-
t=0 0
m=audio 49170 RTP/AVP 0
c=IN IP4 [Link]
a=rtpmap:0 PCMU/8000
a=ice-options:trickle
a=candidate:1 1 UDP 2122260223 [Link] 49170 typ host
a=candidate:2 1 UDP 2122194687 [Link] 49171 typ host
a=candidate:3 1 UDP 1685987071 [Link] 49172 typ srflx raddr 10.0
a=candidate:4 1 UDP 41819902 [Link] 3478 typ relay raddr [Link]
RTCPeerConnection (pc, peer connection)
[Link]
US/docs/Web/API/RTCPeerConnection
This is a class that the browser provides you with which gives you
access to the sdp , lets you create answers / offers , lets you send
media.
This class hides all the complexity of webrtc from the developer
Summary
You need a signaling server , stun server to initiate the webrtc conn
b/w the parties. You can
WebRTC Basic kill these once
implementation, thediscussion
advance conn is made.
3 of 9
You need to include a turn server incase any of the users are on a
restrictive network so you can get back a turn ice candidate as
well.
Connecting the two sides
The steps to create a webrtc connection between 2 sides includes
-
1. Browser 1 creates an RTCPeerConnection
2. Browser 1 creates an offer
3. Browser 1 sets the local description to the offer
4. Browser 1 sends the offer to the other side through the signaling
server
5. Browser 2 receives the offer from the signaling server
6. Browser 2 sets the remote description to the offer
7. Browser 2 creates an answer
8. Browser 2 sets the local description to be the answer
9. Browser 2 sends the answer to the other side through the
signaling server
10. Browser 1 receives the answer and sets the remote description
This is just to establish the p2p connection b/w the two parties
To actually send media, we have to
1. Ask for camera /mic permissions
WebRTC
2. Get the audioBasic
and implementation,
video streams advance discussion 3 of 9
3. Call addTrack on the pc
4. This would trigger a onTrack callback on the other side
Implementation
We will be writing the code in
1. [Link] for the Signaling server. It will be a websocket server that
supports 3 types of messages
1. createOffer
2. createAnswer
3. addIceCandidate
2. React + PeerConnectionObject on the frontend
We’re actually building a slightly complex version of
[Link]
WebRTC Basic implementation, advance discussion 3 of 9
Backend
Create an empty TS project, add ws to it
npm init -y
npx tsc --init
npm install ws @types/ws
Change rootDir and outDir in tsconfig
"rootDir": "./src",
"outDir": "./dist",
Create a simple websocket server
import { WebSocketServer } from 'ws';
const wss = new WebSocketServer({ port: 8080 });
let senderSocket: null | WebSocket = null;
let receiverSocket: null | WebSocket = null;
WebRTC Basic implementation,
[Link]('connection', advance discussion
function connection(ws) {
3 of 9
[Link]('error', [Link]);
[Link]('message', function message(data: any) {
const message = [Link](data);
});
[Link]('something');
});
Try running the server
tsc -b
node dist/[Link]
Add message handlers
import { WebSocket, WebSocketServer } from 'ws';
const wss = new WebSocketServer({ port: 8080 });
let senderSocket: null | WebSocket = null;
let receiverSocket: null | WebSocket = null;
[Link]('connection', function connection(ws) {
[Link]('error', [Link]);
[Link]('message', function message(data: any) {
const message = [Link](data);
if ([Link] === 'sender') {
senderSocket = ws;
} else if ([Link] === 'receiver') {
receiverSocket = ws;
} else if ([Link] === 'createOffer') {
if (ws !== senderSocket) {
return;
}
WebRTC Basic implementation, advance
receiverSocket?.send([Link]({ discussion
type:
3 of 9
'createOffer', sdp: mes
} else if ([Link] === 'createAnswer') {
if (ws !== receiverSocket) {
return;
}
senderSocket?.send([Link]({ type: 'createAnswer', sdp: m
} else if ([Link] === 'iceCandidate') {
if (ws === senderSocket) {
receiverSocket?.send([Link]({ type: 'iceCandidate', candi
} else if (ws === receiverSocket) {
senderSocket?.send([Link]({ type: 'iceCandidate', candid
}
}
});
});
That is all that you need for a simple one way communication b/w
two tabs
To have both the sides be able to send each other media, and
support multiple rooms, see [Link]
Frontend
Create a frontend repo
WebRTC Basic implementation, advance discussion 3 of 9
npm create vite@latest
Add two routes, one for a sender and one for a receiver
import { useState } from 'react'
import './[Link]'
import { Route, BrowserRouter, Routes } from 'react-router-dom'
import { Sender } from './components/Sender'
import { Receiver } from './components/Receiver'
function App() {
return (
<BrowserRouter>
<Routes>
<Route path="/sender" element={<Sender />} />
<Route path="/receiver" element={<Receiver />} />
</Routes>
</BrowserRouter>
)
}
export default App
Remove strict mode in [Link] to get rid of a bunch of webrtc
connections locally (not really needed)
Create components for sender
import { useEffect, useState } from "react"
export const Sender = () => {
const [socket, setSocket] = useState<WebSocket | null>(null);
const [pc, setPC] = useState<RTCPeerConnection | null>(null);
useEffect(() => {
const socket = new WebSocket('[Link]
setSocket(socket);
[Link] = () => {
WebRTC Basic implementation, advance discussion 3 of 9
[Link]([Link]({
type: 'sender'
}));
}
}, []);
const initiateConn = async () => {
if (!socket) {
alert("Socket not found");
return;
}
[Link] = async (event) => {
const message = [Link]([Link]);
if ([Link] === 'createAnswer') {
await [Link]([Link]);
} else if ([Link] === 'iceCandidate') {
[Link]([Link]);
}
}
const pc = new RTCPeerConnection();
setPC(pc);
[Link] = (event) => {
if ([Link]) {
socket?.send([Link]({
type: 'iceCandidate',
candidate: [Link]
}));
}
}
[Link] = async () => {
const offer = await [Link]();
await [Link](offer);
socket?.send([Link]({
type: 'createOffer',
WebRTC
sdp: Basic implementation, advance discussion 3 of 9
[Link]
}));
}
getCameraStreamAndSend(pc);
}
const getCameraStreamAndSend = (pc: RTCPeerConnection) => {
[Link]({ video: true }).then((stre
const video = [Link]('video');
[Link] = stream;
[Link]();
// this is wrong, should propogate via a component
[Link](video);
[Link]().forEach((track) => {
pc?.addTrack(track);
});
});
}
return <div>
Sender
<button onClick={initiateConn}> Send data </button>
</div>
}
Create the component for receiver
import { useEffect } from "react"
export const Receiver = () => {
useEffect(() => {
const socket = new WebSocket('[Link]
[Link] = () => {
[Link]([Link]({
type: 'receiver'
WebRTC
})); Basic implementation, advance discussion 3 of 9
}
startReceiving(socket);
}, []);
function startReceiving(socket: WebSocket) {
const video = [Link]('video');
[Link](video);
const pc = new RTCPeerConnection();
[Link] = (event) => {
[Link] = new MediaStream([[Link]]);
[Link]();
}
[Link] = (event) => {
const message = [Link]([Link]);
if ([Link] === 'createOffer') {
[Link]([Link]).then(() => {
[Link]().then((answer) => {
[Link](answer);
[Link]([Link]({
type: 'createAnswer',
sdp: answer
}));
});
});
} else if ([Link] === 'iceCandidate') {
[Link]([Link]);
}
}
}
return <div>
</div>
}
WebRTC Basic implementation, advance discussion 3 of 9
Final code - [Link]
webrtc
Assignment
Can you change the code so that
1. A single producer can produce to multiple people?
2. Add room logic.
3. Add two way communication.
4. Replace p2p logic with an SFU (mediasoup)
Webrtc stats
You can look at a bunch of stats/sdps in about:webrtc-internals
A lot of times you ask users to dump stats from here for better
debugging
WebRTC Basic implementation, advance discussion 3 of 9
Using libraries for p2p
As you can see, there is a lot of things we had to know to be able to
build a simple app that sends video from one side to another
There are libraries that hide a lot of this complexity (specifically the
complexity of the RTCPeerConnectionObject from you).
[Link]
WebRTC Basic implementation, advance discussion 3 of 9
Other architectures
There are two other popular architectures for doing WebRTC
1. SFU
2. MCU
Problems with p2p
WebRTC
Doesn’t scale Basic
well implementation,
beyond 3-4 peopleadvance discussion
in the 3 of 9
same call
SFU
SFU stands for Selective forwarding unit . This acts as a central media
server which forwards packets b/w users
Popular Open source SFUs -
1. [Link]
2. [Link] (not exactly an SFU but you can
build one on top of it)
MCU
It mixes audio/video together on the server before forwarding it.
This means it needs to
1. decode video/audio (using something like ffmpeg)
2. Mix them (create a video canvas/create a single audio stream)
3. Send out the merged audio stream to everyone
WebRTC Basic implementation, advance discussion 3 of 9
