Protocols and Networking

It is essential for Cyber Security Professionals to have a solid understanding of

how computers communicate. There is much more happening behind the scenes
of computer networks than what can be observed when using applications.

The OSI Model

The OSI ("Open Systems Interconnection") model represents an easy and
intuitive way to standardize the different parts required to communicate across
networks.
The model makes it clear what is required to communicate on a network by
splitting the requirements into multiple layers.
This is what the OSI Model looks like:
Layer What it does

7 - Application Where humans process data and information

6 - Presentation Ensures data is in a usable format

5 - Session Capable of maintaining connections

4 - Transport Data is forwarded to a service capable of handling

requests

3 - Network Layer Responsible for which path packets should travel on a

network

2 - Data Link Responsible for which physical devices packets

should go to

1 - Physical The physical infrastructure to transport data

The top 3 layers are typically implemented in software within the Operating
System:
Layer Where it is implemented

7 - Application Software

6 - Presentation Software

5 - Session Software
The bottom 3 layers are typically implemented in hardware within devices on
the network, e.g. Switches, Routers and Firewalls:
 Layer Where it is implemented

3 - Network Layer Hardware

2 - Data Link Hardware

1 - Physical Hardware
Layer 4, the Transport layer, connects the software with the hardware layers.
SDN ("Software Defined Networking") is technology which allows more layers
of the hardware to be implemented via software.
ADVERTISEMENT

Layer 7 - Application Layer

The business logic and functionality of the application lies here. This is what the
users use to interact with services across a network. Most developers create
applications on the Application Layer.
Most of the applications you use are on the Application Layer, with the
complexity of the other layers hidden.
Examples of Layer 7 Applications:
 HTTP ("Hypertext Transfer Protocol") - Enables us to access web
applications
 FTP ("File Transfer Protocol") - Allows users to transfer files
 SNMP ("Simple Network Management Protocol") - Protocol to read and
update network device configurations
There are many applications which uses these protocols like Google Chrome,
Microsoft Skype and FileZilla.
You are accessing this class via Layer 7!

Layer 6 - Presentation Layer

Typically an unseen layer, but is responsible of adapting, transforming and
translating data. This is to ensure the application and layers beneath can
understand one another.
 Encoding Schemes used to represent text and data, for example
ASCII(American Standard Code for Information Interchange) and
UTF(Unicode Transformation Format).
 Encryption for services, for example SSL ("Secure Sockets Layer") and
TLS ("Transport Security Layer")
 Compression, for example GZip in use in many implementations of
HTTP.
Layer 5 - Session Layer
This layer's responsibility is handling connections between the application and
the layers below. It involves establishing, maintaining and terminating
connections, otherwise referred to as sessions.
Common protocols which represent the Session Layer well are:
 SOCKS - A protocol for sending packets through a proxy server.
 NetBIOS - An older Windows protocol for establishing sessions and
resolving names.
 SIP ("Session Initiation Protocol") - For engaging in VOIP ("Voice Over
IP") communications

Layer 4 - Transport
The layer which allows applications to be represented on the network.
Some well known applications on this layer:
 TCP ("Transmission Control Protocol") - Used for many applications,
ensuring stability, control of how much data can be sent at any given
time, reliability and more.
 UDP ("User Datagram Protocol") - Lightweight and quick protocol use
for many services.
 QUIC ("Quick UDP Internet Connections") - A protocol designed for
faster connections and goes hand-in-hand with the version 2 of the HTTP
protocol.

Layer 3 - Network
A layer responsible of routing packets between networks via routers.
On this layer, the following protocols reside:
 IP ("Internet Protocol") - Used everyday when accessing the Internet.
Comes in two versions, IP version 4 and 6.
 ICMP ("Internet Control Message Protocol") - Used by network devices
and network operators, to diagnose network connections or for devices to
send and respond to error conditions and more.
 IPSec ("Internet Protocol Security") - Allows encrypted and secure
connections between two network devices.

Layer 2 - Link
Link networks, as the name implies, consist of protocols designed to send
packets through the actual links (physical connections) that network nodes are
connected to. A simpler way of thinking of it is that the Link Layer is
responsible for moving data from physical over to logical (to the network layer).
Protocols on this layer include:
 Ethernet - An essential protocol used by most operating systems when
connecting to networks using a physical cable.
 Wi-Fi ("Wireless Fidelity") - For accessing networks via radio signals. It
uses a family of protocols called IEEE 802.11.xx
 NDP ("Neighbor Discovery Protocol") - IP version 6(IPv6) uses this
protocol on the Link Layer to gather information required to
communicate via IPv6

Layer 1 - Physical
Physical layer represents the signaling which allows bits and bytes to transfer
between a physical medium. It can be transferred via radio or signals over a
cable, using electrical signals or light, for example fiber.
Examples of the Physical Layer protocols includes:
 CAN Bus ("Controller Area Network") - Used in microcontrollers and
other devices to communicate to other similar devices, not involving a
computer. Often used in ICS ("Industrial Control Systems").
 Ethernet Physical Layer - Used by Ethernet on the physical layer to send
signals with speeds up to many gigabits of traffic per second.
 Bluetooth Physical Layer - Bluetooth also has its own specifications on
how radio signals should be sent and received.

IP - The Internet Protocol

IP is used to communicate across networks, not just across physical links, but
between networks of routers. The addressing scheme in use is either IPv4 ("IP
Version 4") or IPv6 ("IP Version 6").
IP networks can be broken into different sections, often called subnets. This is
accomplished by adding an extra piece of information, together with the IP
address, called a netmask. The netmask dictates how large a network is and
which packet is routed within the network and which should be routed outside
of the network.
Netmasks can be represented via decimal numbers or with a slash notation.
When using slash notation, the slash follows the systems IP address. Here are
some examples:
IP Address Slash Notation Netmask

10.0.0.1 /8 - Example: 255.0.0.0

10.0.0.1/8

172.16.1.1 /12 - Example: 255.240.0.0

172.16.1.1/12
 192.168.0.1 /16 - Example: 255.255.0.0
192.168.0.1/16

192.168.0.1 /24 - Example: 255.255.255.0

192.168.0.1/24
Some IP networks are reserved for only a certain kind of traffic. The IP
addresses in the table above are reserved for only internal organizational use,
meaning they should not be routed on the Internet. These kinds of IP addresses
are commonly referred to as RFC1918 addresses.

Different Networks
Let us take a look at different networks within RFC1918 and how large the
networks are:
 10.0.0.0/8 - More than 16 million IP addresses
 172.16.0.0/12 - About 1 million IP addresses
 192.168.0.0/16 - 65534 IP addresses
IP segments can be further broken up into smaller and more granular networks.
Each network has a reserved address for broadcasting traffic to every host in the
network, this is called the broadcast address. Broadcasting data means sending
data to everyone on the network instead of sending to just a single host. There
are many applications and protocols which rely on broadcasting traffic in order
for them to work.
For each network segment the broadcast is always last IP address in the
network. For example in the network 192.168.0.0/24 network, the broadcast
address is 192.168.0.255.
The smallest netmask possible is 255.255.255.255, represented as /32. This
network only has one IP address.
If traffic needs to be sent back to the host, e.g. for communications between
applications, it is sent to the localhost address. This address is always 127.0.0.1
and is a /8 network.
In IP networks the traffic is routed by a router. A router is a networking device
which understands the IP format and can forward packets between networks.
This is different than a switch as the switch forwards data within a network,
while the router forwards between networks.

Packets on the network has headers which describe many of the important
details we already discussed within the IP protocol. IPv4 Header looks like this:
Image Credits: By Michel Bakni - Postel, J. (September 1981) RFC 791,
Internet Protocol, DARPA Internet Program Protocol Specification, The
Internet Society, p. 11 DOI: 10.17487/RFC0791., CC BY-SA
4.0, https://commons.wikimedia.org/w/index.php?curid=79949694
The Source Address is the IP address of the system who is sending the packet,
and destination is to whom the packet is intended. There are also other fields in
the header which is used by the many features of the IP protocol, but which are
outside the scope of this introduction class.
You can check your IP address on Windows by running the
command ipconfig within a Command Line Window. On Linux this is done
with the ip addr show or ifconfig command.
When a computer needs to communicate to something which can not be found
on the LAN, it sends traffic to the default gateway as per how the system is
configured. The default gateway being a router which is capable of forwarding
the traffic to the destination IP address.

NAT ("Network Address Translation")

NAT allows a system accepting connections on a public IP address to map those
requests to an internal RFC 1918 IP address or vice versa. Systems which do
NAT'ing are typically firewalls and routers.
A typical implementation of NAT is where the external IP address is used as a
front for multiple internal IP addresses, and the destination port number is used
to decide which server the data should be sent to. This allows internal IP
addresses to receive traffic from external systems.

Another very common implementation is allowing internal IP addresses to

access the internet with an external IP address. The NAT keeps track of
connections from internal to destination addresses and forwards traffic across
the connections.
NAT can be configured in many ways, but in this class you we do not go into
more details of the method.
Note: NAT allows network engineers to be more flexible with their
deployments, allowing many different use-cases to unfold.

IPv6 - IP Version 6
IP version 6 is the latest standard for IP and was made to support more IP
addresses. Instead of using 32 bits of addressing for IP addresses, 128 bits is
used. This allows for enough IP addresses for the foreseeable future while IPv4
has already run out.
IPv6 addresses uses 8 groups of of 4 hexadecimal numbers. An IPv6 address
look like this: 2a00:1450:400f:80a::200e:. Notice it does not have have the 8
groups of 4 hexadecimal numbers. This is because IPv6 addresses can be
shortened via simple rules:
 Leading 0's can be shortened
 Double colon (::) can be used to represent a continuous string of 0's.
The expanded IPv6 address is: 2a00:1450:400f:080a:0000:0000:0000:200e.
The localhost can be reduced into ::1 and ::.
IPv6 has networks, i.e. subnets, just like IPv4 has.
The IPv6 header looks like this:

We can see a much simpler header with a lot more room for IP addressing.
IPv6 is used more and more, and there is built in support for this protocol in
many tools. For example with ping we can switch between IPv4 and IPv6 with
the -4 and -6 flag respectively.
Run ipconfig and see if you see any IPv6 addresses. If you have IPv6 enabled,
try ping -6 google.com and ping -4 google.com. See how the command allows
us to use either IPv4 or IPv6?
Note: If you do not have IPv6 today, there are many public cloud services
which will grant you a public IPv6 address today which you can use to
experiment and explore with.

ICMP
ICMP is often associated with Ping and Traceroute. ICMP can be used for other
things, such as ask a node for its time referred to as an ICMP Timestamp
request. An ICMP Timestamp request simply allows e.g. a Router to ask
another Router to synchronize their time, an important attribute in network
communications.
A common tactic for attackers to check if systems are available on a network, is
to conduct a Ping Sweep. The goal of such activity is to make the target device
in a network range reply to ping requests so that the attacker knows it is
available. This approach is naive as many systems by default block incoming
pings.

Traceroute
Tracerouting is a way to determine which routers are involved in sending a
packet from system A to B. Knowing which routers our packets take can be
useful both better understanding our networks and also in understanding the
attack surface. A router is responsible for routing the package in the right
direction. Imagine this as driving on a road, where road signs at intersections
guide you to the destination. These signs at intersections represent routers.
Traceroute identify these signs and intersections and tells you how far away
they are, measured in milliseconds (ms).
The IPv4 TTL and IPv6 Hop Limit headers have the same function. Every
router who routes a packet will decrement this value by 1, and if the value
reaches 0 the router will discard the packet and return an ICMP Time Exceeded
packet to the sender.
To perform a traceroute on Windows:
tracert google.com
To perform a traceroute on Linux (not installed by default):
traceroute google.com
The process of tracerouting via these tools is simple:
1. The operating system sends a packet google.com, the TTL value is set to
1.
2. The packet is routed on the network, and the first router decrements the
TTL by 1, leaving it at 0. This causes the router to drop the packet and
send "ICMP Time Exceeded" back to the source.
3. The client increases the TTL 1, allowing the packet to be routed through
one additional hop.
This process is repeated, increasing the TTL with 1 until the destination has
been reached.

DNS ("Domain Name System")

DNS is used to map applications, via names, to IP addresses. For example if
you want to use your browser to visit http://google.com, the browser must first
make a request to a DNS server to resolve the IP address behind google.com
Systems are typically configured with a primary and secondary domain name
server. These settings can be configured manually or be provided by a DHCP
server. This allows our computer systems to reach a DNS server for it to resolve
for us.
The DNS server is then responsible for resolving the request. It will then
proceed to check its own cache to see if it already knows the answer. Each DNS
answer can be cached, that is stored temporarily to speed up future requests, for
a certain TTL ("Time To Live"). The TTL is typically set to a couple of
minutes, for example 10 minutes.
If a DNS server does not have a answer in its cache it will then proceed to check
who is responsible for giving the answer. This is done via a recursive process
which involves asking a hierarchical system of name servers which inevitably
will make the DNS request to end up at the Authoritative Name Server.
You can try to do a DNS lookup with Windows or Linux now. From a
command line terminal on Windows type nslookup w3schools.com , or on
Linux type dig w3schools.com. You should see output like this:
The IP address of w3schools.com can be seen in the ;; ANSWER
SECTION. When this screenshot was captured, the IP address behind the
w3schools.com name was 66.29.212.110.
The Authoritative Name Server is the DNS server which is responsible for
giving the definitive answer to a question. For example the IP address of
google.com will be answered by their authoritative name server, and we can see
this server by querying for it:
Finding authoritative name server on Windows:
nslookup -type=SOA google.com
Finding authoritative name server on Linux:
dig -t SOA google.com

DHCP ("Dynamic Host Configuration

Protocol")
As the name implies, the DHCP protocol allows any system on a network to
reach out to a server and receive a configuration. Such configuration typically
implies receiving IP address and network range, default gateway and DNS
servers.
DHCP allows for easy management of clients joining and leaving a network.
If you are curious if you are using DHCP right now you can type ipconfig
/all on a Windows system and look for "DHCP Enabled: Yes" in the output.
Your computer might have multiple network interfaces

VPN ("Virtual Private Network")

A VPN is a system which enables two system to establish encrypted forms for
communication, enabling network traffic to be encrypted in transit. Many VPN's
is a client to server architecture, allowing the client to access multiple services
behind the VPN. VPN's hosted by your work place is also likely to provide
access to resources otherwise only accessible from the inside.
Some VPN services are designed for user privacy and encryption for data in
transit. These services enables users to send network data via the VPN,
effectively masquerading the users IP address when navigating the Internet.
It is generally good practice to use VPN's to secure your network
communications, however we should not use any kind of VPN service. Free
VPN services can sometimes be malicious, inspecting, reading and storing your
sensitive data.

In-Depth Transport and Link Layers

Computer systems often needs to talk to other systems; this is done by putting
them on the same network. Several different technologies are in place to enable
computers to talk over different kinds of networks. In this section we will go
deeper into the protocols which are used in most networks.
The networks we are using consists of multiple protocols, some which are
featured in this class. There are also many other protocols in use in networks, all
which have the potential of having security risks associated with them.

TCP ("Transmission Control Protocol")

Just like IP uses IP addresses for addressing, TCP and UDP uses ports. A port,
as indicated with a number between 0 and 65535, dictates which network
service should process the request.
In the picture below we can see a TCP packet and how it would look like for
anyone inspecting traffic on the network.
We can see the graphic showing 16 bits for both source and destination ports,
this is the same for for UDP. The Sequence and Acknowledgement numbers are
used in the three-way handshake and to reliably transfer data. We can also see
the control bits used to indicate what kind of packet it is. The other headers also
play an important part, but outside of the security course.

TCP 3-Way-Handshake
TCP uses a three-way handshake to allow two systems to engage in
communications. The handshake uses 32 bits of PRNG ("Pseudo Random
Number Generator") numbers to establish the handshake. The handshake
enforces that both parties intends to communicate.
Here is a graphic to illustrate:

Explanation on how TCP engages in communications:

1. Client initiates the communication by sending a packet with the control
bit SYN set in the header, a PRNG number in the Sequence number field
and a target destination port. The Network Layer (Layer 3) allows the
packet to be sent to a remote system. This packet is referred to as a SYN
packet.
2. Server receives the packet, reads the Sequence Number from the Client
and crafts a response. The response sets the Acknowledgement field with
the Sequencer number of the client with the number 1 added to it.
 Furthermore the response contains the controls bits SYN and ACK set
and the Sequence number is set to the Servers PRNG number. This
packet is referred to as a SYN/ACK packet.
3. The Client receives the SYN-ACK packet and to complete the handshake
returns a packet with the ACK control bit set. The final packet also
confirms the PRNG number from the Server by returning it in the
Acknowledgement field of the header.
This process is commonly referred to as the SYN, SYN/ACK, ACK 3-way
handshake.
In the following picture we can see two hosts performing the 3-way handshake:

The screenshot shows the tool Wireshark being used to capture and inspect
packets. Wireshark is a very good utility because it allows us to visualize
packets and inspect them more in-depth. The screenshot shows 3 packets
representing the 3-way handshake. Notice the source and destination columns
representing the two parties communicating. The Info field shows source and
destination port, including (within the square brackets) the SYN, SYN/ACK and
ACK.
The next screenshot shows data being communicated between the parties after
the 3-way handshake. The data inside of the TCP packet has been highlighted
near the bottom of the picture.
A message to W3School students is shown in the screenshot above. Can you see
it?
ADVERTISEMENT

Spoofing Traffic
There are few restrictions on networks today for anyone to create packets as
they desire. Anyone can create packets with any of the fields of the headers set
to whatever value they desire. This is called spoofing, allowing attackers to send
traffic on behalf of others.

TCP has security built into the protocol, but it relies on the strength of the
PRNG ("Pseudo Random Number Generator") number generators. If the
Sequence numbers of the communicating parties can be guessed, the security of
TCP can be compromised in the sense that an attacker can engage in spoofed
communications via TCP.
Many protocols are easily spoofed, but TCP offers some resiliency against this.
Protocols such as UDP and ICMP does not offer similar protection.
Spoofing packets is typically done by attackers with root / system capabilities,
i.e. the highest privileges on the Operating System. The reason being that
Operating Systems enforces the use of API's which forces the user to conform
to the rules of communications as specified in RFC's ("Request For
Comments"). If the attacker does not have the highest privileges, they will not
be able to craft their own packets on the network.

UDP ("User Datagram Protocol")

UDP is used for traffic which does not need the resilience and security of TCP,
typically applications such as VOIP, but in the modern world more applications
are using UDP to support fast packet transfer with resiliency and security built
into the higher levels of the OSI model; QUIC is an example of this.
Looking at the UDP Header we can see the same Source and Destination ports
in use, but no Sequence numbers or Control bits. The protocol has much less
overhead, leading to faster transmission of data.

Because UDP does not have features such as the 3-Way-Handshake, UDP can
be easily spoofed.

Switched Networks
Systems are connected to a LAN ("Local Area Network") through a Switch.
Switches uses MAC ("Media Access Control") addresses for addressing, not the
more well-known IP address. Switches forward traffic across Local Area
Networks, i.e. your home-network or within branches of your organization.
MAC Addresses are designed to be unique, but anyone can change their MAC
Address as long as they have administrator rights.
The MAC address defined by 6 octets, for example: FC:F8:AE:12:34:56
The first three octets represent the organization which manufactured the device
communicating, called OUI ("Organizational Unique Identifier"). The above
MAC address is assigned to Intel Corporate. You can search for MAC
Addresses in many places, for example: https://www.adminsub.net/mac-
address-finder/intel.
The last three octets is determined by the manufacturer.

ARP
ARP ("Address Resolution Protocol") is the protocol which allows computer
systems to know which MAC address belongs to which IP address. If the traffic
has to be routed, the computer system will forward traffic to the Default
Gateway configured on the system.
ARP, like DNS, is a protocol which resolves one address into another. Every
time a system tries to communicate to an IP address which is on the LAN it will
check its ARP cache to see if has recently been resolved.
You can inspect your own ARP. Simply run the command arp -a on both Linux
or Windows. This reveals which systems your system has recently
communicated with.
Alice: Does anyone know the MAC address of 192.168.10.10?
Bob: Sure thing Alice, here is my MAC Address.

VLAN ("Virtual LAN")

VLAN, often called Private VLAN's, is a way for a Switch to embed tags (or a
VLAN ID) within the Frame. Multiple Switches can then make sure that
computer systems on the LAN can only communicate to certain other systems,
i.e. other systems on with the same VLAN ID.
Firewalls
Firewalls are a central architectural element to any network. They are designed
to keep out all network traffic, except traffic which we allow. Firewalls operate
on Layer 4, typically controlling TCP and UDP access to internal assets. Next-
Generation Firewalls operate on all the layers of the OSI model, including
Layer 7.
Traffic entering a network, e.g. through a Firewall, is called ingress traffic.
Traffic leaving is called egress.

Layer 4 Firewall
The traditional Firewall is a Layer 4 Firewall with features such as:
 NAT
 Routing
 Blocking or Allowing traffic
 Track active network connections
 Support VPN connections
Note: These Firewalls are typically cheaper and offers more throughput on the
network than a more modern Next-Generation Firewall.

NGFW ("Next-Generation Firewalls")

A modern Firewall has capabilities that range much wider than a a Layer 4
Firewall. These capabilities are typically security features.
A NGFW Firewall can also track active network connections, but is also
typically capable of tracking:
 Locations via geo-location databases. This means the Firewall can make
blocking or allow actions based on the location of users. Location
services are not always accurate and can often easily be bypassed using
VPN services or by using other services like jump stations for attacks.
 Users
 Applications
 Sessions
 Ports and services
 IP addresses
Other features of a NGFW includes:
 Identify and control applications on the network.
 It can be virtualized to run as a software Firewall.
 Often offers simple and intuitive management.
 Supports protecting against known threats via ("Intrusion Prevention
System").
 Potential to detect and prevent unknown threats via sandboxing solutions.
  Offers a potential to manage unknown traffic, e.g. traffic which cannot be
attributed to an application.
 Capabilities to terminate and inspect encrypted traffic.
 Can control users, not just system via respective IP addresses.

Note: Which features your NGFW comes with often greatly depends on which
licenses have been purchased and the capacity of the hardware running the
Firewall.

Firewall Administration
A Firewall can typically be administered via a proprietary management
application, or via a web-browser accessing the Firewalls management via
HTTP.
Management ports to Firewalls, including other management services of an
organization, should ideally be segmented away from regular user access.
Ideally the segmentation of management services is connected to an
organizations user directory, for example Active Directory for Windows
environments.

Segmentation
Firewalls can segment traffic between hosts and systems into segments,
sometimes called zones. Each segment holds services which are allowed to
communicate between one another.

Any connection to or from the segment should be carefully controlled by the

Firewall, preventing any unauthorized connections to make successful
connections. Smaller segments offer more segregation, but requires more
management.
Without any segmentation, users and systems can talk directly to each other
without the Firewalls enforcement. This is called a flat network.
Adding more segmentation we can envision segments representing services,
where each segment is a service provided in the organization. Each segment
could contain the different servers responsible for making the service
operational. Communications within the segment is allowed, but any access in
and out from the segment is controlled by the Firewall.
Another segmentation idea would be to control segments based on their
functions, for example clustering web applications within a segment with other
web-applications, databases within one segment and other kinds of services
within their segment.
Note: A very common user directory is Microsoft's Windows Active Directory.
It holds information about which users, computers and groupings the
organization holds.

The best and most secure kind of segmentation is called zero-trust architecture,
forcing all systems on the network to explicitly be allowed to communicate to
different services.
To ease management of Firewall rules, the Firewall management is ideally
connected to the organizations user directory. This can allow Firewall
administrators to create careful rules based on employee responsibilities,
allowing the organization to add and remove permissions which are applied on
the network without asking the Firewall administrators for changes anytime
there is a role change. This is sometimes called user-based policy control.
Examples include:
 IT-Administrators should be able to use management protocols to the
different services.
 HR employees should be allowed to access HTTPS to the HR platforms.
 Helpdesk employees can only access helpdesk related services.
 Unrecognizable users can be identified and provisioned accordingly.
Note: A very common user directory is Microsoft's Windows Active Directory.
It holds information about which users, computers and groupings the
organization holds.

IPS ("Intrusion Prevention System")

and IDS ("Intrusion Detection System")
Sometimes IPS and IDS systems are deployed as stand-alone systems on the
network, but very often they are included in a NGFW.
IPS and IDS systems have signatures, algorithms and heuristics to detect attacks
on the network or host. An IDS or IPS deployed on a host is called a HIDS
("Host Intrusion Detection System").

In this course the term IDS and IPS is used interchangeably as the difference
between them is often just a matter of configuration of how they operate. An
IPS system is positioned in such a way it can detect and block threats, while an
IDS system is only capable of detecting threats.
IPS systems can be used to detect and block attackers and often rely on frequent
updates and inspection in encrypted traffic.
Note: A very useful feature of IDS and IPS is the frequent updates of new
signatures of developing threats from the vendors. This allows defenders some
reassurance that new threats will be blocked as the firewall is updated with new
updates.

Content and Application Filtering

The Firewall can make attempts in understanding which applications and
content is traversing the network. Such detection can further activate other
security features like IPS to protect the systems between the Firewall.
URL Filtering
NGFW can also protect content accessed via HTTP. The Firewall can look up
domains in a database containing lists of domains and respective categorization.
Firewall can then enforce only acceptable categories of domains are allowed by
users, for example news is allowed while gambling is not.

Elements such as domain age and validity could also be checked, preventing
users from visiting domains which has recently been created and not yet
categorized, or checking for fraudulent activities by analyzing the content of the
domain.
Instead of rejecting access to websites, the Firewall could intercept the request
and send the user to what is called a captive web portal. On this portal the user
could be warned about immediate danger or the violation of company policy in
e.g. visiting unacceptable content. In some cases you could allow the user to
provide a reason why they need to access the content, then let them continue if
they have provided a reason.
Categories within domains can be many, for example websites hosting content
related to:
 Hacking
 Nudity
 Violence
  Phishing
 Dating
 Instant Messaging
 Entertainment
 Anonymizing services

Applications
The Firewall can try to determine which applications are in use, not just the
protocols. Many protocols are capable of carrying other applications, for
example HTTP can hold thousands of different applications. A Firewall can try
to decode the network streams on Layer 4 and try to determine the content
which is presented on Layer 7.

A screenshot shows what a user could see when an application has been
blocked.
Content Control
As Applications are being identified, the Firewall could try to reveal specific
content within the applications, for example content being downloaded:
 Word documents
 Executables
 Source-code
 Scripts

In such files, the Firewall can try to identify malware, proprietary and
confidential information not supposed to leave the network and much more.
The Firewall could support many different protocols and the content running
through them, for example:
 HTTP
 SMB
 FTP
 IMAP & POP3
 SMTP

Sandboxing
In this context, sandboxing means to have a platform execute files, which are
may be malicious. The sandbox records and monitors the activity of the file to
see if it is malicious or not.

Sandboxing typically allows the Firewall to forward executable files to this

platform and prevent users from downloading the file until a verdict has been
made whether it is malicious or not.
A modern sandbox has capabilities to run files in multiple different platforms,
for example:
 Windows 7,8 and 10.
 Android phones.
 Linux
Files which are interesting to execute and explore in a sandbox is more than just
an executable file. Many files are capable of performing malicious actions on
the operating system of our users:
 ZIP files with content to run
 Office documents
 PDF files
 Java applications
 JavaScript
 Screensavers
There are many sandboxes online you can try yourself, outside of what a
NGFW can provide:
 https://www.joesandbox.com/
 https://www.virustotal.com/
 https://www.hybrid-analysis.com/
 https://any.run/
There are also sandboxes you can install yourself, for example:
 https://cuckoosandbox.org/

Decrypting Traffic
Many Firewalls support installing certificates which allows for decryption of
traffic. If content is decrypted, content can then be inspected for threats.
Decryption can take place on Egress or Ingress traffic, or both. For Ingress
traffic the Firewall could protect servers against incoming traffic. Egress traffic
allows the Firewall to protect users and systems needing to communicate
outbound.

The Firewall will often avoid decrypting traffic such as health care and financial
data as this may have privacy and other implications. Decryption of traffic
requires more effort from the organization to distribute keys to clients which is
used by the firewall to decrypt traffic.
Note: Remember Egress and Ingress traffic? Egress means traffic leaving the
network, while Ingress means traffic arriving to the network.

Unknown Traffic
Some traffic can not be decrypted or understood fully by the Firewall. Many
reasons can apply, for example a proprietary application sends data which the
Firewall does not know of. Such traffic can also be categorized as Unknown. A
Firewall administrator should consider blocking such applications, especially
from networks which are considered high-risk.

WAF ("Web Application Firewall")

While Firewalls can do a decent job, they often lack full understanding of what
a protocol is capable off. Because of such, protocol specific firewalls are also
developed where WAF is one of the more common ones.
A WAF allows more features specific to the HTTP protocol than a regular
Firewall, making it more capable of stopping threats.
While a WAF tries to do a good job in blocking threats on HTTP, it often
provides other very useful utilities for organizations making them very viable
for much more than just blocking threats. Here are some examples:
 WAF can help build redundancy, that is having multiple servers to
present the same service. This allows for for organizations to have a
service in a higher available modus, allowing them to put a server offline
while the other servers are still capable of serving users trying to access
the service. This is useful as concepts like patching often requires you to
restart the service, and the redundancy allows users to still access the
service.
 A WAF can help enforce best practice security rules, for example a single
place to maintain and enforce encryption, multi-factor authentication and
other concepts which this class covers.
 It can be used to develop a single front and protection mechanism for
multiple web-servers which are behind the WAF.
Web Applications are integral to almost everything we do, whether it
is to access the Internet or to remotely control your lawnmower. In
this introduction class we will cover the basics of web application
security.

The HTTP protocol

HTTP is the carrier protocol which allows our browsers and applications to
receive content such as HTML ("Hyper Text Markup Language"), CSS
("Cascading Style Sheets"), images and videos.

URLs, Query Parameters and Scheme

To access a web application we use a URL ("Uniform Resource Locator"), for
example: https://www.google.com/search?
q=w3schools+cyber+security&ie=UTF-8
The URL to google.com contains a domain, a script being accessed and Query
Parameters.
The script we are accessing is called /search. The / indicates it is contained in
the top directory on the server where files are being served. The ? indicates the
input parameters to the script and the & delimits different input parameters. In
our URL the input parameters are:
 q with a value of w3schools cyber security
 ie with a value of UTF-8
The meaning of these inputs is up to the webservers application to determine.
Sometimes you will see just / or /? indicating that a script has been setup to
serve to respond to this address. Typically this script is something like an index
file which catches all requests unless a specific script is specified.
The Scheme is what defined the protocol to use. In our case it is the first part of
the URL: https. When the scheme is not defined in the URL it allows the
application to decide what to use. Schemes can include an entire array of
protocols such as:
 HTTP
 HTTPS
 FTP
 SSH
 SMB

HTTP Headers
The HTTP protocol uses many headers, some custom to the application and
others well defined and accepted by the technology.
Example request to http://google.com
GET /search?q=w3schools+cyber+security&ie=UTF-8 HTTP/1.1
Host: google.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88
Safari/537.36
Accept: image/avif,image/webp,image/apng,image/*,*/*;q=0.8
Referer: https://w3schools.com/
Accept-Encoding: gzip, deflate
Cookie: cookie1=value1;cookie2=value2
The request header specifies what the client wants to perform on the target
webserver. It also has information regarding if it accepts compression, what
kind of client is accessing and any cookies the server has told the client to
present. The HTTP request headers are explained here:
Header Explanation

GET /search... GET is the verb we are using to access the

HTTP/1.1 application. Explained in detail in the section
HTTP Verbs. We also see the path and query
parameters and HTTP version

Host: google.com This header indicates the target service we want to

use. A server can have multiple services as
explained in the section on VHOSTS.

User-Agent A client application, that is the browser in most

cases, can identify itself with the version, engine
and operating system

Accept Defines which content the client can accept

Referer: If the client clicked a link from a different website

https://w3schools.com/ the Referer header is used to say from where the
client came from

Accept-Encoding: Can the content be compressed or encoded? This

gzip, deflate defines what we can accept

Cookie Cookies are values sent by the server in previous

requests which the client sends back in every
subsequent request. Explained in detail in the
section State
With this request, the server will reply with headers and content. Example
headers are seen below:
HTTP/1.1 200 OK
Content-Type: text/html
Set-Cookie: <cookie value>
<website content>
The response header and content is what determines what we will see in our
browser. The HTTP response headers are explained as following:
Header Explanation

HTTP/1.1 200 The HTTP Response code. Explained in detail in the

OK HTTP Response Codes section

Content-Type: Specifies the type of content being returned, e.g. HTML,

text/html JSON or XML

Set-Cookie: Any special values the client should remember and return
in the next request

HTTP Verbs
When accessing a web application the client is instructed on how to send data to
the web application. There are many verbs which can be accepted by the
application.
!Verb Used for

GET Typically used to retrieve values via Query Parameters

POST Used to send data to a script via values in the body of the
Request sent to the webserver. Typically it involves creating,
uploading or sending large quantities of data

PUT Often use to upload or write data to the webserver

DELETE Indicate a resource which should be deleted

PATCH Can be used to update a resource with a new value

These are used as the web application requires. Restful (REST) web services are
especially good at using the full array of HTTP Verbs to define what should be
done on the backend.

HTTP Response Codes

The application running on the webserver can respond with different codes
based on what occurred on the server side. Listed are common response codes
the webserver will issue to the client which security professionals should know
about:
Code Explanation

200 Application returned normally

301 Server asks client to permanently remember a redirect to a new

location where the client should access

302 Redirect temporarily. Client doesn't need to save this reply

400 The client made an invalid request

403 The client is not allowed to access this resource. Authorization is

required

404 The client tried to access a resource which does not exist

500 The server errored in trying to fulfill the request

REST
Rest services, sometimes called RESTful services, employ the full force of
HTTP Verbs and HTTP Response Codes to facilitate the use of the web
application. RESTful services often uses parts of the URL as a query parameter
to determine what happens on the web application. REST is typically used by
API's ("Application Programming Interfaces").
REST URLs will invoke functionality based on the different elements of the
URL.
An example REST URL: http://example.com/users/search/w3schools
This URL will invoke functionality as part of the URL instead of Query
Parameters. We can decipher the URL as:
Parameter Comment

users Accessing the users part of the functionality

search Accessing the search feature

w3schools The user to search for

Sessions & State

There is no built in way for a server to identify a returning visitor in HTTP. For
a webserver to identify the user, a secret value must be communicated to and
from the Client in each request. This is typically done via Cookies in headers,
however other ways are also common such as via GET and POST parameters or
other headers. Passing state via GET parameters is not recommended as such
parameters are often logged on the server or in intermediaries such as a proxy.

Here are some common Cookie examples which allows the application on the
webserver to control sessions and state:
 PHPSESSID
 JSESSIONID
 ASP.NET_SessionID
These values represent a certain state, often called a session, on the server. This
state represents things like:
 What user you have logged in as
 Privileges and authorizations
It is important that session value, sent to the Client, can not be easily guessed or
otherwise identified by others. If they could, an attacker could then present
themselves as other users on the web application.
State can also be saved on the client. This involves the server sending all the
states to the client and relies on the client sending back all the items. Such
implementations relies on encryption to check the integrity of the state the client
is claiming. Examples of implementations using this is listed below:
 JWT ("JSON Web Tokens")
 ASP.Net ViewState
You are using cookies to take this class! You can inspect these cookies in your
web browser by opening up the developer tools. This is done by
hitting F12 within the browser, opening up the developer tools window. Within
this window you should be able to find the correct place where your cookies are
stored.
In Google Chrome, the cookies were identified in the Application tab above.
Note: Can you think of why the cookies have been masked away in the
screenshot so you cannot read them?

Virtual Hosts
One webserver can process many applications via Virtual Hosts, often
abbreviated as Vhosts. To facilitate access to other Virtual Hosts the web server
typically reads off the Host header of the client request, and based on this value
sends the request to the correct application.

URL Encoding
For an application to safely transfer content between the server and client, some
characters must be encoded to ensure they do not impact the protocol. To
preserve the integrity of the communications, URL encoding is used.
URL Encoding replaces unsafe characters with a % and two hexadecimal digits.
For example:
 Percentage is replaced with %25
 Space is replaced with %20
 Quote is replaced with %22
An excellent tool to perform text analysis and run operations such as URL
Decoding is CyberChef. You can try it out in your browser
here: https://gchq.github.io/CyberChef/
Note: Play around with Cyber Chef and see if you can reveal what the following
message in URL encoded characters hold: %48 %65 %6c %6c %6f %20 %64
%65 %61 %72 %20 %77 %33 %73 %63 %68 %6f %6f %6c %73 %20 %73
%74 %75 %64 %65 %6e %74 %2e %20 %48 %6f %70 %65 %20 %79 %6f
%75 %20 %61 %72 %65 %20 %6c %65 %61 %72 %6e %69 %6e %67 %20
%73 %6f %6d %65 %74 %68 %69 %6e %67 %20 %74 %6f %64 %61 %79
%21

JavaScript
To support dynamic content, browsers use the scripting language JavaScript.
This enables developers to program solutions which will run on the client,
enabling more interactive and "alive" web-content.
JavaScript is also involved in many attacks against web-applications and client
applications such as browsers.

Encryption with TLS

The HTTP protocol does not support encryption for data-in-transit, hence a
wrapper around HTTP is added for encryption support. This is indicated with a
S following HTTP, i.e. HTTPS.
The encryption used to be SSL ("Secure Sockets Layer"), but has since been
deprecated. Instead TLS ("Transport Layer Security") is typically used to
enforce encryption.