Ethical Hacking

and Penetration
Testing
UNIT 8
IDS, Firewalls and Honeypots

Prof. Tushar Gohil

 Outline
 Intrusion Detection Systems
 Firewalls
 Honeypots

Module 08 : IDS, Firewalls and Honeypots 2

Intrusion Detection System

Sarvajanik College of Engineering & Technology 3

Intrusion Detection System
 Intrusion detection systems (IDS) play a critical role in the
protection of the IT infrastructure. Intrusion detection involves
monitoring network traffic, detecting attempts to gain
unauthorized access to a system or resource, and notifying the
appropriate individuals so that counteractions can be taken.

Sarvajanik College of Engineering & Technology 4

Intrusion Detection System : Types and Components
 Intrusion detection systems (IDS) originated in the 1980s, introduced
by James Anderson in his paper "Computer Security Threat
Monitoring and Surveillance." IDS can be categorized into Network-
Based IDS (NIDS) and Host-Based IDS (HIDS).
 These systems detect and track potential attacks, alert
administrators, and consist of multiple components:
• Network Sensors: Detect and send data.
• Central Monitoring System: Analyzes data from sensors.
• Report Analysis: Provides countermeasure information.
• Database and Storage: Performs trend analysis and stores attacker data.
• Response Box: Processes information and generates responses.
 The effectiveness of an IDS depends on the strategic placement and
tuning of network sensors to accurately detect suspicious activity.
Sarvajanik College of Engineering & Technology 5
Intrusion Detection System : Types and Components
 The effectiveness of an Intrusion Detection System (IDS) heavily
depends on the strategic placement of its network sensors.
 For instance, a sensor placed in the demilitarized zone (DMZ) is
effective at detecting misuse in that area but will be ineffective for
spotting attackers within the internal network.
 After deciding on sensor placement, specific tuning is necessary to
avoid unnecessary alerts.
 Without this tuning, sensors may flag all traffic matching certain
criteria, even if it doesn't indicate a real threat.
 Therefore, IDS must be carefully trained to recognize genuinely
suspicious activity.

Sarvajanik College of Engineering & Technology 6

Intrusion Detection System

Sarvajanik College of Engineering & Technology 7

Intrusion Detection System : Pattern Matching and Anomaly
Detection
 Intrusion Detection Systems (IDSs) use three primary methods:
anomaly detection, protocol decoding, and pattern matching.
• Anomaly Detection: IDSs create profiles of normal activities and detect
deviations. They require significant time to minimize false negatives.
However, attackers can potentially trick the system by gradually altering
their behavior.
• Protocol Decoding: IDSs analyze protocol-specific behavior by reassembling
packets and comparing them to expected patterns. For example, detecting
abnormal DNS activity might indicate a cache poisoning attack. This method
requires the IDS to maintain state information.
• Pattern Matching: This method relies on a database of known attack
signatures, like Snort. When a match is found, the IDS triggers an alert.
However, this system can only detect attacks that match pre-loaded
signatures, leaving it vulnerable to new or obfuscated attacks.

Sarvajanik College of Engineering & Technology 8

Intrusion Detection System : Pattern Matching and Anomaly
Detection

Sarvajanik College of Engineering & Technology 9

Intrusion Detection System : Snort
 Snort, developed by Martin Roesch and Brian Caswell, is a free,
lightweight, network-based Intrusion Detection System (IDS)
compatible with Linux and Windows platforms.
 It functions as a network sniffer, monitoring and logging activities
that align with predefined signatures.
 These signatures can encompass various traffic types, including IP,
TCP, UDP, and ICMP. Snort rules consist of two main components:
• Rule Header: Specifies the actions to be taken when a rule is triggered.
• Rule Options: Defines the alert messages associated with the rule.

Sarvajanik College of Engineering & Technology 10

Intrusion Detection System : Snort
 Here is a sample rule:
• Alert tcp any any -> any 80 (content: "hacker"; msg: "Hacker Site
Accessed";)
• The text up to the first parentheses is the rule header.
• first part is known as the rule action
– rule actions can include the following:
» Alert
» Log
» Pass
» Activate
» Dynamic
• The next item is the protocol
– In the example, TCP was used.
• After the protocol is the source address and mask
– Although the example uses any any, it could have been a specific network such as
10.10.0.0/16.
Sarvajanik College of Engineering & Technology 11
Intrusion Detection System : Snort
• This is followed by the target IP address and mask
– which again can be specific or listed as any
• The final entry of the rule header designates the port.
– This example specifies 80.
• The section enclosed inside the parentheses specifies the rule options:
content: “hacker”; msg: “Hacker Site Accessed”;.
• Rule options are not required but are usually the reason for creating the rule
• The first portion specifies the action
– Which is to examine port 80 traffic for the word hacker
– If a match occurs, a message should be generated that reads, “Hacker Site
Accessed,” and the IDS would create a record that a hacker site might have been
accessed.
– The rule option is where Snort has a lot of flexibility

Sarvajanik College of Engineering & Technology 12

Intrusion Detection System : Snort
• Table 9-2 lists some common keywords Snort can use.

Sarvajanik College of Engineering & Technology 13

Intrusion Detection System : Snort
• Few basic snort rules.

Sarvajanik College of Engineering & Technology 14

Intrusion Detection System : Snort
 Creating Snort rules is only part of the process; analyzing the alerts
generated by Snort is equally important.
 Signature analysis helps identify malicious activities and trace the
offenders.
 The key categories of malicious activities include:
• Scans and Enumeration
• Denial of Service (DoS) Attacks
• Exploits
 Once connected to the internet, an IDS like Snort can produce
numerous alerts within a few hours.
 For example, a signature of an Nmap ACK scan shows the attacker's IP,
the target, and repeating sequence and acknowledgment numbers—
an indicator of abnormal TCP behavior.
Sarvajanik College of Engineering & Technology 15
Intrusion Detection System : IDS Evasion
 Attackers use several techniques to evade detection by Intrusion Detection Systems
(IDSs):
• Flooding: Attackers overload the IDS with excessive traffic, including low-priority triggers, to
distract the system and allow more harmful attacks to pass unnoticed. They may also use TTL
attacks, sending traffic fragments with different Time-To-Live values to bypass the IDS.
• Insertion and Evasion:
• Insertion Attacks: The IDS accepts packets that the target system rejects, causing the IDS and target to
receive different data streams.
• Evasion Attacks: The target system accepts packets that the IDS has rejected. For example, by
exploiting different fragmentation timeouts between the IDS and target system, an attacker can
deliver a successful attack to the target without detection.
• Session Splicing: This technique involves splitting the payload of an attack across multiple
packets, making it harder for the IDS to detect the attack through simple pattern matching. By
fragmenting the payload and manipulating fragment order and IDs, the attacker complicates
reassembly, potentially bypassing the IDS.
• Shellcode Attacks: Shellcode consists of executable instructions injected into a running
application, typically to open a command shell. Polymorphic shellcode, which varies the attack
code to avoid signature detection, and encoded or ciphered shellcode make detection by IDSs
more difficult.
Sarvajanik College of Engineering & Technology 16
Intrusion Detection System : IDS Evasion
 Additional IDS evasion techniques include:
• False Positives: Attackers trigger numerous false alerts to overwhelm and
desensitize the system, making it harder to identify real threats.
• Obfuscation: Techniques like Unicode encoding, encryption, and ASCII
shellcode are used to obscure the attack, helping it bypass pattern and
signature matching.
• Denial of Service (DoS): Overloading the IDS or its logging server with
excessive data to disrupt its functionality and evade detection.
• Pre-connection SYN: The attacker binds a local port to a socket before
connecting, aiming to confuse the IDS.
• Post-connection SYN: This method attempts to desynchronize the IDS from
the real sequence numbers used by the kernel, disrupting its tracking.
• Invalid RST: Sending reset (RST) packets with invalid checksums to trick the
IDS into stopping data capture, potentially missing the attack.

Sarvajanik College of Engineering & Technology 17

Intrusion Detection System : IDS Evasion
 Several tools are designed to evade IDSs by exploiting various
evasion techniques:
• HTTP Tunneling: Uses proxies, HTTP, or HTTPS to tunnel traffic from inside a
network, effectively bypassing IDS monitoring.
• ADMutate: A polymorphic buffer overflow engine that generates numerous
functionally equivalent buffer overflow exploits with different signatures,
making detection harder.
• Mendax: Creates arbitrary exploits from input text files and applies multiple
evasion techniques before sending the restructured exploit to the target.
• NIDSbench: A toolset that includes fragrouter, tcpreplay, and idstest, which
fragment traffic to obscure its true content from the IDS.
• Nessus: Known for testing IDSs, it also supports session-splicing attacks to
bypass simple pattern matching.

Sarvajanik College of Engineering & Technology 18

 IDSs (Intrusion Detection Systems) are not foolproof and can't
detect every attack, even when properly placed. Various tools and
techniques exist to bypass them.

 For IDSs to remain effective, continuous monitoring and

investigation of network activity are crucial, along with staying
updated on evolving hacking methods.

Sarvajanik College of Engineering & Technology 19

Firewalls

Sarvajanik College of Engineering & Technology 20

Firewalls
 Firewalls, whether hardware or software, are devices designed to
control and filter traffic between trusted and untrusted networks.
 They function similarly to airport security checks: just as
passengers and their belongings are screened before boarding a
plane, firewalls examine network traffic, limit its flow, and block
any traffic deemed suspicious.
 This helps protect the network by controlling access and
preventing potentially harmful activities.

Sarvajanik College of Engineering & Technology 21

Firewalls : Types of Firewalls
 Firewalls serve as a chokepoint, controlling and inspecting traffic
as it enters and exits the network.
 There are several basic types of firewall designs:
• Packet Filters: Examine individual packets and allow or block them based
on predefined rules.
• Application-Level Gateway: Also known as a proxy firewall, this type filters
traffic at the application layer by inspecting data for specific applications or
services.
• Circuit-Level Gateway: Monitors TCP handshakes and sessions to ensure
that connections are legitimate before allowing data exchange.
• Stateful Multilayer Inspection: Combines the features of packet filtering
and circuit-level gateways, tracking the state of active connections and
making decisions based on the context of traffic flows.

Sarvajanik College of Engineering & Technology 22

Firewalls : Types of Firewalls : NAT
 Network Address Translation (NAT) was developed to address the
shortage of IPv4 addresses and is detailed in RFC 1631.
 NAT allows for the translation between private and public IP
addresses.
 Private IP addresses, as defined by RFC 1918, are unroutable on
the public Internet, meaning they cannot be routed by public
Internet routers.
 The three ranges of private addresses are:
• 192.168.0.0–192.168.255.255
• 172.16.0.0–172.31.255.255
• 10.0.0.0–10.255.255.255

Sarvajanik College of Engineering & Technology 23

Firewalls : Types of Firewalls : NAT
 NAT (Network Address Translation) allows a firewall or router to act as an
intermediary between the Internet and a local network.
 It enables multiple private IP addresses within a local network to be
represented by a single public IP address externally.
 This provides a basic level of security by obscuring internal addresses from
external systems, though it's limited and considered security through
obscurity.
 NAT can also complicate the implementation of application-level protocols like
IPsec, which rely on true IP addresses since NAT involves rewriting packets.
 Bogons refer to IP addresses that are reserved but not yet allocated or
delegated by the Internet Assigned Numbers Authority (IANA) or a Regional
Internet Registry (RIR).
 These addresses should not appear in public Internet traffic, and packets
using these addresses are considered invalid or "bogus."
Sarvajanik College of Engineering & Technology 24
Firewalls : Types of Firewalls : Packet Filters
 Packet filters were the first widely used type of firewall, naturally
integrated into routers as they control network access.
 Configured through Access Control Lists (ACLs), packet filters allow
or block traffic based on packet header information.
 As packets pass through the router, they are evaluated against the
ACL rules to determine if they should be permitted or denied.
 For example, a packet filter may permit web traffic on port 80
while blocking Telnet traffic on port 23.

Sarvajanik College of Engineering & Technology 25

Firewalls : Types of Firewalls : Packet Filters
 A sample ACL demonstrates these rules, specifying permissions for
web and FTP traffic, while denying NetBIOS, Telnet, and ICMP
traffic.
no access-list 111
access-list 111 permit tcp 192.168.13.0 0.0.0.255 any eq www
access-list 111 permit tcp 192.168.13.0 0.0.0.255 any eq ftp
access-list 111 deny udp any any eq netbios-ns
access-list 111 deny udp any any eq netbios-dgm
access-list 111 deny udp any any eq netbios-ss
access-list 111 deny tcp any any eq telnet
access-list 111 deny icmp any any
interface ethernet1
ip access-group 111 in

Sarvajanik College of Engineering & Technology 26

Firewalls : Types of Firewalls : Packet Filters
 Access Control Lists (ACLs) make permit or deny decisions based
on various packet header categories, such as :
• Source IP address: Is it from a valid or allowed address?
• Destination IP address: Is this address allowed to receive packets from this
device?
• Source port: Includes TCP, UDP, and ICMP.
• Destination port: Includes TCP, UDP, and ICMP.
• TCP flag: Includes SYN, FIN, ACK, and PSH.
• Protocol: Includes protocols such as FTP, Telnet, SMTP, HTTP, DNS, and
POP3.
• Direction: Can allow or deny inbound or outbound traffic.
• Interface: Can be used to restrict only certain traffic on certain interfaces.

Sarvajanik College of Engineering & Technology 27

Firewalls : Types of Firewalls : Packet Filters
 While packet filters offer a basic level of protection by filtering
based on these criteria, they have significant limitations.
 They cannot prevent IP spoofing, inspect packet payloads, or
maintain state information, meaning they cannot determine
whether a connection was initiated from inside or outside the
organization.
 This limitation makes them vulnerable to sophisticated attacks,
such as ACK scans that exploit the inability of packet filters to track
the state of connections.

Sarvajanik College of Engineering & Technology 28

Firewalls : Types of Firewalls : Packet Filters : Example

Sarvajanik College of Engineering & Technology 29

Firewalls : Types of Firewalls : Packet Filters : Example
 In this example, an organization allows outgoing FTP traffic on port 21 but
blocks inbound FTP traffic.
 If a hacker attempts a full connect scan on port 21 targeting an internal
client, the router will block it.
 However, if the hacker uses an ACK scan on port 21, the packet will bypass
the router and reach the internal client because the router can't maintain
the state of connections.
 This limitation means the router cannot differentiate between different
types of inbound FTP packets.
 Even when blocking a scan, the router might reveal information by sending
an ICMP type 3 code 13 message, indicating that an ACL is blocking traffic.
 This response could inadvertently expose the router's presence and
filtering activities.

Sarvajanik College of Engineering & Technology 30

Firewalls : Types of Firewalls : Stateful Inspection
 Stateful inspection firewalls are an advanced type of firewall that, unlike packet
filters, can track the state of a connection.
 They maintain a state table to record the status of active connections, allowing
them to detect and block unauthorized packets, such as an ACK packet that
does not correspond to an established connection.
 This contrasts with packet filters, which would simply forward packets based on
predefined rules without considering connection status.
 In practice, many organizations use a combination of firewall technologies—
such as packet filters, proxy servers, and stateful inspection—to enhance
security.
 A common network design includes a demilitarized zone (DMZ), a protected
network segment that sits between the untrusted Internet and the trusted
internal network.
 Servers within the DMZ, known as bastion hosts, are specially hardened and
secured to withstand attacks.
 While firewalls and secure hosts are crucial, the overall architecture of the
Sarvajanik College of Engineering & Technology 31
Firewalls : Types of Firewalls : common designs used to secure
networks

Sarvajanik College of Engineering & Technology 32

Firewalls : Identifying Firewalls
 Understanding how to identify firewalls is crucial for ethical
hackers, as it enables them to detect and potentially exploit
vulnerabilities.
 The three primary methods for identifying firewalls are:
• Port Scanning: Detects open ports and services running on a firewall.
• Firewalking: Traces the firewall’s rule set by sending packets with varying
TTL (Time-to-Live) values.
• Banner Grabbing: Extracts information from service banners to identify the
firewall and its version.

Sarvajanik College of Engineering & Technology 33

Firewalls : Identifying Firewalls : Port Scanning
 Port scanning is a widely-used technique for identifying firewalls
and understanding their rule sets.
 Many firewalls have specific ports open, which can help in their
identification.
 Examples include:
• Microsoft Proxy Server: Ports 1080 and 1745.
• NetGuard GuardianPro Firewall: Ports TCP 1500 and UDP 1501.
• Check Point FireWall-1: Ports 256, 257, and 258.

Sarvajanik College of Engineering & Technology 34

Firewalls : Identifying Firewalls : Port Scanning : Traceroute

 Traceroute can also be an effective tool for identifying firewalls.

When used with the -I option in Linux, it sends ICMP packets
instead of UDP packets.
 While not entirely reliable, it helps identify the last hop that
responds, potentially indicating the presence of a firewall or
packet filter.

Sarvajanik College of Engineering & Technology 35

Firewalls : Identifying Firewalls : Port Scanning : Hping

 Hping is a versatile tool for identifying firewalls and internal

clients, offering greater control and flexibility compared to
standard tools.
 Key features include:
• Protocol Flexibility: Supports ICMP, UDP, and TCP, making it useful for various
testing scenarios.
• Firewall and IDS Testing: Can test firewall rules, perform idle scans, and test
Intrusion Detection Systems (IDS).
• Traceroute Bypassing: Effective for tracerouting hosts behind firewalls that
block standard traceroute attempts.
• TCP Verification: Can verify if a host is up even when ICMP packets are blocked.
• Comparison with Netcat: Unlike Netcat, which controls the data portion, Hping
focuses on the packet header, providing low-level control.

Sarvajanik College of Engineering & Technology 36

Firewalls : Identifying Firewalls : Firewalking
 Firewalking is a tool used to discover and enumerate firewalls by
sending packets with a TTL value set to expire one hop beyond the
firewall.
 Key aspects include:
• How It Works: If the firewall forwards the packet, it will expire at the next
hop, triggering an ICMP "TTL expired in transit" message. If blocked, there
will be no response or an "ICMP administratively prohibited" message.
• Requirements: Requires the IP address of the last gateway before the
firewall and an IP address of a host behind the firewall.
• Limitations: Ineffective if the firewall blocks ICMP packets from leaving the
network.

Sarvajanik College of Engineering & Technology 37

 Firewalls : Identifying Firewalls : Firewalking : Example
 In this Firewalking example, the
target is router 3, identified as the
edge device.
 The goal is to determine which
ports it allows or blocks.
 The steps include:
• Hopcount Ramping:
• Firewalk sends packets
with incrementing TTL
values (e.g., TTL=1, 2, 3)
until the target router is
reached.
• Once router 3 is reached at
TTL=3, all subsequent
packets use TTL=4.

Sarvajanik College of Engineering & Technology 38

Firewalls : Identifying Firewalls : Firewalking : Example

• Firewalking:
• TCP or UDP packets with
TTL=4 are sent past router
3.
• If the packet reaches its
destination, an ICMP TTL
type 11 message is
generated.
• If router 3 blocks the
packets, no response is
received, indicating
blocked ports.

Sarvajanik College of Engineering & Technology 39

Firewalls : Identifying Firewalls : Banner Grabbing
 Banner grabbing is a widely-used enumeration technique that
helps attackers gather information to compromise a targeted
network.
 Key points include:
• Targeted Services: Common services that send banners include FTP, Telnet,
and web services.
• Simplicity: No specialized tools are required; attackers can simply use Telnet
to connect to the IP address and specify the port to retrieve the banner.
• Purpose: The information obtained through banners can aid in identifying
vulnerabilities and planning further attacks.

Sarvajanik College of Engineering & Technology 40

Firewalls : Identifying Firewalls : Banner Grabbing : Example

 Example 1: Eagle Raptor Firewall

• Command: telnet 192.168.13.254 21
• Output: "220 Secure Gateway FTP server ready" indicates an open FTP
service on the firewall.
 Example 2: Cisco Router
• Command: telnet 192.168.13.1
• Output: Provides details such as "Connected to router1" and prompts for a
username, suggesting Telnet or SSH may be available for management.
 Security Concerns:
• Telnet Vulnerabilities: Telnet is insecure and susceptible to sniffing and
password guessing.
• Best Practices: If Telnet is used, restrict access with an access list to secure
virtual terminal (vty) lines.

Sarvajanik College of Engineering & Technology 41

Firewalls : Bypassing Firewalls
 No Universal Bypass
• Firewalls can be defeated due to misconfigurations or overly permissive
ACLs. Attackers often obscure or tunnel malicious traffic to resemble normal
network traffic.
 Tunneling Techniques:
• Internet Layer:
• IPv6: Often overlooked by firewalls and IDS due to incomplete support.
• ICMP: Can be used for tunneling, e.g., with tools like ICMPSend and Loki.
• Transport Layer:
• TCP/UDP Manipulation: Tools like AckCmd and UDPTunnel exploit TCP ACK
segments and UDP ports like 53 (DNS).
• Application Layer:
• SSH/HTTP Tunneling: SSH (port 22) and HTTP (port 80) tunnels, including HTTPS
with tools like Cryptcat, are effective due to their encryption.

Sarvajanik College of Engineering & Technology 42

Firewalls : Bypassing Firewalls
 Firewall Limitations:
• Secondary Connections: Unsecured wireless access or remote connections
bypass firewalls.
• Proxy Servers: Used to bypass restrictions.
• Social Engineering: Firewalls cannot protect against social tactics.
• Physical Security & Misconfiguration: Poorly configured firewalls or
physical breaches render defenses ineffective.
• Insider Threats: Firewalls are typically edge-focused and cannot prevent
internal attacks.
 Obfuscation Techniques:
• URL Encoding: Attackers can obscure target addresses in URLs using
hexadecimal or binary representations to bypass filters.
• Example: IP 192.168.13.10 can be represented as http://3232238858.

Sarvajanik College of Engineering & Technology 43

Honeypots

Sarvajanik College of Engineering & Technology 44

Honeypots
 Just as honey attracts bears, a honeypot is designed to attract
hackers.
 Purpose of Honeypots:
• Provide early warning of attacks.
• Track attacker activity and keystrokes.
• Enhance knowledge of hacking methods.
• Distract attackers from real networks.
 Honeypot Configuration:
• Appears to be a valuable network component but is isolated and protected.
• Can be a single computer or a network of honeypots (Honeynet).
• Key concept: Data Control—prevent attackers from using the honeypot to
attack real systems.

Sarvajanik College of Engineering & Technology 45

Honeypots
 Types of Honeypots:
• Low-Interaction:
• Emulates services; generates errors if unexpected actions occur.
• Example: Netcat running on port 80, showing it as open but unresponsive to
banner-grabbing tools.
• High-Interaction:
• Emulates entire systems or networks; allows interaction with seemingly real
applications.
 Examples of Honeypots:
• KFSensor, NetBait, Specter, Honeyd, Tiny Honeypot.
• Sticky Honeypots like LaBrea Tarpit slow down or prevent malicious activity.

Sarvajanik College of Engineering & Technology 47

Honeypots
 Challenges and Considerations:
• Risk of attackers breaking free and using the honeypot to attack others.
• High time and effort for setup, configuration, and monitoring.
• Attackers may detect and avoid honeypots by probing services (e.g., SSL
handshake).
 Tools for Detecting Honeypots:
• THC-Amap, Send-Safe Honeypot Hunter, Hping, Nessus.
• Nessus can craft SSL responses to probe services like HTTPS, SMPTS, IMAPS.

Sarvajanik College of Engineering & Technology 48

THANK YOU