Chapter 15: Security

 The Security Problem

 Program Threats
 System and Network Threats

Chapter 15: Security 


Cryptography as a Security Tool
User Authentication
 Implementing Security Defenses
 Firewalling to Protect Systems and Networks
 Computer-Security Classifications
 An Example: Windows 7

Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.2 Silberschatz, Galvin and Gagne ©2013

Objectives The Security Problem

 To discuss security threats and attacks  System secure if resources used and accessed as intended
under all circumstances
 To explain the fundamentals of encryption, authentication, and
hashing  Unachievable
 To examine the uses of cryptography in computing  Intruders (crackers) attempt to breach security
 To describe the various countermeasures to security attacks  Threat is potential security violation
 Attack is attempt to breach security
 Attack can be accidental or malicious
 Easier to protect against accidental than malicious misuse

Operating System Concepts – 9th Edition 15.3 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.4 Silberschatz, Galvin and Gagne ©2013
 Security Violation Categories Security Violation Methods
 Breach of confidentiality  Masquerading (breach authentication)
 Unauthorized reading of data  Pretending to be an authorized user to escalate privileges
 Breach of integrity  Replay attack
 Unauthorized modification of data  As is or with message modification
 Breach of availability  Man-in-the-middle attack
 Unauthorized destruction of data  Intruder sits in data flow, masquerading as sender to receiver
 Theft of service and vice versa

 Unauthorized use of resources  Session hijacking

 Denial of service (DOS)  Intercept an already-established session to bypass

authentication
 Prevention of legitimate use

Operating System Concepts – 9th Edition 15.5 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.6 Silberschatz, Galvin and Gagne ©2013

Standard Security Attacks Security Measure Levels

 Impossible to have absolute security, but make cost to
perpetrator sufficiently high to deter most intruders
 Security must occur at four levels to be effective:
 Physical
 Data centers, servers, connected terminals
 Human
 Avoid social engineering, phishing, dumpster diving
 Operating System
 Protection mechanisms, debugging
 Network
 Intercepted communications, interruption, DOS
 Security is as weak as the weakest link in the chain
 But can too much security be a problem?

Operating System Concepts – 9th Edition 15.7 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.8 Silberschatz, Galvin and Gagne ©2013
 Program Threats Program Threats (Cont.)
 Many variations, many names  Logic Bomb
 Trojan Horse  Program that initiates a security incident under certain
 Code segment that misuses its environment circumstances
 Stack and Buffer Overflow
 Exploits mechanisms for allowing programs written by users to be
executed by other users  Exploits a bug in a program (overflow either the stack or
 Spyware, pop-up browser windows, covert channels memory buffers)

 Up to 80% of spam delivered by spyware-infected systems  Failure to check bounds on inputs, arguments

 Trap Door  Write past arguments on the stack into the return address
on stack
 Specific user identifier or password that circumvents normal
security procedures  When routine returns from call, returns to hacked address

 Could be included in a compiler  Pointed to code loaded onto stack that executes
malicious code
 How to detect them?
 Unauthorized user or privilege escalation

Operating System Concepts – 9th Edition 15.9 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.10 Silberschatz, Galvin and Gagne ©2013

C Program with Buffer-overflow Condition Layout of Typical Stack Frame

#include <stdio.h>
#define BUFFER SIZE 256
int main(int argc, char *argv[])
{
char buffer[BUFFER SIZE];
if (argc < 2)
return -1;
else {
strcpy(buffer,argv[1]);
return 0;
}
}

Operating System Concepts – 9th Edition 15.11 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.12 Silberschatz, Galvin and Gagne ©2013
 Modified Shell Code Hypothetical Stack Frame
#include <stdio.h>
int main(int argc, char *argv[])
{
execvp(‘‘\bin\sh’’,‘‘\bin \sh’’, NULL);
return 0;
}

Before attack After attack

Operating System Concepts – 9th Edition 15.13 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.14 Silberschatz, Galvin and Gagne ©2013

Great Programming Required? Program Threats (Cont.)

 For the first step of determining the bug, and second step of  Viruses
writing exploit code, yes
 Code fragment embedded in legitimate program
 Script kiddies can run pre-written exploit code to attack a given
 Self-replicating, designed to infect other computers
system
 Very specific to CPU architecture, operating system, applications
 Attack code can get a shell with the processes’ owner’s
permissions  Usually borne via email or as a macro
 Or open a network port, delete files, download a program, etc  Visual Basic Macro to reformat hard drive
 Depending on bug, attack can be executed across a network Sub AutoOpen()
using allowed connections, bypassing firewalls Dim oFS

 Buffer overflow can be disabled by disabling stack execution or Set oFS = CreateObject(’’Scripting.FileSystemObject’’)
adding bit to page table to indicate “non-executable” state vs = Shell(’’c:command.com /k format c:’’,vbHide)
 Available in SPARC and x86 End Sub

 But still have security exploits

Operating System Concepts – 9th Edition 15.15 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.16 Silberschatz, Galvin and Gagne ©2013
 Program Threats (Cont.) A Boot-sector Computer Virus
 Virus dropper inserts virus onto the system
 Many categories of viruses, literally many thousands of viruses
 File / parasitic
 Boot / memory
 Macro
 Source code
 Polymorphic to avoid having a virus signature
 Encrypted
 Stealth
 Tunneling
 Multipartite
 Armored

Operating System Concepts – 9th Edition 15.17 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.18 Silberschatz, Galvin and Gagne ©2013

The Threat Continues System and Network Threats

 Attacks still common, still occurring  Some systems “open” rather than secure by default
 Attacks moved over time from science experiments to tools of  Reduce attack surface
organized crime
 But harder to use, more knowledge needed to administer
 Targeting specific companies
 Network threats harder to detect, prevent
 Creating botnets to use as tool for spam and DDOS delivery
 Protection systems weaker
 Keystroke logger to grab passwords, credit card numbers
 More difficult to have a shared secret on which to base access
 Why is Windows the target for most attacks?
 No physical limits once system attached to internet
 Most common
 Or on network with system attached to internet
 Everyone is an administrator
 Even determining location of connecting system difficult
 Licensing required?
 IP address is only knowledge
 Monoculture considered harmful

Operating System Concepts – 9th Edition 15.19 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.20 Silberschatz, Galvin and Gagne ©2013
 System and Network Threats (Cont.) The Morris Internet Worm
 Worms – use spawn mechanism; standalone program
 Internet worm
 Exploited UNIX networking features (remote access) and bugs
in finger and sendmail programs
 Exploited trust-relationship mechanism used by rsh to access
friendly systems without use of password
 Grappling hook program uploaded main worm program
 99 lines of C code
 Hooked system then uploaded main code, tried to attack
connected systems
 Also tried to break into other users accounts on local system via
password guessing
 If target system already infected, abort, except for every 7th time

Operating System Concepts – 9th Edition 15.21 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.22 Silberschatz, Galvin and Gagne ©2013

System and Network Threats (Cont.) System and Network Threats (Cont.)

 Port scanning  Denial of Service

 Automated attempt to connect to a range of ports on one  Overload the targeted computer preventing it from doing any
or a range of IP addresses useful work
 Detection of answering service protocol  Distributed denial-of-service (DDOS) come from multiple
 Detection of OS and version running on system sites at once
 nmap scans all ports in a given IP range for a response  Consider the start of the IP-connection handshake (SYN)
 nessus has a database of protocols and bugs (and  How many started-connections can the OS handle?
exploits) to apply against a system  Consider traffic to a web site
 Frequently launched from zombie systems  How can you tell the difference between being a target
 To decrease trace-ability and being really popular?
 Accidental – CS students writing bad fork() code
 Purposeful – extortion, punishment

Operating System Concepts – 9th Edition 15.23 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.24 Silberschatz, Galvin and Gagne ©2013
 Sobig.F Worm Cryptography as a Security Tool
 More modern example  Broadest security tool available
 Disguised as a photo uploaded to adult newsgroup via account  Internal to a given computer, source and destination of
created with stolen credit card messages can be known and protected
 Targeted Windows systems  OS creates, manages, protects process IDs,
 Had own SMTP engine to mail itself as attachment to everyone in communication ports
infect system’s address book  Source and destination of messages on network cannot be
 Disguised with innocuous subject lines, looking like it came from trusted without cryptography
someone known  Local network – IP address?
 Attachment was executable program that created WINPPR23.EXE – Consider unauthorized host added
in default Windows system directory  WAN / Internet – how to establish authenticity
Plus the Windows Registry
– Not via IP address
[HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Run]
"TrayX" = %windir%\winppr32.exe /sinc
[HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run]
"TrayX" = %windir%\winppr32.exe /sinc

Operating System Concepts – 9th Edition 15.25 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.26 Silberschatz, Galvin and Gagne ©2013

Cryptography Encryption

 Means to constrain potential senders (sources) and / or  Constrains the set of possible receivers of a message
receivers (destinations) of messages  Encryption algorithm consists of
 Based on secrets (keys)  Set K of keys

 Enables  Set M of Messages

 Set C of ciphertexts (encrypted messages)
 Confirmation of source
 A function E : K → (M→C). That is, for each k  K, Ek is a
 Receipt only by certain destination
function for generating ciphertexts from messages
 Trust relationship between sender and receiver  Both E and Ek for any k should be efficiently computable
functions
 A function D : K → (C → M). That is, for each k  K, Dk is a
function for generating messages from ciphertexts
 Both D and Dk for any k should be efficiently computable
functions

Operating System Concepts – 9th Edition 15.27 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.28 Silberschatz, Galvin and Gagne ©2013
 Encryption (Cont.) Symmetric Encryption
 An encryption algorithm must provide this essential property:  Same key used to encrypt and decrypt
Given a ciphertext c  C, a computer can compute m such  Therefore k must be kept secret
that Ek(m) = c only if it possesses k
 DES was most commonly used symmetric block-encryption algorithm (created
 Thus, a computer holding k can decrypt ciphertexts to by US Govt)
the plaintexts used to produce them, but a computer not
 Encrypts a block of data at a time
holding k cannot decrypt ciphertexts
 Keys too short so now considered insecure
 Since ciphertexts are generally exposed (for example,
sent on the network), it is important that it be infeasible  Triple-DES considered more secure
to derive k from the ciphertexts  Algorithm used 3 times using 2 or 3 keys
 For example
 2001 NIST adopted new block cipher - Advanced Encryption Standard (AES)
 Keys of 128, 192, or 256 bits, works on 128 bit blocks
 RC4 is most common symmetric stream cipher, but known to have
vulnerabilities
 Encrypts/decrypts a stream of bytes (i.e., wireless transmission)
 Key is a input to pseudo-random-bit generator
 Generates an infinite keystream

Operating System Concepts – 9th Edition 15.29 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.30 Silberschatz, Galvin and Gagne ©2013

Secure Communication over Insecure Medium Asymmetric Encryption

 Public-key encryption based on each user having two keys:

 public key – published key used to encrypt data
 private key – key known only to individual user used to
decrypt data
 Must be an encryption scheme that can be made public
without making it easy to figure out the decryption scheme
 Most common is RSA block cipher
 Efficient algorithm for testing whether or not a number is
prime
 No efficient algorithm is know for finding the prime factors
of a number

Operating System Concepts – 9th Edition 15.31 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.32 Silberschatz, Galvin and Gagne ©2013
 Asymmetric Encryption (Cont.) Asymmetric Encryption Example

 Formally, it is computationally infeasible to derive kd,N from  For example. make p = 7and q = 13
ke,N, and so ke need not be kept secret and can be widely  We then calculate N = 7∗13 = 91 and (p−1)(q−1) = 72
disseminated
 We next select ke relatively prime to 72 and< 72, yielding 5
 ke is the public key
 Finally, we calculate kd such that kekd mod 72 = 1, yielding 29
 kd is the private key
 We how have our keys
 N is the product of two large, randomly chosen prime
numbers p and q (for example, p and q are 512 bits each)  Public key, ke,N = 5, 91

 Encryption algorithm is Eke,N(m) = mke mod N, where ke  Private key, kd,N = 29, 91
satisfies kekd mod (p−1)(q −1) = 1  Encrypting the message 69 with the public key results in the
 The decryption algorithm is then Dkd,N(c) = ckd mod N cyphertext 62
 Cyphertext can be decoded with the private key
 Public key can be distributed in cleartext to anyone who
wants to communicate with holder of public key

Operating System Concepts – 9th Edition 15.33 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.34 Silberschatz, Galvin and Gagne ©2013

Encryption using RSA Asymmetric Cryptography Cryptography (Cont.)

 Note symmetric cryptography based on transformations,

asymmetric based on mathematical functions
 Asymmetric much more compute intensive
 Typically not used for bulk data encryption

Operating System Concepts – 9th Edition 15.35 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.36 Silberschatz, Galvin and Gagne ©2013
 Authentication Authentication (Cont.)
 Constraining set of potential senders of a message  For a message m, a computer can generate an authenticator
 Complementary to encryption a  A such that Vk(m, a) = true only if it possesses k
 Also can prove message unmodified  Thus, computer holding k can generate authenticators on
 Algorithm components messages so that any other computer possessing k can
 A set K of keys verify them
 A set M of messages  Computer not holding k cannot generate authenticators on
messages that can be verified using Vk
 A set A of authenticators
 Since authenticators are generally exposed (for example,
 A function S : K → (M→ A)
they are sent on the network with the messages themselves),
That is, for each k  K, Sk is a function for generating

it must not be feasible to derive k from the authenticators
authenticators from messages
 Practically, if Vk(m,a) = true then we know m has not been
 Both S and Sk for any k should be efficiently computable
functions modified and that send of message has k
 A function V : K → (M × A→ {true, false}). That is, for each k  K,  If we share k with only one entity, know where the
Vk is a function for verifying authenticators on messages message originated
 Both V and Vk for any k should be efficiently computable
functions

Operating System Concepts – 9th Edition 15.37 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.38 Silberschatz, Galvin and Gagne ©2013

Authentication – Hash Functions Authentication - MAC

 Basis of authentication  Symmetric encryption used in message-authentication code
 Creates small, fixed-size block of data message digest (hash (MAC) authentication algorithm
value) from m  Cryptographic checksum generated from message using secret
 Hash Function H must be collision resistant on m key
 Must be infeasible to find an m’ ≠ m such that H(m) = H(m’)  Can securely authenticate short values
 If H(m) = H(m’), then m = m’  If used to authenticate H(m) for an H that is collision resistant,
 The message has not been modified then obtain a way to securely authenticate long message by
 Common message-digest functions include MD5, which produces hashing them first
a 128-bit hash, and SHA-1, which outputs a 160-bit hash  Note that k is needed to compute both Sk and Vk, so anyone
 Not useful as authenticators able to compute one can compute the other

 For example H(m) can be sent with a message

 But if H is known someone could modify m to m’ and recompute
H(m’) and modification not detected
 So must authenticate H(m)

Operating System Concepts – 9th Edition 15.39 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.40 Silberschatz, Galvin and Gagne ©2013
 Authentication – Digital Signature Authentication (Cont.)
 Based on asymmetric keys and digital signature algorithm  Why authentication if a subset of encryption?
 Authenticators produced are digital signatures  Fewer computations (except for RSA digital signatures)
 Very useful – anyone can verify authenticity of a message  Authenticator usually shorter than message
 In a digital-signature algorithm, computationally infeasible to  Sometimes want authentication but not confidentiality
derive ks from kv
 Signed patches et al
 V is a one-way function
 Can be basis for non-repudiation
 Thus, kv is the public key and ks is the private key
 Consider the RSA digital-signature algorithm
 Similar to the RSA encryption algorithm, but the key use is
reversed
 Digital signature of message Sks (m) = H(m)ks mod N
 The key ks again is a pair (d, N), where N is the product of
two large, randomly chosen prime numbers p and q
 Verification algorithm is Vkv(m, a) (akv mod N = H(m))
 Where kv satisfies kvks mod (p − 1)(q − 1) = 1

Operating System Concepts – 9th Edition 15.41 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.42 Silberschatz, Galvin and Gagne ©2013

Key Distribution Digital Certificates

 Delivery of symmetric key is huge challenge  Proof of who or what owns a public key
 Sometimes done out-of-band  Public key digitally signed a trusted party
 Asymmetric keys can proliferate – stored on key ring  Trusted party receives proof of identification from entity and
certifies that public key belongs to entity
 Even asymmetric key distribution needs care – man-
in-the-middle attack  Certificate authority are trusted party – their public keys
included with web browser distributions
 They vouch for other authorities via digitally signing their
keys, and so on

Operating System Concepts – 9th Edition 15.43 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.44 Silberschatz, Galvin and Gagne ©2013
 Man-in-the-middle Attack on Asymmetric Cryptography Implementation of Cryptography

 Can be done at various

layers of ISO Reference
Model
 SSL at the Transport layer
 Network layer is typically
IPSec
 IKE for key exchange
 Basis of Virtual Private
Networks (VPNs)

 Why not just at lowest level? Source:

http://en.wikipedia.org/wiki/OSI_mo
 Sometimes need more del

knowledge than available at

low levels
 i.e. User authentication
 i.e. e-mail delivery

Operating System Concepts – 9th Edition 15.45 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.46 Silberschatz, Galvin and Gagne ©2013

Encryption Example - SSL User Authentication

 Insertion of cryptography at one layer of the ISO network model  Crucial to identify user correctly, as protection systems depend on user ID
(the transport layer)  User identity most often established through passwords, can be considered
 SSL – Secure Socket Layer (also called TLS) a special case of either keys or capabilities
 Passwords must be kept secret
 Cryptographic protocol that limits two computers to only exchange
messages with each other  Frequent change of passwords
 Very complicated, with many variations  History to avoid repeats
 Use of “non-guessable” passwords
 Used between web servers and browsers for secure
communication (credit card numbers)  Log all invalid access attempts (but not the passwords themselves)
 The server is verified with a certificate assuring client is talking to  Unauthorized transfer
correct server  Passwords may also either be encrypted or allowed to be used only once
 Asymmetric cryptography used to establish a secure session key  Does encrypting passwords solve the exposure problem?
(symmetric encryption) for bulk of communication during session  Might solve sniffing

 Communication between each computer then uses symmetric key  Consider shoulder surfing
cryptography  Consider Trojan horse keystroke logger
 More details in textbook  How are passwords stored at authenticating site?

Operating System Concepts – 9th Edition 15.47 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.48 Silberschatz, Galvin and Gagne ©2013
 Passwords Implementing Security Defenses
 Encrypt to avoid having to keep secret  Defense in depth is most common security theory – multiple layers of security
 But keep secret anyway (i.e. Unix uses superuser-only readably file  Security policy describes what is being secured
/etc/shadow)  Vulnerability assessment compares real state of system / network compared to
 Use algorithm easy to compute but difficult to invert security policy
 Only encrypted password stored, never decrypted  Intrusion detection endeavors to detect attempted or successful intrusions
 Add “salt” to avoid the same password being encrypted to the same value  Signature-based detection spots known bad patterns
 One-time passwords  Anomaly detection spots differences from normal behavior
 Use a function based on a seed to compute a password, both user and  Can detect zero-day attacks
computer  False-positives and false-negatives a problem
 Hardware device / calculator / key fob to generate the password  Virus protection
 Changes very frequently  Searching all programs or programs at execution for known virus patterns
 Biometrics  Or run in sandbox so can’t damage system
 Some physical attribute (fingerprint, hand scan)  Auditing, accounting, and logging of all or specific system or network activities
 Multi-factor authentication  Practice safe computing – avoid sources of infection, download from only
 Need two or more factors for authentication “good” sites, etc
 i.e. USB “dongle”, biometric measure, and password

Operating System Concepts – 9th Edition 15.49 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.50 Silberschatz, Galvin and Gagne ©2013

Firewalling to Protect Systems and Networks Network Security Through Domain Separation Via Firewall

 A network firewall is placed between trusted and untrusted hosts

 The firewall limits network access between these two security
domains
 Can be tunneled or spoofed
 Tunneling allows disallowed protocol to travel within allowed
protocol (i.e., telnet inside of HTTP)
 Firewall rules typically based on host name or IP address
which can be spoofed
 Personal firewall is software layer on given host
 Can monitor / limit traffic to and from the host
 Application proxy firewall understands application protocol and
can control them (i.e., SMTP)
 System-call firewall monitors all important system calls and apply
rules to them (i.e., this program can execute that system call)

Operating System Concepts – 9th Edition 15.51 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.52 Silberschatz, Galvin and Gagne ©2013
 Computer Security Classifications Example: Windows 7
 U.S. Department of Defense outlines four divisions of computer  Security is based on user accounts
security: A, B, C, and D  Each user has unique security ID
 D – Minimal security  Login to ID creates security access token
 C – Provides discretionary protection through auditing  Includes security ID for user, for user’s groups, and special
privileges
 Divided into C1 and C2
 Every process gets copy of token
 C1 identifies cooperating users with the same level of
 System checks token to determine if access allowed or denied
protection
 Uses a subject model to ensure access security
 C2 allows user-level access control
 A subject tracks and manages permissions for each program that a
 B – All the properties of C, however each object may have user runs
unique sensitivity labels  Each object in Windows has a security attribute defined by a security
 Divided into B1, B2, and B3 descriptor
 For example, a file has a security descriptor that indicates the
 A – Uses formal design and verification techniques to ensure
access permissions for all users
security

Operating System Concepts – 9th Edition 15.53 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition 15.54 Silberschatz, Galvin and Gagne ©2013

Example: Windows 7 (Cont.)

 Win added mandatory integrity controls – assigns integrity

label to each securable object and subject
 Subject must have access requested in discretionary
access-control list to gain access to object
 Security attributes described by security descriptor End of Chapter 15
 Owner ID, group security ID, discretionary access-control
list, system access-control list

Operating System Concepts – 9th Edition 15.55 Silberschatz, Galvin and Gagne ©2013 Operating System Concepts – 9th Edition Silberschatz, Galvin and Gagne ©2013