Chapter Three

Program Security

Computer Networks and Information

Security
(CE 211)
Objectives
 Programming errors with security implications- buffer

overflows, incomplete access control

 Malicious Codes: viruses, worms, Trojan Horses

 Program Development controls against malicious code and

vulnerabilities- software engineering principles and practices.

 Controls to protect against program flaws in execution –

opertaing system support and administrative controls.

2
 Security: When is it software problem
 We can distinguish security problems by the mechanisms requiring changes
to eliminate the vulnerability.
1. Network Problem: requires changing networking mechanisms such as
network protocols.
2. OS Problem: requires changing OS mechanisms such OS resource
management policies.
3. Software Problem: requires changing software implementation or design
 It may be a hybrid problem requiring more than one type of change.
 A change in a mechanism such as protocols may be implemented in software
or hardware.
 It may be possible to eliminate a problem by making a fundamental change
3 in the processor hardware such as the stack mechanism for implementing
 Program Security
 Program security implies some degree of trust that the program

enforces expected level of confidentiality, integrity and

availability.
 Program security characteristics depends on the application and

user’s perception about the software quality:

 One person may decide that code is secure if it has run for a

period of time with no apparent failure.

 And someone may decide that a code is secure because it takes

too long to break through its security controls.

4  A third person may decide that any potential fault in meeting

 Contd.
 If the quality of a software is only about adhering to standards:

 Can be achieved by making the code secure

 Having conventional security approaches (locks in IBM

machines)

 An assessment of security can also be influenced by someone’s

general perspective on software quality.

 Program security should be from sound requirement analysis

to installation & maintenance

 And making sure the program

 Do what it is supposed to do
5
 Contd.
 The quality of program security can be judged using:

1. Fixing Faults:- What the fault is, its causes and the effects of the
fault.

 Fixing Faults can be done using:

A. penetrate and patch in which analysts searched for and

repaired faults.
 Patches Introduces more problems
 Patches cause side effects
 May also affect the non-functional requirements

6
 The patch techniques were largely useless, because they
 Contd.
B. Tiger team would be convened to test a system’s security by

attempting to cause it to fail.

 If the system withstood the attacks, it was considered secured.

2. Unexpected Behavior.:- compare the requirements with the

behavior.

 That is, to understand the program security we can examine

programs to see whether they behave as their designers intended

or users expected.

 Such an expected behavior of a program is know as program

7 security flaw: it is inappropriate program behavior caused

 Contd.
 There are techniques to eliminate or address all program security

flaws.
 There are two reasons for this distressing situation:

1. Program controls apply at the level of the individual program

and programmer

2. Programming and software engineering techniques change and

evolve far more rapidly than do computer security techniques.
 A flaw can be either a fault or failure and vulnerability usually

describes a class of flaws, such as a buffer overflow.

 Program security flaws can derive from any kind of software fault.
8
That is, they cover everything from a misunderstanding of
 Program Flaws

 Taxonomy of flaws:

1. how (genesis)

2. when (time)

3. where (location)

the flaw was introduced into the system

9
 Security Flaws by Genesis
1. Unintentional human errors(Inadvertent)
A. Validation errors(incomplete or inconsistent)
B. Domain Errors
C. Boundary condition violation
D. Inadequate identification & authentication
E. Serialization and aliasing.
F. Other exploitable logic errors

2. Malicious and Intentionally induced errors

A. Malicious flaws: Trojan Horse, Trapdoor, Logic Bomb, Worms,
Virus
10
B. Non-Malicious errors
 Flaws by time

 Time of introduction

1. During development

A. Requirement/specification/design

B. Source code

C. Object code

2. During maintenance

3. During operation

11
 Flaws by Location

1. Software

 Operating system: system initialization, memory management, process

management, device management, file management,

identification/authentication, other

 Support: privileged utilities, unprivileged utilities

 Application

2. Hardware

12
 Unintentional human errors
 Programmers and other developers make many mistakes, most of

which are unintentional and non-malicious.

 Most of such errors cause program malfunctions but do not lead

to more serious security vulnerabilities.

1. Buffer Overflows
 A buffer(or array or string) is a space in which data can be held.

 A buffer resides in memory.

 Because memory is finite, buffer’s capacity is also finite.

 For this reason, in most programs, the programmer needs to set the

required buffer size so that the compiler can set aside that amount
13
of space.
 Contd.
 Lets use the following example to see how a buffer overflow can

happen:
char sample[10];  All programs and data elements are

for(int i=0; i<10; i++) in memory during execution.

sample[i]=‘A’;  Sharing space with the operating
sample[10]=‘B’;
system, other code, and resident

User’s routines.
Data

Memory A A A A A A A A A A B

A. Affects User’s
 If the extra character
data overflows into the user’s data space, it simply
overwrites an existing variable value(or it may be written into as-
14
yet unused space). Affects the program’s result.
 Contd.
User’s Program
User’s Data
Data

Memory A A A A A A A A A A B

B. Affects User’s code

User’s Data System Data

Memory A A A A A A A A A A B

C. Affects System Data

System Program
User’s Data
Data

Memory A A A A A A A A A A B

15 D. Affects Systems Code

 Contd.
 In the second case, the ‘B’ goes into the user’s program area.

 If it overlays an already executed instruction which will not

be executed again), the user should perceive no effects.

 If it overlaps an instruction that is not yet executed, the

machine will try to execute an instruction with operation code

0x42, the internal code for the character ‘B’.
 If there is no instruction with operation code 0x42, the system

will halt on an illegal instruction execution.

 Otherwise, the machine will use subsequent bytes as if they

were the rest of the instruction, with success or failure

16
depending on the meaning of the content.
 Time-of-Check to Time-of-Use Errors
 In software development, time of check to time of use

(TOCTTOU, pronounced "TOCK too") is a class of software bug

caused by changes in a system between the checking of a
condition (such as a security credential) and the use of the results
of that check.
 It is a kind of race condition.

 Access control is a fundamental part of computer security;

 Make sure that only those who should access an object are allowed
that access.
 Every requested access must be governed by an access policy stating
who is allowed access to what: then the request must be mediated by
17
an access policy enforcement agent.
 Contd.
 The time of check to time of use (TOCTTOU) flaw concerns
mediation that is performed with a “bait and switch” in the
middle.
 It is also know as a serialization or synchronization flaw.
 A simple example of serialization flaws is as follows:
 Consider a Web application that allows a user to edit pages, and
also allows administrators to lock pages to prevent editing.
 A user requests to edit a page, getting a form by which he can alter
its content. Before the user submits the form, an administrator
locks the page, which should prevent editing.
 However, since the user has already begun editing, when he
submits the form, his edits are accepted. When the user began
editing, his authorization was checked, and he was indeed allowed
18
to edit.
 Malicious (Rogue) Program Errors
 Is the general name for unanticipated or undesired effects in

programs or program parts, caused by an agent intent on damage.

 The agent is the writer of the program or the person who causes

its distribution.
 By this definition, most faults found in software inspection,

reviews and testing do not qualify as a malicious codes, because

we think of them as unintentional faults.

 Unintentional faults can invoke the same response as intentional

faults.
19
 A benign cause can still lead to a disastrous effect.
 Contd.
 Here are some of the malicious attack types:-

1. Viruses:- is a program that can pass on malicious code to other non-

malicious programs by modifying them
 Attackers can develop viruses, and plant them into systems.

 Viruses can also be spread via e-mail and disks.

 A virus can be either transient or resident virus.

1. Transient Virus:- has a life that depends on the life cycle of its host:
the virus runs when its attached program executes and terminates
when its attached program ends.
2. Resident Virus:- locates itself in the memory: then it can remain
20 active or be activated as a stand-alone program, even after its
 Contd.
2. Trojan horses:- are malicious programs or software code hidden
inside what looks like a normal program.
 When a user runs the normal program, the hidden code runs as

well.
 It can then start deleting files and causing other damage to the

computer.
 Trojan horses are normally spread by e-mail attachments.

3. Worms:- are programs that copy themselves from one system

to another over a network, without the assistance of a human
being.
21  Worms usually propagate themselves by transferring from
 Contd.
5. Logic Bomb, time bomb:- programmed threats that lie dormant for
an extended period of time until they are triggered.
 When triggered, malicious code is executed.

6. Trapdoor or backdoor:- secret, undocumented entry point into a

program, used to grant access without normal methods of access
authentication.

7. Dropper:- Not a virus or infected file. When executed, it installs a

virus into memory, on to the disk, or into a file.

8. Rabbit:- is a self-replicating malicious code which replicates itself

without a bound, with the intention of exhausting some computing
22
resources.
 Types of Malicious Codes
Code Type Characteristics

Virus Attaches itself to program and propagates copies of itself

to other programs

Trojan Horse Contains unexpected , additional functionality

Logic Bomb Triggers action when specific condition occurs

Time Bomb Triggers action when specified time occurs

Trapdoor Allows unauthorized access to functionality

Worm Propagates copies of itself through the network

Rabbit Replicates it self without limit to exhaust resource.

23
 Virus Life cycle

1. Dormant phase: the virus is idle. (not all viruses have this stage)

2. Propagation phase: the virus places an identical copy of itself into

other programs of into certain system areas.

3. Triggering phase: the virus is activated to perform the function for

which it was created.

4. Execution phase: the function is performed. The function may be

harmless or damaging.

24
 Virus Types
1. Parasitic virus: most common form.

 Attaches itself to a file and replicates when the infected program is

executed.

2. Memory resident virus: lodged in main memory as part of a resident

system program.

 Virus may infect every program that executes.

3. Stealth virus: a form of virus explicitly designed to hide from detection

by antivirus software.

4. Polymorphic virus: a virus that mutates with every infection making

25 detection by the “signature” of the virus difficult.

 Contd.
5. Boot Sector Viruses:
 Infects the boot record and spreads when system is booted.

 Gains control of machine before the virus detection tools.

 Very hard to notice

 Carrier files: AUTOEXEC.BAT, CONFIG.SYS,IO.SYS

26
 How Viruses Append
1. Appended Viruses:- a program virus attaches itself to a program: then ,

whenever the program is run, the virus is activated.

 This kind of attachment is easy to program.

 In the simplest case, a virus inserts a copy of itself into the executable

program file before the first executable instruction.

 Then, all the virus instructions execute first.

 This kind of attachment is simple and usually effective.

 The virus writer does not need to know anything about the program to
27
which the virus will be attached.
 Contd.
 The virus performs its task and transfers to the original program.

 The user is unaware of the effect of the virus if the original program

still does all that is used.

 Most viruses attach in this manner.

+ virus = virus
Original
program
Original
program

28
Fig. Virus appended to program
 Contd.
2. Virus that surround a Program:- an alternative the attachment is a

virus runs the original program but has control before and after its

execution. + virus = Virus-1

Original
program
Original
program

Virus-2

29
Fig. Virus surrounding program
 Contd.
3. Integrated Virus:- occurs when a virus replaces some program

instructions by integrating itself into the original code of the target.

 The virus writer has to know the exact structure of the original

program to know where to insert which pieces of the virus.

+ virus = Virus-1
Original Original
program program
Virus-2
Original
program
Fig. Virus integrated to the program Virus-3
30
How Viruses Gain Control
 Virus V has to be invoked instead of target T.

 V overwrites T

 V changes pointers from T to V

 High risk virus properties:

 Hard to detect

 Hard to destroy

 Spread infection widely

 Can re-infect

 Easy to create

31  Machine independent
 Virus Signatures
1. Storage pattern
Code always located on a specific address

Increased file size

2. Execution pattern

3. Transmission pattern

4. Polymorphic Viruses

32
Antivirus Approaches
1. Detection: determine infection and locate the virus.

2. Identification: identify the specific virus.

3. Removal: remove the virus from all infected systems, so

the disease cannot spread further.

4. Recovery: restore the system to its original state.

33
 Preventing Virus Infection
Prevention:
Good source of software installed

Isolated testing phase

Use virus detectors

Limit damage:
Make bootable diskette

Make and retain backup copies important resources

34
 Worm
 Self-replicating (like virus)

 Objective: system penetration (intruder)

 Phases: dormant, propagation, triggering, and execution

 Propagation:
 Searches for other systems to infect (e.g., host tables)

 Establishes connection with remote system

 Copies itself to remote system

 Execute

35
 Covert Channel - Trojan Horse

Only John
John MS Word
is permitted
to access
the document
Document TH

install
copy

Spy’s
Spy Document
copy

36
Covert Channel
Need:
 Two active agents

Sender (has access to unauthorized information) – e.g., TH in MS

Word
Receiver ( reads sent information) – e.g., program creating the copy

 Encoding schema

How the information is sent – e.g.,

File F exists  0
File F is does not exist  1
 Synchronization – e.g., when to check for existence of F

37
Storage Covert Channels
 Based on properties of resources

 Examples:

File locks

Delete/create file

Memory allocation
Time Covert Channels
Time is the factor – how fast

Examples:

Processing time

38 Transmission time
 Covert Channel Detection and Removal

Identification:
Shared resources

Program code correctness

Information flow analysis

Removal:
Total removal – may not be possible

Reduce bandwidth

39