Intro to Program Security

Barbara Endicott-Popovsky
CSSE592/491

In collaboration with:
Deborah Frincke, Ph.D.
Director, Center for Secure and Dependable
Systems
University of Idaho
Goal of Computer Security

re
Confidentiality ec u
S

Integrity Availability

Source: Pfleeger & Pfleeger

Text Book
 Both broad survey and focused
 Chapters 1-2 lay groundwork
 Chapters 3 –7 Software
• Chapter 3
• Secure Programs
• Non-malicious Program Errors
• Viruses and other Malicious Code
• Targeted Malicious Code
• Controls Against Program Threats
 Chapter 8 Management
 Chapter 9 Privacy, ethics, the law
 Chapter 10 Cryptography – the how
In this section of the course we
will look at…

 The heart of security--programming

 What you, the developer, can do

• Keep programs free from flaws

• Protect computing resources from such flaws

Source: Pfleeger & Pfleeger

Agenda
 I. Overview
 II. Program Flaws
• Buffer overflow
• Incomplete mediation
• TOC/TOU
 III. Exploitation
• Anonymous Malicious—Viruses
• Targeted Malicious
 IV. Controls
• Developmental—Software Engineering
• Operating System
• Administrative
I. Overview
 2 Types of General Flaws
• Those that compromise data
• Those that affect service
 3 Types of Controls
• Development
• Operating system
• Administrative

Source: Pfleeger & Pfleeger

 Purpose and Benefit of Controls
Control Purpose Benefit

Development Limit mistakes Produce better

Make malicious software
code difficult
Operating Limit access to Promotes safe
System system sharing of info

Administrative Limit actions of Improve usability,

people reusability and
maintainability
Source: Pfleeger & Pfleeger
Assessment of Security

 Depends on characteristics
--What contributes to security?
 Influenced by perspective
--Conformance to specs?
 Based on quantity and types of faults:
--errors cause
--faults can result in (programmer’s view)
--system failures (user’s view)

Source: Pfleeger & Pfleeger

II. Program Flaws

 Fixing Faults—History

 Taxonomy of Program Flaws

 Types of Flaws
• Buffer overflow
• Incomplete mediation
• TOC/TOU
Fixing Faults--History
 Penetrate and patch
• “tiger teams”
• Fix has unintended consequences

 Discover unexpected program behavior

• Testing components/programs
• Can’t exhaustively test
• Technology changes too fast

 Computer security approach

• Analyze what can go wrong
• Protect against it
• Tools
• Techniques

Source: Pfleeger & Pfleeger

Taxonomy of Program Flaw’s
 Unintentional
• Validation error (incomplete or inconsistent)
• Domain error
• Serialization and aliasing
• Inadequate ID and authentication (Session 4)
• Boundary condition violation
• Other exploitable logic errors
 Intentional
• Malicious
• Nonmalicious

Source: Landwehr (1994)

Buffer Overflow Flaw:
Inadvertent/Nonmalicious
 Many languages require buffer size declaration
• C language statement char sample[10];
• Execute statement sample[10] = ‘A’
• Out of bounds subscript
• Some compilers don’t check for exceeding bounds
 Where does ‘A’ go
• Depends on what is adjacent
• Affects user’s data
• Affects users code
• Affects system data
• Affects system code

Source: Pfleeger & Pfleeger

Buffer Overflow Flaw:
Implications
 Insert malicious data values/instruction codes
 Buffer overflow  OS, replace procedures
(Mudge 1995)
• Gain control of OS
• Raise privileges
 Buffer overflow  the stack
• Change old stack pointer or address
• Redirect execution
 Variant--pass very long string to web server
 Buffer overflows now disastrous

Source: Pfleeger & Pfleeger

Incomplete Mediation Flaw
Inadvertent/Nonmalicious
 Sensitive data in exposed, uncontrolled condition

 Implications
Example:
http://www.things.com/order/final&custID=101&part=555A
&qy=20&price=10&ship=boat&shipcost=5&total=205
• Anticipate problems
• Drop-down box
• Test values
• Still exposed—Change price = 1, total = 25

Source: Pfleeger & Pfleeger

Time-of-Check, Time-of-Use Flaw
Inadvertent/Nonmalicious
 Synchronization--Mediation with “bait and switch”
 Implications
• Change occurs between time access checked and time
access used
 Prevention
• Be aware of time lags
• Use digital certificates

Source: Pfleeger & Pfleeger

III. Exploitation

 Anonymous Malicious—Viruses

 Targeted Malicious
Exploitation of Flaws:
Anonymous Malicious Code--Viruses

 Wide varieties
 Benign or harmful
 Not new
• Transferred from trusted sources
• Exploitation of flaws

Source: Pfleeger & Pfleeger

“Viruses” come in many forms
Code Type Characteristics
Virus Attaches to program, propagates copies of
itself to other programs
Trojan horse Contains unexpected, additional
functionality
Logic bomb Triggers action when condition occurs

Time bomb Triggers action when specified time occurs

Trapdoor Allows unauthorized access to functionality

Worm Propagates copies of itself thru a network

Rabbit Replicates itself without limit to exhaust

resource
Source: Pfleeger & Pfleeger
Virus Characteristics
 Hard to detect
 Not easily destroyed or deactivated
 Spreads infection widely
 Reinfect programs
 Easy to create
 Machine and OS independent

Source: Pfleeger & Pfleeger

How Viruses Work
 Attach
• Append to program, e-mail
• Executes with program
• Surrounds program
• Executes before and after program
• Erases its tracks
• Integrates or replaces program code
 Gain control
• Virus replaces target
 Reside
• In boot sector
• Memory
• Application program
• Libraries

Source: Pfleeger & Pfleeger

How Viruses Work (Cont’d.)
 Detection
• Virus signatures
• Storage patterns
• Execution patterns
• Transmission patterns
 Prevention
• Don’t share executables
• Use commercial software from reliable sources
• Test new software on isolated computers
• Open only safe attachments
• Keep recoverable system image in safe place
• Backup executable system file copies
• Use virus detectors
• Update virus detectors often

Source: Pfleeger & Pfleeger

Virus Cause /Effect Analysis
Virus Effect How it is caused
Modify file directory

Attach to executable Write to executable program file

Modify directory

Attach to data/control file Rewrite data

Append to data
Append data to self

Intercept interrupt by modifying interrupt handler address table

Remain in memory Load self in non-transient memory area

Intercept interrupt

Infect disks Intercept OS call (to format disk, for example)

Modify system file
Modify ordinary executable program

Intercept system calls that would reveal self and falsify results
Conceal self Classify self as “hidden” file

Infect boot sector

Spread self Infect systems program

Infect ordinary program
Infect data ordinary program reads to control its executable

Activate before deactivating program and block deactivation

Prevent deactivation Store copy to reinfect after deactivation

Source: Pfleeger & Pfleeger

7 Truths about Viruses

 Viruses can infect any platform

 Viruses can modify “hidden” / “read only files
 Viruses can appear anywhere in system
 Viruses spread anywhere sharing occurs
 Viruses cannot remain in memory
 Viruses infect software that runs hardware
 Viruses can be benevolent, malevolent and benign

Source: Pfleeger & Pfleeger

The Internet Worm
 Case Study on Web site:

• 1988
• Invaded VAX and Sun-3 computers running
versions of Berkeley UNIX
• Used their resources to attack still more computers
• Within hours spread across the U.S
• Infected hundreds / thousands of computers
• Made many computers unusable
Exploitation of Flaws:
Targeted Malicious Code
 Trapdoors
• Program stubs during testing
• Intentionally or unintentionally left
• Forgotten
• Left for testing or maintenance
• Left for covert access

 Salami attack
• Merges inconsequential pieces to get big results
• Ex: deliberate diversion of fractional cents
• Too difficult to audit

Source: Pfleeger & Pfleeger

Exploitation of Flaws:
Targeted Malicious Code (cont’d.)
 Covert Channels
• Programs that leak information
• Trojan horse
• Discovery
• Analyze system resources for patterns
• Flow analysis from a program’s syntax (automated)
• Difficult to close
• Not much documented
• Potential damage is extreme

Source: Pfleeger & Pfleeger

IV. Controls--Against Program Flaws

 Developmental—Software Engineering

 Operating System

 Administrative
Developmental Controls Against
Program Flaws:
Software Engineering Disciplines
 SDLC—Design, Define, Develop, Do
 Collaborative work teams
 Reviews
 Documentation
 Configuration Management
 Project Management

Source: Pfleeger & Pfleeger

Building Solid Software
 Peer reviews
 Hazard analysis
 Testing
 Good design
 Prediction
 Static analysis
 Configuration management
 Analysis of mistakes

Source: Pfleeger 2001

Good Design
 Modularity
• Single purpose
• Small
• Simple
• Independent (high cohesion, low coupling)
 Encapsulation
• Sharing minimized
• Interfaces well-documented
 Information hiding
• Limits impacts on other components
• Limits malicious acts

Source: Pfleeger & Pfleeger

Good Design (cont’d.)
 Using philosophy of fault tolerance
• Anticipate faults
• Use redundancy
• Isolate damage, minimize disruption
 Consistent policy for handling failures
• Retrying
• Correcting
• Reporting
 Capture design rationale and history
 Use design patterns
Value of Good Design

 Easy maintenance
 Understandability
 Reuse
 Correctness
 Testing

Source: Pfleeger & Pfleeger

Peer Reviews
 Reviews
• Informal
• Team of reviewers
• Gain consensus on solutions
 Walk-throughs
• Developer walks team through code
• Discover flaws
 Inspection
• Formalized
• Statistical measures

Source: Pfleeger & Pfleeger

Hazard Analysis
 Systematic exposure of hazardous system states
 Technique
• Hazard lists
• What-if scenarios
• System-wide view (not just code)
• Begins Day 1
• Continues throughout SDLC
 Tools
• HAZOP
• FMEA
• FTA

Source: Pfleeger & Pfleeger

Testing Plan—
Black or White Box
 Unit testing
 Integration testing
 Function testing
Before Delivery
 Performance testing
 Acceptance testing With Customer

 Installation testing
 Regression testing (after incorporating changes

Source: Pfleeger & Pfleeger

Configuration Management
 Reasons for software modification
• Corrective change
• Adaptive change
• Perfective change
• Preventive change
 Configuration management process
• Configuration identification--baselines
• Configuration control and management—version control
• Deltas
• Compilations
• Configuration auditing
• Status accounting
 Configuration Control Board
 Security benefits
• Prevents/limits malicious modification
• Archives changes

Source: Pfleeger & Pfleeger

Additional Development
Controls
 Formal methods—proof of program correctness
 Lesson learned
 Prediction
 Static Analysis

No guarantees!

Source: Pfleeger & Pfleeger

 Operating System Controls
Against Program Flaws:
 Trusted software
• Functional correctness
• Enforcement of integrity
• Limited privileges
• Appropriate confidence level
 Mutual suspicion—between programs
 Confinement—limit system resource access
 Access log
• Audit frequently
• Look for patterns

More later……. Source: Pfleeger & Pfleeger

Administrative Controls
Against Program Flaws:
 Standards of program development
 Codes of conduct
 Security audits
 Separation of duties

More later……. Source: Pfleeger & Pfleeger