CSCM28 – Security Vulnerabilities and Penetration Testing
Week 5 - Buffer Overflow
Jens Blanck
Jens Blanck CSCM28 Pen Testing 1 / 23
Recap and Today
Recap and Today
Recap Today
Last week we considered some ways of Today we look at security vulnerabilities:
hacking systems: Smashing the stack.
Password cracking
Gaining network access We will start with some background on
Rootkits executables, and debugging tools.
Jens Blanck CSCM28 Pen Testing 2 / 23
Executables
Minimal C Program
# include <s t d i o . h>
i n t main ()
{
p r i n t f ( ” Hello World ! \ n” ) ;
return 0 ;
}
Jens Blanck CSCM28 Pen Testing 3 / 23
Executables
Compiled Code and Data
Compiled code sits in memory as machine code:
Instructions are indistinguishable from data, essentially.
Instruction Pointer register holds address of next instruction.
If the IP points to user modifiable data, we can essentially trick the processor to
execute arbitrary code.
Processor architecture has tried to separate code from data through various schemes
of memory management. But there are loopholes in this.
Jens Blanck CSCM28 Pen Testing 4 / 23
Executables
Compiled Code and Data
Compiled code sits in memory as machine code:
Instructions are indistinguishable from data, essentially.
Instruction Pointer register holds address of next instruction.
If the IP points to user modifiable data, we can essentially trick the processor to
execute arbitrary code.
Processor architecture has tried to separate code from data through various schemes
of memory management. But there are loopholes in this.
Jens Blanck CSCM28 Pen Testing 4 / 23
Executables
Compiled Code and Data
Compiled code sits in memory as machine code:
Instructions are indistinguishable from data, essentially.
Instruction Pointer register holds address of next instruction.
If the IP points to user modifiable data, we can essentially trick the processor to
execute arbitrary code.
Processor architecture has tried to separate code from data through various schemes
of memory management. But there are loopholes in this.
Jens Blanck CSCM28 Pen Testing 4 / 23
Executables
Compiled Code and Data
Compiled code sits in memory as machine code:
Instructions are indistinguishable from data, essentially.
Instruction Pointer register holds address of next instruction.
If the IP points to user modifiable data, we can essentially trick the processor to
execute arbitrary code.
Processor architecture has tried to separate code from data through various schemes
of memory management. But there are loopholes in this.
Jens Blanck CSCM28 Pen Testing 4 / 23
Executables
Assembly
Dump of assembly code for function main
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14 >: mov $0x0 ,% eax
6 0 x0000000000401159 <+19 >: pop %rbp
7 0 x000000000040115a <+20 >: ret
Assembly format
Address: opcode parameters
Jens Blanck CSCM28 Pen Testing 5 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Assembly
1 0 x0000000000401146 <+0>: push %rbp
2 0 x0000000000401147 <+1>: mov %rsp ,% rbp
3 0 x000000000040114a <+4>: mov $0x402004 ,% edi
4 0 x000000000040114f <+9>: call 0 x401030 <puts@plt >
5 0 x0000000000401154 <+14>: mov $0x0 ,% eax
6 0 x0000000000401159 <+19>: pop %rbp
7 0 x000000000040115a <+20>: ret
1 Push value of base pointer onto stack.
2 Move previous stack pointer into base pointer.
3 Move address of string to edi register.
4 Call puts function (Compiler optimisation.)
5 Set accumulator to 0 (ready for return).
6 Pop value off stack to restore base pointer.
7 Return to address at top of stack.
Jens Blanck CSCM28 Pen Testing 6 / 23
Executables
Call Stack
Function calls need to have memory
allocated to store:
its parameters,
the return address, and
local variables.
This is allocated dynamically on a stack in
memory.
Jens Blanck CSCM28 Pen Testing 7 / 23
Executables
CPU Registers (x86)
Some of the CPU registers:
rbp/ebp base pointer (points to start of stack)
rsp/esp stack pointer (points to top of stack)
rip/eip instruction pointer (next instruction)
rax/eax accumulator (used by opcodes to store intermediate results, represents
return values)
rsi/esi source index
rdi/edi destination index
There are many more.
Jens Blanck CSCM28 Pen Testing 8 / 23
gdb – GNU Debugger
gdb – GNU Debugger
GDB, the GNU Project debugger, allows you to see what is going on ‘inside’ another
program while it executes. Features from webpage:
Start your program, specifying anything that might affect its behavior.
Make your program stop on specified conditions.
Examine what has happened, when your program has stopped.
Change things in your program, so you can experiment with correcting the effects of
one bug and go on to learn about another.
Jens Blanck CSCM28 Pen Testing 9 / 23
gdb – GNU Debugger
gdb Cheat Sheet
Command Description
gdb <program> <args> Start gdb on program with args.
run Run the program to be debugged.
step Execute the next instruction.
break <where> Set breakpoint.
clear Delete breakpoints.
Examine memory
x/nfu
n – count, f – format1 , u – unit2 .
disassemble <where> Disassemble.
info locals Information of local variables.
info registers Information on registers.
info args Information on arguments.
set type<address>=value Sets memory to given value (type indicates e.g. char).
1
a: pointer, c: character, x: hex, …
2
b: byte, w: word, g: giant byte, …
Jens Blanck CSCM28 Pen Testing 10 / 23
gdb – GNU Debugger
gdb – Demo
$ gdb hello
GNU gdb (GDB; openSUSE Tumbleweed) 13.2
Copyright (C) 2023 Free Software Foundation , Inc.
[ ... ]
Reading symbols from hello ...
(gdb) run
Starting program: /home/jens/cscm28/slides/hello
Hello World!
[Inferior 1 (process 31388) exited normally]
(gdb) disas main
Dump of assembler code for function main:
0x0000000000401146 <+0>: push %rbp
0x0000000000401147 <+1>: mov %rsp ,%rbp
0x000000000040114a <+4>: mov $0x402004 ,%edi
0x000000000040114f <+9>: call 0x401030 <puts@plt >
0x0000000000401154 <+14>: mov $0x0 ,%eax
0x0000000000401159 <+19>: pop %rbp
0x000000000040115a <+20>: ret
End of assembler dump.
(gdb) x/s 0x402004
0x402004: "Hello World !"
(gdb) q
$
Jens Blanck CSCM28 Pen Testing 11 / 23
Buffer Overflow
Smashing the Stack
Buffer overflow – Basic idea
Put more data into a variable than the
space available. It leaks out, overwriting
other stuff in an interesting manner.
Smashing the Stack
This is a stack-based buffer overflow
vulnerability.
(I recommend you take a look at the Phrack Magazine)
“Smashing The Stack For Fun & Profit” by Aleph One
In Vol 49 of Phrack: Result
[Link] Alteration of data or execution of arbitrary
code!
Jens Blanck CSCM28 Pen Testing 12 / 23
Buffer Overflow
Smashing the Stack
Buffer overflow – Basic idea
Put more data into a variable than the
space available. It leaks out, overwriting
other stuff in an interesting manner.
Smashing the Stack
This is a stack-based buffer overflow
vulnerability.
(I recommend you take a look at the Phrack Magazine)
“Smashing The Stack For Fun & Profit” by Aleph One
In Vol 49 of Phrack: Result
[Link] Alteration of data or execution of arbitrary
code!
Jens Blanck CSCM28 Pen Testing 12 / 23
Buffer Overflow
Smashing the Stack
Buffer overflow – Basic idea
Put more data into a variable than the
space available. It leaks out, overwriting
other stuff in an interesting manner.
Smashing the Stack
This is a stack-based buffer overflow
vulnerability.
(I recommend you take a look at the Phrack Magazine)
“Smashing The Stack For Fun & Profit” by Aleph One
In Vol 49 of Phrack: Result
[Link] Alteration of data or execution of arbitrary
code!
Jens Blanck CSCM28 Pen Testing 12 / 23
Buffer Overflow
Example
# include <s t d i o . h>
# include <s t r i n g . h>
void broken () { Top of stack
bar
char foo [ 2 0 ] ;
char bar [ 1 0 ] ; foo
broken’s frame
p r i n t f ( ”What i s your name? ” ) ; saved frame pointer
g e t s ( foo ) ;
s t r c p y ( bar , foo ) ; return pointer
p r i n t f ( ” Hello , %s \n” , foo ) ;
p r i n t f ( ” Hello , %s \n” , bar ) ;
} main’s frame
i n t main () {
broken ( ) ;
return 0 ;
}
Jens Blanck CSCM28 Pen Testing 13 / 23
Buffer Overflow
Long Strings
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
longer than 9 chars?
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
longer than 9 chars?
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
longer than 9 chars?
Fits inside foo, but… strcpy() overflows
bar, corrupting foo.
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
What happens if a user enters a string longer that 19 chars?
longer than 9 chars?
Fits inside foo, but… strcpy() overflows
bar, corrupting foo.
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
What happens if a user enters a string longer that 19 chars?
longer than 9 chars?
Fits inside foo, but… strcpy() overflows
bar, corrupting foo.
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
$ ./ broken
What is your name? Cohen the Barbarian123456789012345678901234567890
Hello , Barbaria0123456789012345678901234567890
Hello , Cohen the Barbaria0123456789012345678901234567890
Segmentation fault (core dumped)
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
What happens if a user enters a string longer that 19 chars?
longer than 9 chars?
Fills foo and ‘clobbers’ whatever comes
Fits inside foo, but… strcpy() overflows next.
bar, corrupting foo.
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
$ ./ broken
What is your name? Cohen the Barbarian123456789012345678901234567890
Hello , Barbaria0123456789012345678901234567890
Hello , Cohen the Barbaria0123456789012345678901234567890
Segmentation fault (core dumped)
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
Long Strings
What happens if a user enters a string
What happens if a user enters a string longer that 19 chars?
longer than 9 chars?
Fills foo and ‘clobbers’ whatever comes
Fits inside foo, but… strcpy() overflows next.
bar, corrupting foo. Eventually overwrites saved frame pointer,
return pointer, …
$ ./ broken
What is your name? Cohen
Hello , Cohen
Hello , Cohen
$ ./ broken
What is your name? Cohen the Barbarian
Hello , Barbarian
Hello , Cohen the Barbarian
$ ./ broken
What is your name? Cohen the Barbarian123456789012345678901234567890
Hello , Barbaria0123456789012345678901234567890
Hello , Cohen the Barbaria0123456789012345678901234567890
Segmentation fault (core dumped)
Jens Blanck CSCM28 Pen Testing 14 / 23
Buffer Overflow
The Crux
If return pointer is overwritten, program’s control flow is altered.
If this happens accidentally, it probably contains junk. Probably contains address outside
process’ space. Segmentation fault.
What if the overflow is carefully crafted?
1 Overwrite data with malicious code.
2 Overwrite return pointer with address of that code.
3 That code gets executed when function ‘returns’.
Jens Blanck CSCM28 Pen Testing 15 / 23
Buffer Overflow
The Crux
If return pointer is overwritten, program’s control flow is altered.
If this happens accidentally, it probably contains junk. Probably contains address outside
process’ space. Segmentation fault.
What if the overflow is carefully crafted?
1 Overwrite data with malicious code.
2 Overwrite return pointer with address of that code.
3 That code gets executed when function ‘returns’.
Jens Blanck CSCM28 Pen Testing 15 / 23
Buffer Overflow
The Crux
If return pointer is overwritten, program’s control flow is altered.
If this happens accidentally, it probably contains junk. Probably contains address outside
process’ space. Segmentation fault.
What if the overflow is carefully crafted?
1 Overwrite data with malicious code.
2 Overwrite return pointer with address of that code.
3 That code gets executed when function ‘returns’.
Jens Blanck CSCM28 Pen Testing 15 / 23
Buffer Overflow
Smashing the Stack
Top of stack
bar
foo
foo overwritten
with code machine code:
execute a shell
clobbered saved frame pointer
overwritten return pointer
main’s frame
Jens Blanck CSCM28 Pen Testing 16 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defence: Be vigilant!
Don’t allow buffer overflows to happen! Be vigilant!
Every operation must be incapable of overflowing buffer.
Some functions are just dangerous with untrusted data.
A great argument in favour of higher level languages:
isolation from the stack frame,
management of dynamic resources for you,
but not always possible!
Jens Blanck CSCM28 Pen Testing 17 / 23
Buffer Overflow
Defense: Compiler Tricks
Canaries or canary words are known values that are placed between a buffer and control
data on the stack to monitor buffer overflows.
Jens Blanck CSCM28 Pen Testing 18 / 23
Buffer Overflow
Defense: Compiler Tricks
Canaries or canary words are known values that are placed between a buffer and control
data on the stack to monitor buffer overflows.
Jens Blanck CSCM28 Pen Testing 18 / 23
Buffer Overflow
Defense: Compiler Tricks
Canaries or canary words are known values that are placed between a buffer and control
data on the stack to monitor buffer overflows.
When the buffer overflows, the first data to be corrupted will usually be the canary, and a
failed verification of the canary data is therefore an alert of an overflow, which can then be
handled.
Jens Blanck CSCM28 Pen Testing 18 / 23
Buffer Overflow
Defense: DEP
DEP Data execution prevention
Mark the stack as non executable.
Jens Blanck CSCM28 Pen Testing 19 / 23
Buffer Overflow
Defense: DEP
DEP Data execution prevention
Mark the stack as non executable.
Done.
Jens Blanck CSCM28 Pen Testing 19 / 23
Buffer Overflow
Defense: DEP
DEP Data execution prevention
Mark the stack as non executable.
Done.
But wait, perhaps I can get my return pointer to point to an executable area of memory
(much harder, but possible).
Jens Blanck CSCM28 Pen Testing 19 / 23
Buffer Overflow
Defense: DEP
DEP Data execution prevention
Mark the stack as non executable.
Done.
But wait, perhaps I can get my return pointer to point to an executable area of memory
(much harder, but possible).
Also does not stop heap overflows.
Jens Blanck CSCM28 Pen Testing 19 / 23
Buffer Overflow
Defense: ASLR
ASLR Address space layout randomization
Randomize where things are stored in memory including:
stack,
heap,
libraries, etc.
Can still be overcome in some cases, but is really really hard.
Jens Blanck CSCM28 Pen Testing 20 / 23
Buffer Overflow
Defense: ASLR
ASLR Address space layout randomization
Randomize where things are stored in memory including:
stack,
heap,
libraries, etc.
Can still be overcome in some cases, but is really really hard.
Jens Blanck CSCM28 Pen Testing 20 / 23
Buffer Overflow
So why are we learning this?
Jens Blanck CSCM28 Pen Testing 21 / 23
Buffer Overflow
Demo
Jens Blanck CSCM28 Pen Testing 22 / 23
Summary
Summary
We have explored a fairly technical
vulnerability:
C and assembly Lab: Guess… Buffer overflows!
Memory layouts and pointers
Next week: Web application
gdb
vulnerabilities.
Buffer overflows
Techniques to overcome them
Jens Blanck CSCM28 Pen Testing 23 / 23
