Foundations of Cyber

Security II
Debugging

Michael Schwarz | CISPA | June, 2025

 What you (should) know

• x86-64 architecture and memory layout

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout
• C++ vtables

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout
• C++ vtables
• Types of memory safety violations

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout
• C++ vtables
• Types of memory safety violations
• Types of bugs, e.g., buffer overflow, type confusion

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout
• C++ vtables
• Types of memory safety violations
• Types of bugs, e.g., buffer overflow, type confusion
• How they “work”, e.g., writing out of bounds, wrong
free

1 CySec II SS 2025 - Debugging

 What you (should) know

• x86-64 architecture and memory layout

• Memory layout
• Stack/heap layout
• C++ vtables
• Types of memory safety violations
• Types of bugs, e.g., buffer overflow, type confusion
• How they “work”, e.g., writing out of bounds, wrong
free
• What they can do, e.g., overwrite return addresses

1 CySec II SS 2025 - Debugging

 Return-oriented programming Summary

Return-oriented programming (ROP)...

• uses parts of functions to build shellcode

2 CySec II SS 2025 - Debugging

 Return-oriented programming Summary

Return-oriented programming (ROP)...

• uses parts of functions to build shellcode
• is like solving a puzzle - there are tools for finding
gadgets, but constructing the shellcode is still hard
work

2 CySec II SS 2025 - Debugging

 Return-oriented programming Summary

Return-oriented programming (ROP)...

• uses parts of functions to build shellcode
• is like solving a puzzle - there are tools for finding
gadgets, but constructing the shellcode is still hard
work
• does not inject own code, uses existing parts of the
binary

2 CySec II SS 2025 - Debugging

 Return-oriented programming Summary

Return-oriented programming (ROP)...

• uses parts of functions to build shellcode
• is like solving a puzzle - there are tools for finding
gadgets, but constructing the shellcode is still hard
work
• does not inject own code, uses existing parts of the
binary
• works on 32-bit and 64-bit systems

2 CySec II SS 2025 - Debugging

Finding Bugs
 What is a Bug?

• “Bug” used as a term for malfunctions since 1870s

3 CySec II SS 2025 - Debugging

 What is a Bug?

• “Bug” used as a term for malfunctions since 1870s

• Asimov 1944: bugs are issues with robots (I, Robot)

3 CySec II SS 2025 - Debugging

 What is a Bug?

• “Bug” used as a term for malfunctions since 1870s

• Asimov 1944: bugs are issues with robots (I, Robot)
• First computer bug report → 09.09.1947

3 CySec II SS 2025 - Debugging

 What is a Bug?

• “Bug” used as a term for malfunctions since 1870s

• Asimov 1944: bugs are issues with robots (I, Robot)
• First computer bug report → 09.09.1947
→ Grace Hopper’s team

3 CySec II SS 2025 - Debugging

 What is a Bug?

• “Bug” used as a term for malfunctions since 1870s

• Asimov 1944: bugs are issues with robots (I, Robot)
• First computer bug report → 09.09.1947
→ Grace Hopper’s team
• “Debugging” aircraft engines (1945)

3 CySec II SS 2025 - Debugging

 What is a Bug?

• Error causing unintended behavior

4 CySec II SS 2025 - Debugging

 What is a Bug?

• Error causing unintended behavior

• Software vs. hardware bugs

4 CySec II SS 2025 - Debugging

 What is a Bug?

• Error causing unintended behavior

• Software vs. hardware bugs
• Bugs vs. backdoors

4 CySec II SS 2025 - Debugging

 What is a Bug?

• Error causing unintended behavior

• Software vs. hardware bugs
• Bugs vs. backdoors
• Bugs vs. specification flaws

4 CySec II SS 2025 - Debugging

 Finding Bugs

• Human Expert

5 CySec II SS 2025 - Debugging

 Finding Bugs

• Human Expert
• Static Analysis

5 CySec II SS 2025 - Debugging

 Finding Bugs

• Human Expert
• Static Analysis
• Sanitizer

5 CySec II SS 2025 - Debugging

 Finding Bugs

• Human Expert
• Static Analysis
• Sanitizer
• Debugging

5 CySec II SS 2025 - Debugging

 Finding Bugs

• Human Expert
• Static Analysis
• Sanitizer
• Debugging
→ Access to Source/Binary?

5 CySec II SS 2025 - Debugging

Human Expert
 Manual Code Analysis

• Find bugs that occur rarely (not found in automated

testing)

6 CySec II SS 2025 - Debugging

 Manual Code Analysis

• Find bugs that occur rarely (not found in automated

testing)
• Find flaws which might be exploitable with future
code changes

6 CySec II SS 2025 - Debugging

 Manual Code Analysis

• Find bugs that occur rarely (not found in automated

testing)
• Find flaws which might be exploitable with future
code changes
• Develop better practices

6 CySec II SS 2025 - Debugging

 Code Review

• Read code, try to find flaws

7 CySec II SS 2025 - Debugging

 Code Review

• Read code, try to find flaws

• Common: peer-review code when committing it

7 CySec II SS 2025 - Debugging

 Code Review

• Read code, try to find flaws

• Common: peer-review code when committing it
• Most effective way to find and eliminate complex
problems

7 CySec II SS 2025 - Debugging

 Code Review

• Read code, try to find flaws

• Common: peer-review code when committing it
• Most effective way to find and eliminate complex
problems
→ All patches in the Linux kernel are first reviewed

7 CySec II SS 2025 - Debugging

 Limitations of Code Review

• Slow and expensive: potentially as many reviewers as

developers

8 CySec II SS 2025 - Debugging

 Limitations of Code Review

• Slow and expensive: potentially as many reviewers as

developers
• Humans are error-prone and bugs remain

8 CySec II SS 2025 - Debugging

 Limitations of Code Review

• Slow and expensive: potentially as many reviewers as

developers
• Humans are error-prone and bugs remain
• Can waste a lot of time if no tests exist

8 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks
• Identify bugs early

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks
• Identify bugs early
• Three step process

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks
• Identify bugs early
• Three step process
• Decompose

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks
• Identify bugs early
• Three step process
• Decompose
• Determine threats

9 CySec II SS 2025 - Debugging

 Application Threat Modeling

• Complement code review

• Identify, quantify, and address security risks
• Identify bugs early
• Three step process
• Decompose
• Determine threats
• Determine mitigations

9 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 1 – Decompose

• Create use cases

10 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 1 – Decompose

• Create use cases

• Model interaction with an attacker

10 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 1 – Decompose

• Create use cases

• Model interaction with an attacker
• Model assets (target data)

10 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 1 – Decompose

• Create use cases

• Model interaction with an attacker
• Model assets (target data)
• Strictly defined procedure

10 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 1 – Decompose

• Create use cases

• Model interaction with an attacker
• Model assets (target data)
• Strictly defined procedure
→ Avoid incorrect or incomplete deductions

10 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:
• Damage potential

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:
• Damage potential
• Reproducibility

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:
• Damage potential
• Reproducibility
• Exploitability

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:
• Damage potential
• Reproducibility
• Exploitability
• Affected users

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 2 – Determine
Threats

• Use a threat categorization technique (STRIDE/ASF)

• Various models to put a number on a potential attack
• DREAD Score (Microsoft) averages over:
• Damage potential
• Reproducibility
• Exploitability
• Affected users
• Discoverability

11 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 3 – Determine
Mitigations

• For every potential attack there must be a mitigation

12 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 3 – Determine
Mitigations

• For every potential attack there must be a mitigation

• If you figure out that there is no mitigation:

12 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 3 – Determine
Mitigations

• For every potential attack there must be a mitigation

• If you figure out that there is no mitigation:
• that’s a problem

12 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 3 – Determine
Mitigations

• For every potential attack there must be a mitigation

• If you figure out that there is no mitigation:
• that’s a problem
• but it’s good that you found it

12 CySec II SS 2025 - Debugging

 Application Threat Modeling – Step 3 – Determine
Mitigations

• For every potential attack there must be a mitigation

• If you figure out that there is no mitigation:
• that’s a problem
• but it’s good that you found it
• you already know how significant the problem is

12 CySec II SS 2025 - Debugging

Static Analysis
 Abstract Interpretation

• Can we (automatically) find bugs...

13 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Can we (automatically) find bugs...

• ...without even executing the program?

13 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Can we (automatically) find bugs...

• ...without even executing the program?
• Execute an abstracted variant of the program

13 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Can we (automatically) find bugs...

• ...without even executing the program?
• Execute an abstracted variant of the program
• Still maintain semantics

13 CySec II SS 2025 - Debugging

 Abstract Interpretation Example

// randomly select and return one

argument
• argc ∈ [−∞, +∞]
int main ( int argc , char * argv []) {
if ( argc < 0) return -1;
int index = rand () % argc ;
return argv [ index ];
}

14 CySec II SS 2025 - Debugging

 Abstract Interpretation Example

// randomly select and return one

argument
• argc ∈ [−∞, +∞]
int main ( int argc , char * argv []) { • argc ∈ [0, +∞]
if ( argc < 0) return -1;
int index = rand () % argc ;
return argv [ index ];
}

14 CySec II SS 2025 - Debugging

 Abstract Interpretation Example

// randomly select and return one

argument
• argc ∈ [−∞, +∞]
int main ( int argc , char * argv []) { • argc ∈ [0, +∞]
if ( argc < 0) return -1;
int index = rand () % argc ; • rand() % argc → possible
return argv [ index ];
division by zero
}

14 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Formalized to allow sound approximation

15 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Formalized to allow sound approximation

→ Symbolic execution (next lecture)

15 CySec II SS 2025 - Debugging

 Abstract Interpretation

• Formalized to allow sound approximation

→ Symbolic execution (next lecture)
• Basis of Static Analysis

15 CySec II SS 2025 - Debugging

 Static Code Analysis

Works on source code (could also be applied to binary

code)
• Examine every possible code path (→ symbolic
execution)

16 CySec II SS 2025 - Debugging

 Static Code Analysis

Works on source code (could also be applied to binary

code)
• Examine every possible code path (→ symbolic
execution)
• Consider every possible input (→ symbolic execution)

16 CySec II SS 2025 - Debugging

 Static Code Analysis

Works on source code (could also be applied to binary

code)
• Examine every possible code path (→ symbolic
execution)
• Consider every possible input (→ symbolic execution)
→ Good coverage

16 CySec II SS 2025 - Debugging

 Static Code Analysis

Works on source code (could also be applied to binary

code)
• Examine every possible code path (→ symbolic
execution)
• Consider every possible input (→ symbolic execution)
→ Good coverage
• Should be run continuously during development

16 CySec II SS 2025 - Debugging

 Static Code Analysis

Works on source code (could also be applied to binary

code)
• Examine every possible code path (→ symbolic
execution)
• Consider every possible input (→ symbolic execution)
→ Good coverage
• Should be run continuously during development
• Also afterwards to find bugs

16 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

17 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

• Equivalent to solving the halting problem

17 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

• Equivalent to solving the halting problem
→ There are false negatives!

17 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

• Equivalent to solving the halting problem
→ There are false negatives!
• Can we have false positives?

17 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

• Equivalent to solving the halting problem
→ There are false negatives!
• Can we have false positives?
• Yes, if assumptions are too loose

17 CySec II SS 2025 - Debugging

 Static Code Analysis

• Can Static Code Analysis find all problems?

• Equivalent to solving the halting problem
→ There are false negatives!
• Can we have false positives?
• Yes, if assumptions are too loose
• e.g., a check is not evident in the source code

17 CySec II SS 2025 - Debugging

 clang scan-build

• Popular static code analyzer

18 CySec II SS 2025 - Debugging

 clang scan-build

• Popular static code analyzer

• Easy to integrate

18 CySec II SS 2025 - Debugging

 clang scan-build

• Popular static code analyzer

• Easy to integrate
→ scan-build make

18 CySec II SS 2025 - Debugging

 clang scan-build

• Popular static code analyzer

• Easy to integrate
→ scan-build make
• Generates an error report

18 CySec II SS 2025 - Debugging

 clang scan-build (Example 1)

19 CySec II SS 2025 - Debugging

 clang scan-build (Example 1)

19 CySec II SS 2025 - Debugging

 clang scan-build (Example 1)

19 CySec II SS 2025 - Debugging

 clang scan-build (Example 1)

19 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

 clang scan-build (Example 2)

20 CySec II SS 2025 - Debugging

Sanitizers
 Address Sanitizer

Address Sanitizer (ASan)...

• is a compiler and runtime extension in gcc and clang

21 CySec II SS 2025 - Debugging

 Address Sanitizer

Address Sanitizer (ASan)...

• is a compiler and runtime extension in gcc and clang
• detects many memory errors at runtime

21 CySec II SS 2025 - Debugging

 Address Sanitizer

Address Sanitizer (ASan)...

• is a compiler and runtime extension in gcc and clang
• detects many memory errors at runtime
• should only be used as debugging tool

21 CySec II SS 2025 - Debugging

 Address Sanitizer

Address Sanitizer (ASan)...

• is a compiler and runtime extension in gcc and clang
• detects many memory errors at runtime
• should only be used as debugging tool
• is activated with compiler flag -fsanitize=address

21 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals

22 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals
• Use-after-free

22 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals
• Use-after-free
• Use-after-return

22 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals
• Use-after-free
• Use-after-return
• Use-after-scope

22 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals
• Use-after-free
• Use-after-return
• Use-after-scope
• Double-free, invalid free

22 CySec II SS 2025 - Debugging

 Address Sanitizer

ASan detects
• Out-of-bounds accesses to heap, stack and globals
• Use-after-free
• Use-after-return
• Use-after-scope
• Double-free, invalid free
• Memory leaks (experimental)

22 CySec II SS 2025 - Debugging

 Address Sanitizer
typedef struct {
void (* print ) ( char *) ;
} operation ;

int main ( int argc , char * argv []) {

operation * io = ( operation *) malloc ( sizeof ( operation ) ) ;
io - > print = puts ;
io - > print ( " Hallo " ) ;
free ( io ) ;

if ( argc > 1) {
char * buffer = ( char *) malloc (8) ;
strncpy ( buffer , argv [1] , 7) ;
io - > print ( buffer ) ;
free ( buffer ) ;
}
return 0;
23 CySec II SS }2025 - Debugging
 Address Sanitizer

• malloc and free are replaced

24 CySec II SS 2025 - Debugging

 Address Sanitizer

• malloc and free are replaced

• Memory around malloc segments is poisoned

24 CySec II SS 2025 - Debugging

 Address Sanitizer

• malloc and free are replaced

• Memory around malloc segments is poisoned
• Free’d memory is poisoned and moved to quarantine

24 CySec II SS 2025 - Debugging

 Address Sanitizer

• malloc and free are replaced

• Memory around malloc segments is poisoned
• Free’d memory is poisoned and moved to quarantine
• Every memory access is first checked if poisoned

24 CySec II SS 2025 - Debugging

 Address Sanitizer

• malloc and free are replaced

• Memory around malloc segments is poisoned
• Free’d memory is poisoned and moved to quarantine
• Every memory access is first checked if poisoned

* address = ...;

24 CySec II SS 2025 - Debugging

 Address Sanitizer

• malloc and free are replaced

• Memory around malloc segments is poisoned
• Free’d memory is poisoned and moved to quarantine
• Every memory access is first checked if poisoned

* address = ...;

if ( IsPoisoned ( address ) ) {
ReportError ( address , kAccessSize , kIsWrite ) ;
}
* address = ...;

24 CySec II SS 2025 - Debugging

 Address Sanitizer

Limitations
• Slowdown (factor 2)

25 CySec II SS 2025 - Debugging

 Address Sanitizer

Limitations
• Slowdown (factor 2)
• Increased memory usage (factor 2-5)

25 CySec II SS 2025 - Debugging

 Address Sanitizer

Limitations
• Slowdown (factor 2)
• Increased memory usage (factor 2-5)
• Cannot prevent all arbitrary memory corruptions

25 CySec II SS 2025 - Debugging

 Address Sanitizer

Limitations
• Slowdown (factor 2)
• Increased memory usage (factor 2-5)
• Cannot prevent all arbitrary memory corruptions
• Does not protect against integer overflows

25 CySec II SS 2025 - Debugging

 Address Sanitizer

Limitations
• Slowdown (factor 2)
• Increased memory usage (factor 2-5)
• Cannot prevent all arbitrary memory corruptions
• Does not protect against integer overflows
• Adjacent buffers in structs/classes are not protected

25 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang
• should only be used as debugging tool

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang
• should only be used as debugging tool
• detects undefined behavior

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang
• should only be used as debugging tool
• detects undefined behavior
• activated with compiler flag -fsanitize=undefined

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang
• should only be used as debugging tool
• detects undefined behavior
• activated with compiler flag -fsanitize=undefined
• detects suspicious/unwanted behavior

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

Undefined-Behavior Sanitizer ...

• implemented in gcc and clang
• should only be used as debugging tool
• detects undefined behavior
• activated with compiler flag -fsanitize=undefined
• detects suspicious/unwanted behavior
• activated with compiler flag
-fsanitize=undefined,integer

26 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

# include < stdio .h >

int main ( int argc , char * argv []) {

char * val [] = { " Hello " , " World " };
char * secret = " secret " ;
char s = atoi ( argv [1]) ;
if ( atoi ( argv [1]) >= 0 && s < 2)
printf ( " % s \ n " , val [ s ]) ;
else
printf ( " Invalid ID \ n " ) ;
return 0;
}

27 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

clang - fsanitize = undefined , integer test . c -o test

28 CySec II SS 2025 - Debugging

 Undefined-Behavior Sanitizer

clang - fsanitize = undefined , integer test . c -o test

./ test 255
test . c :6:14: runtime error : implicit conversion from type ’int’ of value 255 (32-bit,
signed) to type ’char’ changed the value to -1 (8-bit, signed)
SUMMARY : U n d e f i n e d B e h a v i o r S a n i t i z e r : undefined - behavior test . c :6:14 in
test . c :8:24: runtime error : index -1 out of bounds for type ’char *[2]’
SUMMARY : U n d e f i n e d B e h a v i o r S a n i t i z e r : undefined - behavior test . c :8:24 in
secret

28 CySec II SS 2025 - Debugging

 Data Races

• If one resource is used in parallel (→ multithreading)...

29 CySec II SS 2025 - Debugging

 Data Races

• If one resource is used in parallel (→ multithreading)...

• ...and not all accesses are read-only

29 CySec II SS 2025 - Debugging

 Data Races

• If one resource is used in parallel (→ multithreading)...

• ...and not all accesses are read-only
→ Data race

29 CySec II SS 2025 - Debugging

 Data Races

• If one resource is used in parallel (→ multithreading)...

• ...and not all accesses are read-only
→ Data race
→ Unclear what the result is (reading old vs new value)

29 CySec II SS 2025 - Debugging

 Data Race Example

balance: 1000

A> int x = balance; A: x = 1000 (amount = 200)

x -= amount;
balance = x; B: x = (amount = 100)

30 CySec II SS 2025 - Debugging

 Data Race Example

balance: 1000

int x = balance; A: x = 800 (amount = 200)

A> x -= amount;
balance = x; B: x = (amount = 100)

30 CySec II SS 2025 - Debugging

 Data Race Example

balance: 1000

B> int x = balance; A: x = 800 (amount = 200)

A> x -= amount;
balance = x; B: x = 1000 (amount = 100)

30 CySec II SS 2025 - Debugging

 Data Race Example

balance: 1000

int x = balance; A: x = 800 (amount = 200)

B> A> x -= amount;
balance = x; B: x = 900 (amount = 100)

30 CySec II SS 2025 - Debugging

 Data Race Example

balance: 900

int x = balance; A: x = 800 (amount = 200)

A> x -= amount;
B> balance = x; B: x = 900 (amount = 100)

30 CySec II SS 2025 - Debugging

 Data Race Example

balance: 800

int x = balance; A: x = 800 (amount = 200)

x -= amount;
A> balance = x; B: x = 900 (amount = 100)

30 CySec II SS 2025 - Debugging

 Thread Sanitizer

Thread Sanitizer ...

• implemented in gcc and clang

31 CySec II SS 2025 - Debugging

 Thread Sanitizer

Thread Sanitizer ...

• implemented in gcc and clang
• detects data races

31 CySec II SS 2025 - Debugging

 Thread Sanitizer

Thread Sanitizer ...

• implemented in gcc and clang
• detects data races
• “Fast happens-before” alogrithm (cf. FastTrack
algorithm)

31 CySec II SS 2025 - Debugging

 Thread Sanitizer

Thread Sanitizer ...

• implemented in gcc and clang
• detects data races
• “Fast happens-before” alogrithm (cf. FastTrack
algorithm)
• is activated with compiler flag -fsanitize=thread

31 CySec II SS 2025 - Debugging

 How does Thread Sanitizer work?

• Rewrite code, add calls to read/write access check

function

32 CySec II SS 2025 - Debugging

 How does Thread Sanitizer work?

• Rewrite code, add calls to read/write access check

function
• Performance overhead: 5 − 15×

32 CySec II SS 2025 - Debugging

 How does Thread Sanitizer work?

• Rewrite code, add calls to read/write access check

function
• Performance overhead: 5 − 15×
• Memory impact: 5 − 10×

32 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit)

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit)
• Timestamp (42 bit)

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit)
• Timestamp (42 bit)
• Position/Offset in 8-byte
word (5 bit)

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit)
• Timestamp (42 bit)
• Position/Offset in 8-byte
word (5 bit)
• Read/Write (1 bit)

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit)
• Timestamp (42 bit)
• Position/Offset in 8-byte
word (5 bit)
• Read/Write (1 bit)
→ 8 bytes

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit) Buffer the last N ∈ [4, 9]
• Timestamp (42 bit) memory acceses
• Position/Offset in 8-byte • per 8-byte region
word (5 bit)
• Read/Write (1 bit)
→ 8 bytes

33 CySec II SS 2025 - Debugging

 Thread Sanitizer Implementation

Store in shadow memory for

each 8-byte region:
• TID (16 bit) Buffer the last N ∈ [4, 9]
• Timestamp (42 bit) memory acceses
• Position/Offset in 8-byte • per 8-byte region
word (5 bit) → Memory impact: 5 − 10×
• Read/Write (1 bit)
→ 8 bytes

33 CySec II SS 2025 - Debugging

 Thread Sanitizer - Example 1 - Can you see the
problem?
# include < pthread .h >
int value ;

void * thread ( void * x ) {

value = 42;
return x ;
}

int main () {
pthread_t t ;
pthr ead_cr eate (& t , NULL , thread , NULL ) ;
value = 43;
pthread_join (t , NULL ) ;
return value ;
}

34 CySec II SS 2025 - Debugging

 Thread Sanitizer - Example 1

35 CySec II SS 2025 - Debugging

 Memory Sanitizer

Memory Sanitizer
• detects use of uninitialized values

36 CySec II SS 2025 - Debugging

 Memory Sanitizer

Memory Sanitizer
• detects use of uninitialized values
• implemented in clang

36 CySec II SS 2025 - Debugging

 Memory Sanitizer

Memory Sanitizer
• detects use of uninitialized values
• implemented in clang
• is activated with compiler flag -fsanitize=memory

36 CySec II SS 2025 - Debugging

 Memory Sanitizer Details

• Typical slowdown: 3×

37 CySec II SS 2025 - Debugging

 Memory Sanitizer Details

• Typical slowdown: 3×
• Memory impact: 2 − 3×

37 CySec II SS 2025 - Debugging

 Memory Sanitizer Details

• Typical slowdown: 3×
• Memory impact: 2 − 3×
• reserves a 64 TB region virtual address space

37 CySec II SS 2025 - Debugging

 Memory Sanitizer - Example

int main ( int argc , char ** argv ) {

int * a = malloc (10 * sizeof ( int ) ) ;
a [5] = 0;
volatile int b = a [ argc ];
if ( b )
printf ( " not zero \ n " ) ;
return 0;
}

38 CySec II SS 2025 - Debugging

 Memory Sanitizer

39 CySec II SS 2025 - Debugging

Debugging
 Debugger

• Debugger allows stepping through program

40 CySec II SS 2025 - Debugging

 Debugger

• Debugger allows stepping through program

→ Execute instruction-by-instruction

40 CySec II SS 2025 - Debugging

 Debugger

• Debugger allows stepping through program

→ Execute instruction-by-instruction
• Inspect register and memory

40 CySec II SS 2025 - Debugging

 Debugger

• Debugger allows stepping through program

→ Execute instruction-by-instruction
• Inspect register and memory
• Modify internal program state

40 CySec II SS 2025 - Debugging

 GDB

• GDB: default debugger on Linux (terminal-based)

41 CySec II SS 2025 - Debugging

 GDB

• GDB: default debugger on Linux (terminal-based)

• Can be configured using ~/.gdbinit

41 CySec II SS 2025 - Debugging

 GDB

• GDB: default debugger on Linux (terminal-based)

• Can be configured using ~/.gdbinit
→ See CMS for a sample configuration

41 CySec II SS 2025 - Debugging

 GDB - Basic Commands

Command Description

run Start program execution

b main Set breakpoint (here: at main)
si Single step (follow call target)
ni Single step (continue after call)
continue Run until next breakpoint

x/x 0x8048000 Print value at 0x8048000 as hex

x/16x $rsp Print 20 hex values starting at stack pointer
x/s 0x8048000 Print string at 0x8048000

42 CySec II SS 2025 - Debugging

 GDB - Basic Commands II

Command Description

bt Backtrace (call chain until here)

info frame Info about current stack frame

call <function> Call a function

jump *<address> Jump to the address
set $rax = 4 Set register (RAX) to value
set int0x8048000 = 0x90 Set memory location to value

43 CySec II SS 2025 - Debugging

 GDBGUI

• GDB is extremely powerful

44 CySec II SS 2025 - Debugging

 GDBGUI

• GDB is extremely powerful

• Many commands → hard to remember

44 CySec II SS 2025 - Debugging

 GDBGUI

• GDB is extremely powerful

• Many commands → hard to remember
• Graphical applications often easier

44 CySec II SS 2025 - Debugging

 GDBGUI

• GDB is extremely powerful

• Many commands → hard to remember
• Graphical applications often easier
→ GDBGUI is a graphical interface

44 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB
→ GEF is helpful for exploit development

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB
→ GEF is helpful for exploit development
• Analyze heap

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB
→ GEF is helpful for exploit development
• Analyze heap
• Modify code

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB
→ GEF is helpful for exploit development
• Analyze heap
• Modify code
• Find ROP gadgets

45 CySec II SS 2025 - Debugging

 GEF - GDB Enhanced Features

• Advanced users often prefer terminal

• Multiple scripts to improve GDB
→ GEF is helpful for exploit development
• Analyze heap
• Modify code
• Find ROP gadgets
• ...

45 CySec II SS 2025 - Debugging

 Source Debugging

• How does a debugger work?

46 CySec II SS 2025 - Debugging

 Source Debugging

• How does a debugger work?

• Let’s build a debugger!

46 CySec II SS 2025 - Debugging

 Source Debugging

• How does a debugger work?

• Let’s build a debugger!
• What do we need?

46 CySec II SS 2025 - Debugging

 Building a Debugger

• Read and write register and memory of another

process

47 CySec II SS 2025 - Debugging

 Building a Debugger

• Read and write register and memory of another

process
→ ptrace()

47 CySec II SS 2025 - Debugging

 Building a Debugger

• Read and write register and memory of another

process
→ ptrace()
• Interrupt a process at an arbitrary instruction

47 CySec II SS 2025 - Debugging

 Building a Debugger

• Read and write register and memory of another

process
→ ptrace()
• Interrupt a process at an arbitrary instruction
→ Interrupts interrupt processes

47 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

48 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

• Permission system:

48 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

• Permission system:
• Trace other threads of your own process

48 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

• Permission system:
• Trace other threads of your own process
• Trace child processes

48 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

• Permission system:
• Trace other threads of your own process
• Trace child processes
• Trace any process if you’re root (CAP_SYS_PTRACE)

48 CySec II SS 2025 - Debugging

 ptrace()
# include < sys / ptrace .h >
long ptrace ( enum _ _ p t r a c e _ r e q u e s t request , pid_t pid , void * addr , void * data ) ;

ptrace(): control execution of process, examine/change memory and

registers

• Used for breakpoint debugging and system call tracing

• Permission system:
• Trace other threads of your own process
• Trace child processes
• Trace any process if you’re root (CAP_SYS_PTRACE)
→ Check ptrace() man page!

48 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers
→ PTRACE_GETREGSET, PTRACE_SETREGS

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers
→ PTRACE_GETREGSET, PTRACE_SETREGS
• Monitor syscalls

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers
→ PTRACE_GETREGSET, PTRACE_SETREGS
• Monitor syscalls
→ PTRACE_SYSCALL, ...

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers
→ PTRACE_GETREGSET, PTRACE_SETREGS
• Monitor syscalls
→ PTRACE_SYSCALL, ...
• Receive a SIGTRAP upon specific or all system calls

49 CySec II SS 2025 - Debugging

 ptrace() Capabilities

• Read and write memory

→ PTRACE_PEEKTEXT, PTRACE_PEEKUSER, ...
• Read and write registers
→ PTRACE_GETREGSET, PTRACE_SETREGS
• Monitor syscalls
→ PTRACE_SYSCALL, ...
• Receive a SIGTRAP upon specific or all system calls

• Several others

49 CySec II SS 2025 - Debugging

 Setting up ptrace()

• Fork

50 CySec II SS 2025 - Debugging

 Setting up ptrace()

• Fork
• Child calls ptrace(PTRACE_TRACEME, 0, 0, 0);

50 CySec II SS 2025 - Debugging

 Setting up ptrace()

• Fork
• Child calls ptrace(PTRACE_TRACEME, 0, 0, 0);
• Parent calls ptrace(PTRACE_ATTACH, child_pid, 0, 0);

50 CySec II SS 2025 - Debugging

 Setting up ptrace()

• Fork
• Child calls ptrace(PTRACE_TRACEME, 0, 0, 0);
• Parent calls ptrace(PTRACE_ATTACH, child_pid, 0, 0);
→ Parent now controls the child

50 CySec II SS 2025 - Debugging

 Interrupts

• OS provides special interrupt handling for debuggers

51 CySec II SS 2025 - Debugging

 Interrupts

• OS provides special interrupt handling for debuggers

• Software interrupt 3

51 CySec II SS 2025 - Debugging

 Interrupts

• OS provides special interrupt handling for debuggers

• Software interrupt 3
• Single-byte instruction, machine code 0xCC

51 CySec II SS 2025 - Debugging

 Interrupts

• OS provides special interrupt handling for debuggers

• Software interrupt 3
• Single-byte instruction, machine code 0xCC
• How can we use that?

51 CySec II SS 2025 - Debugging

 Looking at an Example Program

# include < stdio .h >

signed int s1 = -4;

unsigned int u1 = 2;

int main ()
{
if ( s1 < u1 ) {
printf ( " In math we trust .\ n " ) ;
} else {
printf ( " Some men just want to watch the world burn .\ n " ) ;
}
}

52 CySec II SS 2025 - Debugging

 Looking at an Example Program

Assembly of main

1151: 8b 05 b9 2 e 00 00 mov 0 x2eb9 (% rip ) ,% edx # 4010 <s1 >

1159: 8b 05 b5 2 e 00 00 mov 0 x2eb5 (% rip ) ,% eax # 4014 <u1 >
115 f : 39 c2 cmp % eax ,% edx
1161: 73 0e jae 1171 < main +0 x28 >
1163: 48 8d 3 d 9 e 0 e 00 00 lea 0 xe9e (% rip ) ,% rdi
116 a : e8 e1 fe ff ff callq 1050 < puts@plt >
116 f : eb 0c jmp 117 d < main +0 x34 >
1171: 48 8d 3 d a8 0 e 00 00 lea 0 xea8 (% rip ) ,% rdi
1178: e8 d3 fe ff ff callq 1050 < puts@plt >
117 d : b8 00 00 00 00 mov $0x0 ,% eax
1183: c3 retq

How can we interrupt a program here?

53 CySec II SS 2025 - Debugging

 Looking at an Example Program

Idea: Replace the instruction at that position with a software interrupt...

1151: 8b 05 b9 2 e 00 00 mov 0 x2eb9 (% rip ) ,% edx # 4010 <s1 >

1159: 8b 05 b5 2 e 00 00 mov 0 x2eb5 (% rip ) ,% eax # 4014 <u1 >
115 f : 39 c2 cmp % eax ,% edx
1161: cc int3
1162: 0e ( bad )
1163: 48 8d 3 d 9 e 0 e 00 00 lea 0 xe9e (% rip ) ,% rdi
116 a : e8 e1 fe ff ff callq 1050 < puts@plt >
116 f : eb 0c jmp 117 d < main +0 x34 >
1171: 48 8d 3 d a8 0 e 00 00 lea 0 xea8 (% rip ) ,% rdi
1178: e8 d3 fe ff ff callq 1050 < puts@plt >
117 d : b8 00 00 00 00 mov $0x0 ,% eax
1183: c3 retq

How do we get the processor to execute our original instruction again?

54 CySec II SS 2025 - Debugging
 Building Breakpoints

Idea: The OS interrupt handler restores the original byte

1151: 8b 05 b9 2 e 00 00 mov 0 x2eb9 (% rip ) ,% edx # 4010 <s1 >

1159: 8b 05 b5 2 e 00 00 mov 0 x2eb5 (% rip ) ,% eax # 4014 <u1 >
115 f : 39 c2 cmp % eax ,% edx
1161: 73 ???
1162: 0e ( bad )
1163: 48 8d 3 d 9 e 0 e 00 00 lea 0 xe9e (% rip ) ,% rdi
116 a : e8 e1 fe ff ff callq 1050 < puts@plt >
116 f : eb 0c jmp 117 d < main +0 x34 >
1171: 48 8d 3 d a8 0 e 00 00 lea 0 xea8 (% rip ) ,% rdi
1178: e8 d3 fe ff ff callq 1050 < puts@plt >
117 d : b8 00 00 00 00 mov $0x0 ,% eax
1183: c3 retq

But we are at the wrong location... What now?

55 CySec II SS 2025 - Debugging
 Building Breakpoints

Idea: The OS can manipulate %eip/%rip

1151: 8b 05 b9 2 e 00 00 mov 0 x2eb9 (% rip ) ,% edx # 4010 <s1 >

1159: 8b 05 b5 2 e 00 00 mov 0 x2eb5 (% rip ) ,% eax # 4014 <u1 >
115 f : 39 c2 cmp % eax ,% edx
1161: 73 0e jae 1171 < main +0 x28 >
1163: 48 8d 3 d 9 e 0 e 00 00 lea 0 xe9e (% rip ) ,% rdi
116 a : e8 e1 fe ff ff callq 1050 < puts@plt >
116 f : eb 0c jmp 117 d < main +0 x34 >
1171: 48 8d 3 d a8 0 e 00 00 lea 0 xea8 (% rip ) ,% rdi
1178: e8 d3 fe ff ff callq 1050 < puts@plt >
117 d : b8 00 00 00 00 mov $0x0 ,% eax
1183: c3 retq

Program can continue

56 CySec II SS 2025 - Debugging

 Let’s see what happens when ...

# include < stdio .h >

signed int s1 = -4;

unsigned int u1 = 2;

int main ()
{
asm ( " int $3 " ) ;
if ( s1 < u1 ) {
printf ( " In math we trust .\ n " ) ;
} else {
printf ( " Some men just want to watch the world burn .\ n " ) ;
}
}

57 CySec II SS 2025 - Debugging

 So This is How Breakpoints Work!

58 CySec II SS 2025 - Debugging

 “Wolf fence” algorithm (git bisect)

• Powerful application of version control

59 CySec II SS 2025 - Debugging

 “Wolf fence” algorithm (git bisect)

• Powerful application of version control

• Bisection search (binary search) over commits

59 CySec II SS 2025 - Debugging

 “Wolf fence” algorithm (git bisect)

• Powerful application of version control

• Bisection search (binary search) over commits
→ Find a specific change (such as a bug)

59 CySec II SS 2025 - Debugging

 “Wolf fence” algorithm (git bisect)

• Powerful application of version control

• Bisection search (binary search) over commits
→ Find a specific change (such as a bug)
• Efficient: O(log(n))

59 CySec II SS 2025 - Debugging

 git bisect example

• Open-source software, ≈ 2000 commits

60 CySec II SS 2025 - Debugging

 git bisect example

• Open-source software, ≈ 2000 commits

• A reproducible bug in the current version

60 CySec II SS 2025 - Debugging

 git bisect example

• Open-source software, ≈ 2000 commits

• A reproducible bug in the current version
• Unclear what the source of the bug is

60 CySec II SS 2025 - Debugging

 git bisect example

• Open-source software, ≈ 2000 commits

• A reproducible bug in the current version
• Unclear what the source of the bug is

> git bisect start

> git bisect bad
> git bisect good < commit without bug >
Bisecting : 1098 revisions left to test after this ( roughly
10 steps )

60 CySec II SS 2025 - Debugging

 git bisect example

• ≈ 2000 commits → only ≈ 10 revisions to test

61 CySec II SS 2025 - Debugging

 git bisect example

• ≈ 2000 commits → only ≈ 10 revisions to test

→ Can also be automated: git bisect run

61 CySec II SS 2025 - Debugging

 git bisect example

• ≈ 2000 commits → only ≈ 10 revisions to test

→ Can also be automated: git bisect run
• Only need a script that tests a commit:

61 CySec II SS 2025 - Debugging

 git bisect example

• ≈ 2000 commits → only ≈ 10 revisions to test

→ Can also be automated: git bisect run
• Only need a script that tests a commit:

> git bisect start

> git bisect bad
> git bisect good < commit >
> git bisect run < script >

61 CySec II SS 2025 - Debugging

 Lecture Challenges

• Find a bug in a huge git repository

62 CySec II SS 2025 - Debugging

 Lecture Challenges

• Find a bug in a huge git repository

• Experiment with conditional breakpoints

62 CySec II SS 2025 - Debugging

 Lecture Challenges

• Find a bug in a huge git repository

• Experiment with conditional breakpoints
• Automate your debugging session

62 CySec II SS 2025 - Debugging

 Lecture Challenges

• Find a bug in a huge git repository

• Experiment with conditional breakpoints
• Automate your debugging session
• Debug a Linux kernel

62 CySec II SS 2025 - Debugging

 printf Debugging

Proper debugging may be impossible for various reasons

• System without available debugger

63 CySec II SS 2025 - Debugging

 printf Debugging

Proper debugging may be impossible for various reasons

• System without available debugger
• Booting a kernel

63 CySec II SS 2025 - Debugging

 printf Debugging

Proper debugging may be impossible for various reasons

• System without available debugger
• Booting a kernel
• Race conditions disappear if timing changes

63 CySec II SS 2025 - Debugging

 printf Debugging

Proper debugging may be impossible for various reasons

• System without available debugger
• Booting a kernel
• Race conditions disappear if timing changes
• Debugger crashes

63 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

64 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

→ See how far it gets

64 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

→ See how far it gets
• More extreme form: print register debugging

64 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

→ See how far it gets
• More extreme form: print register debugging
→ If you can only get register values upon crash and
nothing else

64 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

→ See how far it gets
• More extreme form: print register debugging
→ If you can only get register values upon crash and
nothing else
• Most extreme form: LED debugging

64 CySec II SS 2025 - Debugging

 printf Debugging

• printf in interesting lines

→ See how far it gets
• More extreme form: print register debugging
→ If you can only get register values upon crash and
nothing else
• Most extreme form: LED debugging
→ If you can only get LED on or off as feedback

64 CySec II SS 2025 - Debugging

 Rubberduck Debugging

• Explaining code helps

65 CySec II SS 2025 - Debugging

 Rubberduck Debugging

• Explaining code helps

→ Discrepancy of what the code does and what it should
do

65 CySec II SS 2025 - Debugging

 Rubberduck Debugging

• Explaining code helps

→ Discrepancy of what the code does and what it should
do
• Save your colleague some time

65 CySec II SS 2025 - Debugging

 Rubberduck Debugging

• Explaining code helps

→ Discrepancy of what the code does and what it should
do
• Save your colleague some time
• Explain it to the duck as if it was a colleague

65 CySec II SS 2025 - Debugging

 Rubberduck Debugging

• Explaining code helps

→ Discrepancy of what the code does and what it should
do
• Save your colleague some time
• Explain it to the duck as if it was a colleague
• Speaking aloud helps

65 CySec II SS 2025 - Debugging

 Finding Bugs Conclusion

• Debugging needs experts

66 CySec II SS 2025 - Debugging

 Finding Bugs Conclusion

• Debugging needs experts

• Experts are expensive

66 CySec II SS 2025 - Debugging

 Finding Bugs Conclusion

• Debugging needs experts

• Experts are expensive
• Debugging is expensive

66 CySec II SS 2025 - Debugging

 Next on CySec II

• More ways of finding bugs

67 CySec II SS 2025 - Debugging

 Next on CySec II

• More ways of finding bugs

• Dealing with applications without source

67 CySec II SS 2025 - Debugging

 Next on CySec II

• More ways of finding bugs

• Dealing with applications without source
• Automating everything

67 CySec II SS 2025 - Debugging

Foundations of Cyber
Security II
Debugging

Michael Schwarz | CISPA | June, 2025

 Further Reading i

References

[Goo20] Google. AddressSanitizer, ThreadSanitizer, MemorySanitizer.

2020. url: [Link]
[Lat11] C. Lattner. What Every C Programmer Should Know About
Undefined Behavior. 2011. url: [Link]
[Link].

CySec II SS 2025 - Debugging

 Further Reading ii

[Moz09] Mozilla. Abstract Interpretation. 2009. url:

[Link]

[MS08] N. S. Matloff and P. J. Salzman. The art of debugging with

GDB, DDD, and Eclipse. No Starch Press, 2008.
[Sho14] A. Shostack. Threat modeling: Designing for security. John
Wiley & Sons, 2014.

CySec II SS 2025 - Debugging