Data Encoding and Malware
Countermeasures
Aaron Sedlacek
Data Encoding and Malware
10/06/2015 1
Countermeasures
Agenda
Data Encoding Techniques
Reversing Encoding Techniques
Malware Countermeasures
Data Encoding and Malware
10/06/2015 2
Countermeasures
Data Encoding - Why?
•Encryption of network-based
communication
•Disguise internal workings
○ Obfuscate strings, decoding them only when
needed
○ Hide config. information
Data Encoding and Malware
10/06/2015 3
Countermeasures
Data Encoding - How?
•Simple ciphers
○ XOR, Base64, ROT, etc.
•Standard crypto algorithms
○ RSA, OpenSSL, etc.
•Custom encoding schemes
Data Encoding and Malware
10/06/2015 4
Countermeasures
Simple Ciphers - XOR
•xor
○ Tends to be pretty obvious
○ NULL-preserving single-byte xor
■ skips bytes that are:
● NULL (From the PMA Book)
● The key itself
Less obvious, isn’t it? →
(From the PMA Book)
Data Encoding and Malware
10/06/2015 5
Countermeasures
Simple Ciphers
•There are plenty of other simple encoding
schemes
○ add and sub
○ rol and ror
○ rot
○ Multibyte xor
Data Encoding and Malware
10/06/2015 6
Countermeasures
Simple Ciphers - Base64
1.‘=’ character is
used
as padding
2. Indexing Strings
→
Data Encoding and Malware
10/06/2015 7
Countermeasures
Common Crypto Algorithms
•Easiest ways to discover the algorithm:
○ Look through strings and imports
■ ‘OpenSSL’
■ ‘rijndael’
○ Cryptographic Constants
■ Fixed magic constants
○ Entropy
Data Encoding and Malware
10/06/2015 8
Countermeasures
Cryptographic Constants and Entropy
•Some very useful IDA Pro Plugins
○ FindCrypt2 and KANAL (Krypto ANALyzer)
■ looks for common cryptographic constants
during initial analysis
IDA Entropy Plugin
Data Encoding and Malware
10/06/2015 9
Countermeasures
Custom Encoding
•Homegrown encoding schemes
○ Layer multiple simple encoding methods
○ Entirely custom algorithm(s)
•The plugins we mentioned earlier will
probably be useless, lol
Data Encoding and Malware
10/06/2015 10
Countermeasures
Agenda
Data Encoding Techniques
Reversing Encoding Techniques
Malware Countermeasures
Data Encoding and Malware
10/06/2015 11
Countermeasures
Reversing Encoding
[Link] program execution, looking for
encoding or decoding functions
[Link] out when these functions are used
[Link] the malware against itself!
[Link] the functions
[Link] the functions as they exist in the malware
c.???
[Link]!
Data Encoding and Malware
10/06/2015 12
Countermeasures
Using the Malware Against Itself
•Run the malware in a debugger and set
breakpoints before/after encoding/
decoding
○ The malware may not decrypt the info you are
interested in
○ You can’t figure out how to get the malware to run
the decrypt function
•Use existing implementations from code
libraries (Like PyCrypto)
Data Encoding and Malware
10/06/2015 13
Countermeasures
Other Clever Ideas
•Patch the malware to make it do what you
want!
•Set a breakpoint before decoding, and
change the memory referenced
○ Either the pointer to the memory, or the content
itself!
•Other Hacky Things! Patching is fun!
Data Encoding and Malware
10/06/2015 14
Countermeasures
Demo
Data Encoding and Malware
10/06/2015 15
Countermeasures
Agenda
Data Encoding Techniques
Reversing Encoding Techniques
Malware Countermeasures
Data Encoding and Malware
10/06/2015 16
Countermeasures
Malware Indicators
•Physical Indicators
○ Hashes of the file
○ Strings in the file
○ Known behavior
•Network Indicators
○ Specific Domains
○ IP Addresses
○ HTTP Request content
Data Encoding and Malware
10/06/2015 17
Countermeasures
Terms Commonly Used In Industry
•Sinkhole
○ An host on the internal network that receives
redirected traffic from known malicious domains
•Intrusion Detection System
•Intrusion Prevention System
•Operations Security (OPSEC)
○ The process of preventing adversaries from
obtaining sensitive information
Data Encoding and Malware
10/06/2015 18
Countermeasures
Operations Security
• Extra care needs to be taken in order to ensure that
the malware author is not aware of you.
○ Spear-phishing emails with unique links.
○ Embed an unused domain in malware, and watch
for attempts to resolve the domain.
Data Encoding and Malware
10/06/2015 19
Countermeasures
Snort
•One of the most popular IDSs
•Used to create a signature or rule that links
together a series of elements
•Lots more on this in the textbook
Data Encoding and Malware
10/06/2015 20
Countermeasures
Questions?
Data Encoding and Malware
10/06/2015 21
Countermeasures
References
1. Sikorski, Michael, and Andrew Honig. Practical Malware Analysis the
Hands-on Guide to Dissecting Malicious Software. San Francisco: No
Starch, 2012. Print.
Malware - 09/08/2015 Advanced Static Analysis 22
