The Life of Binaries

Topics include but are not limited to:

• Scanning and tokenizing source code.
• Parsing a grammar.
• Different targets for x86 assembly object files generation. (E.g. relocatable vs. position independent code).
• Linking object files together to create a well-formed binary.
• Detailed descriptions of the high level similarities and low level differences between the Windows PE and Linux ELF binary formats.
• How an OS loads a binary into memory and links it on the fly before executing it.

Along the way we discuss the relevance of security at different stages of a binary’s life, from the tricks that can be played by a malicious compiler, to how viruses really work, to the way which malware “packers” duplicate OS process execution functionality, to the benefit of a security-enhanced OS loader which implements address space layout randomization (ASLR).

Lab work includes:

• Using the new “Binary Scavenger Hunt” tool which creates randomized PE binaries and asks randomized questions about the material you just learned!
• Manipulating compiler options to change the type of assembly which is output.
• Manipulating linker options to change the structure of binary formats.
• Reading and understanding PE files with PEView.
• Reading and understanding ELF files with Readelf.
• Using WinDbg and/or GDB to watch the loader dynamically link an executable.
• Using Thread Local Storage (TLS) to obfuscate control flow and serve as a basic anti-debug mechanism.
• Creating a simple example virus for PE.
• Analyze the changes made to the binary format when a file is packed with UPX.
• Using the rootkit technique of Import Address Table (IAT) hooking to subvert the integrity of a program’s calls to external libraries, allowing files to be hidden.