Topics include but are not limited to:
- Scanning and tokenizing source code.
- Parsing a grammar.
- Different targets for x86 assembly object files generation. (E.g. relocatable vs. position independent code).
- Linking object files together to create a well-formed binary.
- Detailed descriptions of the high level similarities and low level differences between the Windows PE and Linux ELF binary formats.
- How an OS loads a binary into memory and links it on the fly before executing it.
Along the way we discuss the relevance of security at different stages of a binary’s life, from the tricks that can be played by a malicious compiler, to how viruses really work, to the way which malware “packers” duplicate OS process execution functionality, to the benefit of a security-enhanced OS loader which implements address space layout randomization (ASLR).
Lab work includes:
- Using the new “Binary Scavenger Hunt” tool which creates randomized PE binaries and asks randomized questions about the material you just learned!
- Manipulating compiler options to change the type of assembly which is output.
- Manipulating linker options to change the structure of binary formats.
- Reading and understanding PE files with PEView.
- Reading and understanding ELF files with Readelf.
- Using WinDbg and/or GDB to watch the loader dynamically link an executable.
- Using Thread Local Storage (TLS) to obfuscate control flow and serve as a basic anti-debug mechanism.
- Creating a simple example virus for PE.
- Analyze the changes made to the binary format when a file is packed with UPX.
- Using the rootkit technique of Import Address Table (IAT) hooking to subvert the integrity of a program’s calls to external libraries, allowing files to be hidden.
- Do a deep dive into the big picture of how binaries are executed on most OSes.
- Provide detailed information about Windows/Linux binary formats which will be useful to future reverse engineers.
- Show the security-relevance of knowledge about all stages of a binary's life.