• Online, Self-Paced
Course Description

Topics include but are not limited to:

  • Scanning and tokenizing source code.
  • Parsing a grammar.
  • Different targets for x86 assembly object files generation. (E.g. relocatable vs. position independent code).
  • Linking object files together to create a well-formed binary.
  • Detailed descriptions of the high level similarities and low level differences between the Windows PE and Linux ELF binary formats.
  • How an OS loads a binary into memory and links it on the fly before executing it.

Along the way we discuss the relevance of security at different stages of a binary’s life, from the tricks that can be played by a malicious compiler, to how viruses really work, to the way which malware “packers” duplicate OS process execution functionality, to the benefit of a security-enhanced OS loader which implements address space layout randomization (ASLR).

Lab work includes:

  • Using the new “Binary Scavenger Hunt” tool which creates randomized PE binaries and asks randomized questions about the material you just learned!
  • Manipulating compiler options to change the type of assembly which is output.
  • Manipulating linker options to change the structure of binary formats.
  • Reading and understanding PE files with PEView.
  • Reading and understanding ELF files with Readelf.
  • Using WinDbg and/or GDB to watch the loader dynamically link an executable.
  • Using Thread Local Storage (TLS) to obfuscate control flow and serve as a basic anti-debug mechanism.
  • Creating a simple example virus for PE.
  • Analyze the changes made to the binary format when a file is packed with UPX.
  • Using the rootkit technique of Import Address Table (IAT) hooking to subvert the integrity of a program’s calls to external libraries, allowing files to be hidden.

Learning Objectives

  • Do a deep dive into the big picture of how binaries are executed on most OSes.
  • Provide detailed information about Windows/Linux binary formats which will be useful to future reverse engineers.
  • Show the security-relevance of knowledge about all stages of a binary's life.

Framework Connections

The materials within this course focus on the NICE Framework Task, Knowledge, and Skill statements identified within the indicated NICE Framework component(s):

Specialty Areas

  • Technology R&D