The IDA Pro Book, 2nd Edition

The IDA Pro Book, 2nd Edition

by Chris Eagle
Released June 2011
Publisher(s): No Starch Press
ISBN: 9781593272890

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on ebooks.com
Start your free trial

Book description

IDA Pro is a commercial disassembler and debugger used by reverse engineers to dissect compiled computer programs, and is the industry standard tool for analysis of hostile code. The IDA Pro Book provides a comprehensive, top-down overview of IDA Pro and its use for reverse engineering software. Author Chris Eagle, a recognized expert in the field, takes readers from the basics of disassembly theory to the complexities of using IDA Pro in real-world situations. Topics are introduced in the order most frequently encountered, allowing experienced users to easily jump in at the most appropriate point. Eagle covers a variety of real-world reverse engineering challenges and offers strategies to deal with them, such as disassembly manipulation, graphing, and effective use of cross references. This second edition of The IDA Pro Book has been completely updated and revised to cover the new features and cross-platform interface of IDA Pro 6.0. Other additions include expanded coverage of the IDA Pro Debugger, IDAPython, and the IDA Pro SDK.

Table of contents

  1. The IDA Pro Book
    1. PRAISE FOR THE FIRST EDITION OF THE IDA PRO BOOK
    2. Acknowledgments
    3. Introduction
    4. I. Introduction to IDA
      1. 1. Introduction to Disassembly
        1. Disassembly Theory
        2. The What of Disassembly
        3. The Why of Disassembly
          1. Malware Analysis
          2. Vulnerability Analysis
          3. Software Interoperability
          4. Compiler Validation
          5. Debugging Displays
        4. The How of Disassembly
          1. A Basic Disassembly Algorithm
          2. Linear Sweep Disassembly
          3. Recursive Descent Disassembly
            1. Sequential Flow Instructions
            2. Conditional Branching Instructions
            3. Unconditional Branching Instructions
            4. Function Call Instructions
            5. Return Instructions
        5. Summary
      2. 2. Reversing and Disassembly Tools
        1. Classification Tools
          1. file
          2. PE Tools
          3. PEiD
        2. Summary Tools
          1. nm
          2. ldd
          3. objdump
          4. otool
          5. dumpbin
          6. c++filt
        3. Deep Inspection Tools
          1. strings
          2. Disassemblers
        4. Summary
      3. 3. IDA Pro Background
        1. Hex-Rays’ Stance on Piracy
        2. Obtaining IDA Pro
          1. IDA Versions
          2. IDA Licenses
          3. Purchasing IDA
          4. Upgrading IDA
        3. IDA Support Resources
        4. Your IDA Installation
          1. Windows Installation
          2. OS X and Linux Installation
          3. IDA and SELinux
          4. 32-bit vs. 64-bit IDA
          5. The IDA Directory Layout
        5. Thoughts on IDA’s User Interface
        6. Summary
    5. II. Basic IDA Usage
      1. 4. Getting Started with IDA
        1. Launching IDA
          1. IDA File Loading
          2. Using the Binary File Loader
        2. IDA Database Files
          1. IDA Database Creation
          2. Closing IDA Databases
          3. Reopening a Database
        3. Introduction to the IDA Desktop
        4. Desktop Behavior During Initial Analysis
        5. IDA Desktop Tips and Tricks
        6. Reporting Bugs
        7. Summary
      2. 5. IDA Data Displays
        1. The Principal IDA Displays
          1. The Disassembly Window
            1. IDA Graph View
            2. IDA Text View
          2. The Functions Window
          3. The Output Window
        2. Secondary IDA Displays
          1. The Hex View Window
          2. The Exports Window
          3. The Imports Window
          4. The Structures Window
          5. The Enums Window
        3. Tertiary IDA Displays
          1. The Strings Window
          2. The Names Window
          3. The Segments Window
          4. The Signatures Window
          5. The Type Libraries Window
          6. The Function Calls Window
          7. The Problems Window
        4. Summary
      3. 6. Disassembly Navigation
        1. Basic IDA Navigation
          1. Double-Click Navigation
          2. Jump to Address
          3. Navigation History
        2. Stack Frames
          1. Calling Conventions
            1. The C Calling Convention
            2. The Standard Calling Convention
            3. The fastcall Convention for x86
            4. C++ Calling Conventions
            5. Other Calling Conventions
          2. Local Variable Layout
          3. Stack Frame Examples
          4. IDA Stack Views
        3. Searching the Database
          1. Text Searches
          2. Binary Searches
        4. Summary
      4. 7. Disassembly Manipulation
        1. Names and Naming
          1. Parameters and Local Variables
          2. Named Locations
          3. Register Names
        2. Commenting in IDA
          1. Regular Comments
          2. Repeatable Comments
          3. Anterior and Posterior Lines
          4. Function Comments
        3. Basic Code Transformations
          1. Code Display Options
          2. Formatting Instruction Operands
          3. Manipulating Functions
            1. Creating New Functions
            2. Deleting Functions
            3. Function Chunks
            4. Function Attributes
            5. Stack Pointer Adjustments
          4. Converting Data to Code (and Vice Versa)
        4. Basic Data Transformations
          1. Specifying Data Sizes
          2. Working with Strings
          3. Specifying Arrays
        5. Summary
      5. 8. Datatypes and Data Structures
        1. Recognizing Data Structure Use
          1. Array Member Access
            1. Globally Allocated Arrays
            2. Stack-Allocated Arrays
            3. Heap-Allocated Arrays
          2. Structure Member Access
            1. Globally Allocated Structures
            2. Stack-Allocated Structures
            3. Heap-Allocated Structures
            4. Arrays of Structures
        2. Creating IDA Structures
          1. Creating a New Structure (or Union)
          2. Editing Structure Members
          3. Stack Frames as Specialized Structures
        3. Using Structure Templates
        4. Importing New Structures
          1. Parsing C Structure Declarations
          2. Parsing C Header Files
        5. Using Standard Structures
        6. IDA TIL Files
          1. Loading New TIL Files
          2. Sharing TIL Files
        7. C++ Reversing Primer
          1. The this Pointer
          2. Virtual Functions and Vtables
          3. The Object Life Cycle
          4. Name Mangling
          5. Runtime Type Identification
          6. Inheritance Relationships
          7. C++ Reverse Engineering References
        8. Summary
      6. 9. Cross-References and Graphing
        1. Cross-References
          1. Code Cross-References
          2. Data Cross-References
          3. Cross-Reference Lists
          4. Function Calls
        2. IDA Graphing
          1. IDA External (Third-Party) Graphing
            1. External Flowcharts
            2. External Call Graphs
            3. External Cross-Reference Graphs
            4. Custom Cross-Reference Graphs
          2. IDA’s Integrated Graph View
        3. Summary
      7. 10. The Many Faces of IDA
        1. Console Mode IDA
          1. Common Features of Console Mode
          2. Windows Console Specifics
          3. Linux Console Specifics
          4. OS X Console Specifics
        2. Using IDA’s Batch Mode
        3. Summary
    6. III. Advanced IDA Usage
      1. 11. Customizing IDA
        1. Configuration Files
          1. The Main Configuration File: ida.cfg
          2. The GUI Configuration File: idagui.cfg
          3. The Console Configuration File: idatui.cfg
        2. Additional IDA Configuration Options
          1. IDA Colors
          2. Customizing IDA Toolbars
        3. Summary
      2. 12. Library Recognition Using FLIRT Signatures
        1. Fast Library Identification and Recognition Technology
        2. Applying FLIRT Signatures
        3. Creating FLIRT Signature Files
          1. Signature-Creation Overview
          2. Identifying and Acquiring Static Libraries
          3. Creating Pattern Files
          4. Creating Signature Files
          5. Startup Signatures
        4. Summary
      3. 13. Extending IDA’s Knowledge
        1. Augmenting Function Information
          1. IDS Files
          2. Creating IDS Files
        2. Augmenting Predefined Comments with loadint
        3. Summary
      4. 14. Patching Binaries and Other IDA Limitations
        1. The Infamous Patch Program Menu
          1. Changing Individual Database Bytes
          2. Changing a Word in the Database
          3. Using the Assemble Dialog
        2. IDA Output Files and Patch Generation
          1. IDA-Generated MAP Files
          2. IDA-Generated ASM Files
          3. IDA-Generated INC Files
          4. IDA-Generated LST Files
          5. IDA-Generated EXE Files
          6. IDA-Generated DIF Files
          7. IDA-Generated HTML Files
        3. Summary
    7. IV. Extending IDA’s Capabilities
      1. 15. IDA Scripting
        1. Basic Script Execution
        2. The IDC Language
          1. IDC Variables
          2. IDC Expressions
          3. IDC Statements
          4. IDC Functions
          5. IDC Objects
          6. IDC Programs
          7. Error Handling in IDC
          8. Persistent Data Storage in IDC
        3. Associating IDC Scripts with Hotkeys
        4. Useful IDC Functions
          1. Functions for Reading and Modifying Data
          2. User Interaction Functions
          3. String-Manipulation Functions
          4. File Input/Output Functions
          5. Manipulating Database Names
          6. Functions Dealing with Functions
          7. Code Cross-Reference Functions
          8. Data Cross-Reference Functions
          9. Database Manipulation Functions
          10. Database Search Functions
          11. Disassembly Line Components
        5. IDC Scripting Examples
          1. Enumerating Functions
          2. Enumerating Instructions
          3. Enumerating Cross-References
          4. Enumerating Exported Functions
          5. Finding and Labeling Function Arguments
          6. Emulating Assembly Language Behavior
        6. IDAPython
          1. Using IDAPython
        7. IDAPython Scripting Examples
          1. Enumerating Functions
          2. Enumerating Instructions
          3. Enumerating Cross-References
          4. Enumerating Exported Functions
        8. Summary
      2. 16. The IDA Software Development Kit
        1. SDK Introduction
          1. SDK Installation
          2. SDK Layout
          3. Configuring a Build Environment
        2. The IDA Application Programming Interface
          1. Header Files Overview
          2. Netnodes
            1. Creating Netnodes
            2. Data Storage in Netnodes
            3. Deleting Netnodes and Netnode Data
          3. Useful SDK Datatypes
          4. Commonly Used SDK Functions
            1. Basic Database Access
            2. User Interface Functions
            3. Manipulating Database Names
            4. Function Manipulation
            5. Structure Manipulation
            6. Segment Manipulation
            7. Code Cross-References
            8. Data Cross-References
          5. Iteration Techniques Using the IDA API
            1. Enumerating Functions
            2. Enumerating Structure Members
            3. Enumerating Cross-References
        3. Summary
      3. 17. The IDA Plug-in Architecture
        1. Writing a Plug-in
          1. The Plug-in Life Cycle
          2. Plug-in Initialization
          3. Event Notification
          4. Plug-in Execution
        2. Building Your Plug-ins
        3. Installing Plug-ins
        4. Configuring Plug-ins
        5. Extending IDC
        6. Plug-in User Interface Options
          1. Using the SDK’s Chooser Dialogs
          2. Creating Customized Forms with the SDK
          3. Windows-Only User Interface–Generation Techniques
          4. User Interface Generation with Qt
        7. Scripted Plug-ins
        8. Summary
      4. 18. Binary Files and IDA Loader Modules
        1. Unknown File Analysis
        2. Manually Loading a Windows PE File
        3. IDA Loader Modules
        4. Writing an IDA Loader Using the SDK
          1. The Simpleton Loader
          2. Building an IDA Loader Module
          3. A pcap Loader for IDA
        5. Alternative Loader Strategies
        6. Writing a Scripted Loader
        7. Summary
      5. 19. IDA Processor Modules
        1. Python Byte Code
        2. The Python Interpreter
        3. Writing a Processor Module Using the SDK
          1. The processor_t Struct
          2. Basic Initialization of the LPH Structure
          3. The Analyzer
          4. The Emulator
          5. The Outputter
          6. Processor Notifications
          7. Other processor_t Members
        4. Building Processor Modules
        5. Customizing Existing Processors
        6. Processor Module Architecture
        7. Scripting a Processor Module
        8. Summary
    8. V. Real-World Applications
      1. 20. Compiler Personalities
        1. Jump Tables and Switch Statements
        2. RTTI Implementations
        3. Locating main
        4. Debug vs. Release Binaries
        5. Alternative Calling Conventions
        6. Summary
      2. 21. Obfuscated Code Analysis
        1. Anti–Static Analysis Techniques
          1. Disassembly Desynchronization
          2. Dynamically Computed Target Addresses
            1. Opcode Obfuscation
          3. Imported Function Obfuscation
          4. Targeted Attacks on Analysis Tools
        2. Anti–Dynamic Analysis Techniques
          1. Detecting Virtualization
          2. Detecting Instrumentation
          3. Detecting Debuggers
          4. Preventing Debugging
        3. Static De-obfuscation of Binaries Using IDA
          1. Script-Oriented De-obfuscation
          2. Emulation-Oriented De-obfuscation
            1. x86emu Initialization
            2. Basic x86emu Operation
            3. Emulator-Assisted De-obfuscation
            4. Additional x86emu Features
            5. x86emu and Anti-debugging
        4. Virtual Machine-Based Obfuscation
        5. Summary
      3. 22. Vulnerability Analysis
        1. Discovering New Vulnerabilities with IDA
        2. After-the-Fact Vulnerability Discovery with IDA
        3. IDA and the Exploit-Development Process
          1. Stack Frame Breakdown
          2. Locating Instruction Sequences
          3. Finding Useful Virtual Addresses
        4. Analyzing Shellcode
        5. Summary
      4. 23. Real-World IDA Plug-ins
        1. Hex-Rays
        2. IDAPython
        3. collabREate
        4. ida-x86emu
        5. Class Informer
        6. MyNav
        7. IdaPdf
        8. Summary
    9. VI. The IDA Debugger
      1. 24. The IDA Debugger
        1. Launching the Debugger
        2. Basic Debugger Displays
        3. Process Control
          1. Breakpoints
          2. Tracing
          3. Stack Traces
          4. Watches
        4. Automating Debugger Tasks
          1. Scripting Debugger Actions
          2. Automating Debugger Actions with IDA Plug-ins
        5. Summary
      2. 25. Disassembler/Debugger Integration
        1. Background
        2. IDA Databases and the IDA Debugger
        3. Debugging Obfuscated Code
          1. Launching the Process
          2. Simple Decryption and Decompression Loops
          3. Import Table Reconstruction
          4. Hiding the Debugger
        4. IdaStealth
        5. Dealing with Exceptions
        6. Summary
      3. 26. Additional Debugger Features
        1. Remote Debugging with IDA
          1. Using a Hex-Rays Debugging Server
          2. Attaching to a Remote Process
          3. Exception Handling During Remote Debugging
          4. Using Scripts and Plug-ins During Remote Debugging
        2. Debugging with Bochs
          1. Bochs IDB Mode
          2. Bochs PE Mode
          3. Bochs Disk Image Mode
        3. Appcall
        4. Summary
    10. A. Using IDA Freeware 5.0
      1. Restrictions on IDA Freeware
      2. Using IDA Freeware
    11. B. IDC/SDK Cross-Reference
    12. Index
    13. About the Author

Product information

  • Title: The IDA Pro Book, 2nd Edition
  • Author(s): Chris Eagle
  • Release date: June 2011
  • Publisher(s): No Starch Press
  • ISBN: 9781593272890