Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems, Second Edition

Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems, Second Edition

by Hung Q. Nguyen, Bob Johnson, Michael Hackett
Released June 2003
Publisher(s): Wiley
ISBN: 9780471201007

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

* Includes updates to material on testing Web applications.

* Contains new coverage of testing for wireless applications.

* From the coauthor of the bestselling testing book of all time.

* Each test type is backed up with a testing example and error examples.

Table of contents

  1. Copyright
  2. Preface
  3. Foreword
  4. Acknowledgments
  5. About the Authors
  6. 1. Introduction
    1. 1. Welcome to Web Testing
      1. 1.1. Why Read This Chapter?
      2. 1.2. Introduction
      3. 1.3. The Evolution of Software Testing
      4. 1.4. The Gray-Box Testing Approach
      5. 1.5. Real-World Software Testing
      6. 1.6. Themes of This Book
      7. 1.7. What's New in the Second Edition
        1. 1.7.1. New Contents and Significant Updates
        2. 1.7.2. What Remains from the First Edition
    2. 2. Web Testing versus Traditional Testing
      1. 2.1. Why Read This Chapter?
      2. 2.2. Introduction
      3. 2.3. The Application Model
      4. 2.4. Hardware and Software Differences
      5. 2.5. The Differences between Web and Traditional Client-Server Systems
        1. 2.5.1. Client-Side Applications
        2. 2.5.2. Event Handling
        3. 2.5.3. Application Instance and Windows Handling
        4. 2.5.4. UI Controls
      6. 2.6. Web Systems
        1. 2.6.1. Hardware Mix
        2. 2.6.2. Software Mix
        3. 2.6.3. Server-Based Applications
        4. 2.6.4. Distributed Server Configurations
        5. 2.6.5. The Network
      7. 2.7. Bug Inheritance
      8. 2.8. Back-End Data Accessing
      9. 2.9. Thin-Client versus Thick-Client Processing
      10. 2.10. Interoperability Issues
      11. 2.11. Testing Considerations
      12. 2.12. Bibliography
  7. 2. Methodology and Technology
    1. 3. Software Testing Basics
      1. 3.1. Why Read This Chapter?
      2. 3.2. Introduction
      3. 3.3. Basic Planning and Documentation
      4. 3.4. Common Terminology and Concepts
        1. 3.4.1. Test Conditions
          1. 3.4.1.1. Static Operating Environments
          2. 3.4.1.2. Dynamic Operating Environments
            1. 3.4.1.2.1. Resource Contention Example
        2. 3.4.2. Test Types
          1. 3.4.2.1. Acceptance Testing
            1. 3.4.2.1.1. Development Acceptance Test
            2. 3.4.2.1.2. Release Acceptance Test
            3. 3.4.2.1.3. Functional Acceptance Simple Test
            4. 3.4.2.1.4. Deployment Acceptance Test
          2. 3.4.2.2. Feature-Level Testing
            1. 3.4.2.2.1. Task-Oriented Functional Test
            2. 3.4.2.2.2. Forced-Error Test
            3. 3.4.2.2.3. Boundary Test
            4. 3.4.2.2.4. System-Level Test
            5. 3.4.2.2.5. Real-World User-Level Test
            6. 3.4.2.2.6. Exploratory Test
            7. 3.4.2.2.7. Load/Volume Test
            8. 3.4.2.2.8. Stress Test
            9. 3.4.2.2.9. Performance Test
            10. 3.4.2.2.10. Fail-over Test
            11. 3.4.2.2.11. Availability Test
            12. 3.4.2.2.12. Reliability Test
            13. 3.4.2.2.13. Scalability Testing
            14. 3.4.2.2.14. API Test
            15. 3.4.2.2.15. Regression Test
            16. 3.4.2.2.16. Compatibility and Configuration Test
            17. 3.4.2.2.17. Documentation Test
            18. 3.4.2.2.18. Online Help Test
            19. 3.4.2.2.19. Utilities/Toolkits and Collateral Test
            20. 3.4.2.2.20. Install/Uninstall Test
            21. 3.4.2.2.21. User Interface Tests
            22. 3.4.2.2.22. Usability Tests
            23. 3.4.2.2.23. Accessibility Tests
            24. 3.4.2.2.24. External Beta Testing
            25. 3.4.2.2.25. Dates Testing
            26. 3.4.2.2.26. Security Tests
            27. 3.4.2.2.27. Unit Tests
        3. 3.4.3. Phases of Development
      5. 3.5. Test-Case Development
        1. 3.5.1. Equivalence Class Partitioning and Boundary Condition Analysis
        2. 3.5.2. State Transition
        3. 3.5.3. Use Cases
          1. 3.5.3.1. Example Test Cases from Use Cases
          2. 3.5.3.2. Test Cases Built from Use Cases
          3. 3.5.3.3. Templates for Use-Case Diagram, Text, and Test Case
        4. 3.5.4. Condition Combination
        5. 3.5.5. The Combinatorial Method
      6. 3.6. Bibliography
    2. 4. Networking Basics
      1. 4.1. Why Read This Chapter?
      2. 4.2. Introduction
      3. 4.3. The Basics
        1. 4.3.1. The Networks
        2. 4.3.2. The Internet
          1. 4.3.2.1. Local Area Networks (LANs)
          2. 4.3.2.2. Wide Area Networks (WANs)
          3. 4.3.2.3. Connecting Networks
          4. 4.3.2.4. Connectivity Services
            1. 4.3.2.4.1. Dial-Up Connection
          5. 4.3.2.5. Direct Connection
            1. 4.3.2.5.1. Internet Connection Hardware
          6. 4.3.2.6. Other Network Connectivity Devices
        3. 4.3.3. TCP/IP Protocols
          1. 4.3.3.1. The TCP/IP Architecture
            1. 4.3.3.1.1. The Application Layer
            2. 4.3.3.1.2. The Transport Layer
            3. 4.3.3.1.3. The Internet Layer
            4. 4.3.3.1.4. The Data Link Layer
            5. 4.3.3.1.5. The Physical Layer
        4. 4.3.4. Testing Scenarios
          1. 4.3.4.1. Connection Type Testing
            1. 4.3.4.1.1. Potential Modem Compatibility Issues
            2. 4.3.4.1.2. Potential Dialer Compatibility Issues
          2. 4.3.4.2. Connectivity Device Testing
      4. 4.4. Other Useful Information
        1. 4.4.1. IP Addresses and DNS
          1. 4.4.1.1. IP Address
          2. 4.4.1.2. Network Classes
          3. 4.4.1.3. Domain Name System (DNS)
        2. 4.4.2. Subnet
          1. 4.4.2.1. Subnet Masks
          2. 4.4.2.2. Custom Subnets
        3. 4.4.3. A Testing Example
          1. 4.4.3.1. Host Name and IP Resolution Tests
      5. 4.5. Testing Considerations
      6. 4.6. Bibliography
    3. 5. Web Application Components
      1. 5.1. Why Read This Chapter?
      2. 5.2. Introduction
      3. 5.3. Overview
        1. 5.3.1. Distributed Application Architecture
          1. 5.3.1.1. Traditional Client-Server Systems
          2. 5.3.1.2. Thin- versus Thick-Client Systems
          3. 5.3.1.3. Web-Based Client-Server Systems
        2. 5.3.2. Software Components
          1. 5.3.2.1. Operating Systems
          2. 5.3.2.2. Application Service Components
          3. 5.3.2.3. Third-Party Components
          4. 5.3.2.4. Integrated Application Components
        3. 5.3.3. Dynamic Link Library (DLL)
          1. 5.3.3.1. Potential DLL-Related Errors
          2. 5.3.3.2. Scripts
      4. 5.4. Web Application Component Architecture
        1. 5.4.1. Server-Side Components
          1. 5.4.1.1. Core Application Service Components
            1. 5.4.1.1.1. Web Servers
            2. 5.4.1.1.2. Database Servers
            3. 5.4.1.1.3. Application Servers
          2. 5.4.1.2. Markup Language Pages
          3. 5.4.1.3. XML with SOAP
          4. 5.4.1.4. Web-to-Database Connectivity
            1. 5.4.1.4.1. Common Gateway Interface (CGI)
            2. 5.4.1.4.2. Web Server Extension-Based Programs
            3. 5.4.1.4.3. Web Server Extension-Based Scripts
            4. 5.4.1.4.4. ASP/JSP versus CGI
            5. 5.4.1.4.5. ASP/JSP versus Web Server Extension-Based Programs
          5. 5.4.1.5. Other Application Service Components
            1. 5.4.1.5.1. Search Servers
            2. 5.4.1.5.2. Proxy Servers and Firewalls
            3. 5.4.1.5.3. Communication-Related Servers
            4. 5.4.1.5.4. E-Commerce-Related Servers
            5. 5.4.1.5.5. Multimedia-Related Servers
        2. 5.4.2. Client-Side Components
          1. 5.4.2.1. Web Browsers
          2. 5.4.2.2. Add-on/Plug-in Components
            1. 5.4.2.2.1. Communication-Related Components
      5. 5.5. Testing Discussion
        1. 5.5.1. Test-Case Design Analysis
        2. 5.5.2. Test Partitioning
      6. 5.6. Testing Considerations
        1. 5.6.1. DLL Testing Issues
        2. 5.6.2. Script Testing Issues
          1. 5.6.2.1. Characteristics of a Script
          2. 5.6.2.2. Use of Scripts in Web Applications
          3. 5.6.2.3. Testing Scripts in Web Applications
          4. 5.6.2.4. Coding-Related Problems
          5. 5.6.2.5. Script Configuration Testing
      7. 5.7. Bibliography
    4. 6. Mobile Web Application Platform
      1. 6.1. Why Read This Chapter?
      2. 6.2. Introduction
      3. 6.3. What Is a Mobile Web Application?
      4. 6.4. Various Types of Mobile Web Client
        1. 6.4.1. Palm-Sized PDA Devices
        2. 6.4.2. Data Synchronizing
        3. 6.4.3. Web Connectivity
        4. 6.4.4. Various Types of Palm-Sized PDA Devices
        5. 6.4.5. Handheld PCs
      5. 6.5. WAP-Based Phones
        1. 6.5.1. i-Mode Devices
        2. 6.5.2. Smart Phones or Mobile Phone/PDA Combos
      6. 6.6. Mobile Web Application Platform Test Planning Issues
        1. 6.6.1. Microbrowsers
        2. 6.6.2. Web Clipping Application: How Does It Work?
        3. 6.6.3. Handheld Device Hardware Restrictions
        4. 6.6.4. Software-Related Issues
        5. 6.6.5. Wireless Network Issues
          1. 6.6.5.1. Wireless Network Standards
            1. 6.6.5.1.1. 1G
            2. 6.6.5.1.2. 2G
            3. 6.6.5.1.3. 3G/2.5G
            4. 6.6.5.1.4. Ready for 4G?
          2. 6.6.5.2. Wireless Modem
          3. 6.6.5.3. Wireless LAN and Bluetooth
        6. 6.6.6. Other Software Development Platforms and Support Infrastructures
      7. 6.7. The Device Technology Converging Game: Who Is the Winner?
      8. 6.8. Bibliography and Additional Resources
        1. 6.8.1. Bibliography
        2. 6.8.2. Additional Resources
    5. 7. Test Planning Fundamentals
      1. 7.1. Why Read This Chapter?
      2. 7.2. Introduction
      3. 7.3. Test Plans
        1. 7.3.1. Test-Plan Documentation
        2. 7.3.2. Test-Plan Templates
        3. 7.3.3. Test-Plan Section Definitions
      4. 7.4. LogiGear One-Page Test Plan
        1. 7.4.1. Developing a One-Page Test Plan
          1. 7.4.1.1. Step 1: Test Task Definition
          2. 7.4.1.2. Step 2: Task Completion Time
          3. 7.4.1.3. Step 3: Placing the Test Tasks into Context
          4. 7.4.1.4. Step 4: Table Completion
          5. 7.4.1.5. Step 5: Resource Estimation
        2. 7.4.2. Using the LogiGear One-Page Test Plan
      5. 7.5. Testing Considerations
        1. 7.5.1. Issue Reports
        2. 7.5.2. Weekly Status Reports
        3. 7.5.3. Automated Testing
        4. 7.5.4. Milestone Criteria and Milestone Tests
      6. 7.6. Bibliography
    6. 8. Sample Application
      1. 8.1. Why Read This Chapter?
      2. 8.2. Introduction
      3. 8.3. Application Description
      4. 8.4. Technical Overview
      5. 8.5. System Requirements
      6. 8.6. Functionality of the Sample Application
        1. 8.6.1. Installing the Sample Application
        2. 8.6.2. Getting Started
        3. 8.6.3. Division Databases
        4. 8.6.4. Importing Report Data
        5. 8.6.5. System Setup
        6. 8.6.6. Project Setup
        7. 8.6.7. E-Mail Notification
        8. 8.6.8. Submitting Defect Reports
        9. 8.6.9. Generating Metrics
        10. 8.6.10. Documentation
      7. 8.7. Bibliography
    7. 9. Sample Test Plan
      1. 9.1. Why Read This Chapter?
      2. 9.2. Introduction
      3. 9.3. Gathering Information
        1. 9.3.1. Step 1: Testing-Task Definitions for the Sample Application
        2. 9.3.2. Step 2: Task Completion Time
        3. 9.3.3. Step 3: Placing Test Tasks into the Project Plan
        4. 9.3.4. Step 4: Calculate Hours and Resource Estimates
      4. 9.4. Sample One-Page Test Plan
      5. 9.5. Bibliography
  8. 3. Testing Practice
    1. 10. User Interface Tests
      1. 10.1. Why Read This Chapter?
      2. 10.2. Introduction
      3. 10.3. User Interface Design Testing
        1. 10.3.1. Profiling the Target User
          1. 10.3.1.1. Computer Experience
          2. 10.3.1.2. Web Experience
          3. 10.3.1.3. Domain Knowledge
          4. 10.3.1.4. Application-Specific Experience
        2. 10.3.2. Considering the Design
          1. 10.3.2.1. Design Approach
          2. 10.3.2.2. User Interaction (Data Input)
            1. 10.3.2.2.1. User Interface Controls
            2. 10.3.2.2.2. Dynamic User Interface Controls
            3. 10.3.2.2.3. Navigation Methods
            4. 10.3.2.2.4. Mouse/Keyboard Action Matrices
            5. 10.3.2.2.5. Action Commands
            6. 10.3.2.2.6. Feedback and Error Messages
          3. 10.3.2.3. Data Presentation (Data Output)
      4. 10.4. User Interface Implementation Testing
        1. 10.4.1. Miscellaneous User Interface Elements
          1. 10.4.1.1. Display Compatibility Matrix
      5. 10.5. Usability and Accessibility Testing
        1. 10.5.1. Accessibility Testing
      6. 10.6. Testing Considerations
      7. 10.7. Bibliography and Additional Resources
        1. 10.7.1. Bibliography
        2. 10.7.2. Recommended Reading
        3. 10.7.3. Useful Links
    2. 11. Functional Tests
      1. 11.1. Why Read This Chapter?
      2. 11.2. Introduction
      3. 11.3. An Example of Cataloging Features in Preparation for Functional Tests
        1. 11.3.1. Testing the Sample Application
      4. 11.4. Testing Methods
        1. 11.4.1. Functional Acceptance Simple Tests
        2. 11.4.2. Task-Oriented Functional Tests
        3. 11.4.3. Forced-Error Tests
        4. 11.4.4. Boundary Condition Tests and Equivalent Class Analysis
        5. 11.4.5. Exploratory Testing
        6. 11.4.6. Software Attacks
        7. 11.4.7. Which Method Is It?
      5. 11.5. Bibliography
    3. 12. Server-Side Testing
      1. 12.1. Why Read This Chapter?
      2. 12.2. Introduction
      3. 12.3. Common Server-Side Testing Issues
        1. 12.3.1. Connectivity Issues
          1. 12.3.1.1. Time-Out Issues
          2. 12.3.1.2. Maintaining State
        2. 12.3.2. Resource Issues
        3. 12.3.3. Backup and Restore Issues
        4. 12.3.4. Fail-over Issues
        5. 12.3.5. Multithreading Issues
      4. 12.4. Server Side Testing Tips
        1. 12.4.1. Using Log Files
        2. 12.4.2. Using Monitoring Tools
        3. 12.4.3. Creating Test Interfaces or Test Drivers
        4. 12.4.4. The Testing Environment
          1. 12.4.4.1. Working with Live Systems
          2. 12.4.4.2. Resetting the Server
        5. 12.4.5. Using Scripts in Server-Side Testing
      5. 12.5. Bibliography
        1. 12.5.1. Additional Resources
          1. 12.5.1.1. Testing Tools for Run-Time Testing
    4. 13. Using Scripts to Test
      1. 13.1. Why Read This Chapter?
      2. 13.2. Introduction
      3. 13.3. Batch or Shell Commands
        1. 13.3.1. Batch Files and Shell Scripts
      4. 13.4. Scripting Languages
        1. 13.4.1. Why Not Just Use a Compiled Program Language?
        2. 13.4.2. What Should You Script?
      5. 13.5. Application of Scripting to Testing Tasks
        1. 13.5.1. System Administration: Automating Tasks
        2. 13.5.2. Discovering Information about the System
        3. 13.5.3. Testing the Server Directly: Making Server-Side Requests
        4. 13.5.4. Working with the Application Independent of the UI
        5. 13.5.5. Examining Data: Log Files and Reports
        6. 13.5.6. Using Scripts to Understand Test Rexssults
        7. 13.5.7. Using Scripts to Improve Productivity
          1. 13.5.7.1. A Script to Test Many Files
          2. 13.5.7.2. A Set of Scripts That Run Many Times
        8. 13.5.8. Executing Tests That Cannot Be Run Manually
      6. 13.6. Scripting Project Good Practice
      7. 13.7. Scripting Good Practice
      8. 13.8. Resource Lists
        1. 13.8.1. General Resources for Learning More about Scripting
        2. 13.8.2. Windows Script Host (WSH)
        3. 13.8.3. Batch and Shell
        4. 13.8.4. Perl
        5. 13.8.5. Tcl
        6. 13.8.6. AWK
        7. 13.8.7. Learn SQL
        8. 13.8.8. Where to Find Tools and Download Scripts
      9. 13.9. Bibliography and Useful Reading
    5. 14. Database Tests
      1. 14.1. Why Read This Chapter?
      2. 14.2. Introduction
      3. 14.3. Relational Database Servers
        1. 14.3.1. Structured Query Language
          1. 14.3.1.1. Database Producers and Standards
          2. 14.3.1.2. Database Extensions
          3. 14.3.1.3. Example of SQL
      4. 14.4. Client/SQL Interfacing
        1. 14.4.1. Microsoft Approach to CLI
        2. 14.4.2. Java Approach to CLI
      5. 14.5. Testing Methods
        1. 14.5.1. Common Types of Errors to Look For
        2. 14.5.2. Database Stored Procedures and Triggers
        3. 14.5.3. White-Box Methods
          1. 14.5.3.1. Code Walk-through
          2. 14.5.3.2. Redundancy Coding Error Example
          3. 14.5.3.3. Inefficiency Coding Error Example
          4. 14.5.3.4. Executing the SQL Statements One at a Time
          5. 14.5.3.5. Executing the Stored Procedures One at a Time
          6. 14.5.3.6. Testing Triggers
          7. 14.5.3.7. External Interfacing
        4. 14.5.4. Black-Box Methods
          1. 14.5.4.1. Designing Test Cases
          2. 14.5.4.2. Testing for Transaction Logic
          3. 14.5.4.3. Testing for Concurrency Issues
          4. 14.5.4.4. Preparation for Database Testing
          5. 14.5.4.5. Setup/Installation Issues
          6. 14.5.4.6. Testing with a Clean Database
      6. 14.6. Database Testing Considerations
      7. 14.7. Bibliography and Additional Resources
        1. 14.7.1. Bibliography
        2. 14.7.2. Additional Resources
    6. 15. Help Tests
      1. 15.1. Why Read This Chapter?
      2. 15.2. Introduction
      3. 15.3. Help System Analysis
        1. 15.3.1. Types of Help Systems
          1. 15.3.1.1. Application Help Systems
          2. 15.3.1.2. Reference Help Systems
          3. 15.3.1.3. Tutorial Help Systems
          4. 15.3.1.4. Sales and Marketing Help Systems
        2. 15.3.2. Evaluating the Target User
        3. 15.3.3. Evaluating the Design Approach
        4. 15.3.4. Evaluating the Technologies
          1. 15.3.4.1. Standard HTML (W3 Standard)
          2. 15.3.4.2. Java Applets
          3. 15.3.4.3. Netscape NetHelp
          4. 15.3.4.4. ActiveX Controls
          5. 15.3.4.5. Help Elements
      4. 15.4. Approaching Help Testing
        1. 15.4.1. Two-Tiered Testing
          1. 15.4.1.1. Stand-alone Testing
          2. 15.4.1.2. Interaction between the Application and the Help System
        2. 15.4.2. Types of Help Errors
      5. 15.5. Testing Considerations
      6. 15.6. Bibliography
    7. 16. Installation Tests
      1. 16.1. Why Read This Chapter?
      2. 16.2. Introduction
      3. 16.3. The Roles of Installation/Uninstallation Programs
        1. 16.3.1. Installer
        2. 16.3.2. Uninstaller
      4. 16.4. Common Features and Options
        1. 16.4.1. User Setup Options
        2. 16.4.2. Installation Sources and Destinations
          1. 16.4.2.1. Server Distribution Configurations
          2. 16.4.2.2. Server-Side Installation Example
          3. 16.4.2.3. Media Types
        3. 16.4.3. Branching Options
      5. 16.5. Common Server-Side-Specific Installation Issues
      6. 16.6. Installer/Uninstaller Testing Utilities
        1. 16.6.1. Comparison-Based Testing Tools
        2. 16.6.2. InControl4 and InControl5
        3. 16.6.3. Norton Utilities' Registry Tracker and File Compare
      7. 16.7. Testing Considerations
      8. 16.8. Bibliography and Additional Resources
        1. 16.8.1. Additional Resources
    8. 17. Configuration and Compatibility Tests
      1. 17.1. Why Read This Chapter?
      2. 17.2. Introduction
      3. 17.3. The Test Cases
      4. 17.4. Approaching Configuration and Compatibility Testing
        1. 17.4.1. Considering Target Users
        2. 17.4.2. When to Run Compatibility and Configuration Testing
        3. 17.4.3. Potential Outsourcing
      5. 17.5. Comparing Configuration Testing with Compatibility Testing
      6. 17.6. Configuration/Compatibility Testing Issues
        1. 17.6.1. COTS Products versus Hosted Systems
        2. 17.6.2. Distributed Server Configurations
        3. 17.6.3. Client-Side Issues
        4. 17.6.4. Web Browsers
      7. 17.7. Testing Considerations
      8. 17.8. Bibliography
      9. 17.9. Additional Resources
    9. 18. Web Security Testing
      1. 18.1. Why Read This Chapter?
      2. 18.2. Introduction
        1. 18.2.1. What Is Computer Security?
      3. 18.3. Security Goals
        1. 18.3.1. From Which Threats Are We Protecting Ourselves?
        2. 18.3.2. Common Sources of Security Threats
        3. 18.3.3. What Is the Potential Damage?
      4. 18.4. Anatomy of an Attack
        1. 18.4.1. Information Gathering
        2. 18.4.2. Network Scanning
        3. 18.4.3. Attacking
      5. 18.5. Attacking Intents
      6. 18.6. Security Solution Basics
        1. 18.6.1. Strategies, People, and Processes
          1. 18.6.1.1. Education
          2. 18.6.1.2. Corporate Security Policies
          3. 18.6.1.3. Corporate Responses
        2. 18.6.2. Authentication and Authorization
          1. 18.6.2.1. Passwords
          2. 18.6.2.2. Authentication between Software Applications or Components
          3. 18.6.2.3. Cryptography
            1. 18.6.2.3.1. Public Key Cryptography Basics
            2. 18.6.2.3.2. Digital Certificates
            3. 18.6.2.3.3. Pretty Good Privacy
            4. 18.6.2.3.4. Secure Multipurpose Internet Mail Extensions (S/MIME)
          4. 18.6.2.4. Other Web Security Technologies
        3. 18.6.3. Perimeter-Based Security: Firewalls, DMZs, and Intrusion Detection Systems
          1. 18.6.3.1. Firewalls
            1. 18.6.3.1.1. Packet-Screening Firewalls (Routers)
            2. 18.6.3.1.2. Proxy-Based Firewalls (Gateways)
          2. 18.6.3.2. Setting Up a DMZ
          3. 18.6.3.3. Intrusion Detection Systems (IDS)
      7. 18.7. Common Vulnerabilities and Attacks
        1. 18.7.1. Software Bugs, Poor Design, and Programming Practice
          1. 18.7.1.1. Buffer Overflows
          2. 18.7.1.2. Malicious Input Data
          3. 18.7.1.3. Command-Line (Shell) Execution
          4. 18.7.1.4. Backdoors
          5. 18.7.1.5. JavaScript
          6. 18.7.1.6. CGI Programs
          7. 18.7.1.7. Java
          8. 18.7.1.8. ActiveX
        2. 18.7.2. Cookies
        3. 18.7.3. Spoofing
        4. 18.7.4. Malicious Programs
          1. 18.7.4.1. Virus and Worm
          2. 18.7.4.2. Trojan Horses
        5. 18.7.5. Misuse Access Privilege Attacks
        6. 18.7.6. Password Cracking
        7. 18.7.7. Denial-of-Service Attacks
        8. 18.7.8. Physical Attacks
        9. 18.7.9. Exploiting the Trust Computational Base
        10. 18.7.10. Information Leaks
          1. 18.7.10.1. Social Engineering
          2. 18.7.10.2. Keystroke Capturing
          3. 18.7.10.3. Garbage Rummaging
          4. 18.7.10.4. Packet Sniffing
          5. 18.7.10.5. Scanning and Probing
          6. 18.7.10.6. Network Mapping
        11. 18.7.11. Network Attacks
      8. 18.8. Testing Goals and Responsibilities
        1. 18.8.1. Functionality Side Effect: An Error-Handling Bug Example
      9. 18.9. Testing for Security
        1. 18.9.1. Testing the Requirements and Design
          1. 18.9.1.1. Requirements Are Key
          2. 18.9.1.2. Trusted Computational Base (TCB)
          3. 18.9.1.3. Access Control
          4. 18.9.1.4. Which Resources Need to Be Protected?
          5. 18.9.1.5. Client Privacy Issues: What Information Needs to Be Private?
        2. 18.9.2. Testing the Application Code
          1. 18.9.2.1. Backdoors
          2. 18.9.2.2. Exception Handling and Failure Notification
          3. 18.9.2.3. ID and Password Testing
          4. 18.9.2.4. Testing for Information Leaks
          5. 18.9.2.5. Random Numbers versus Unique Numbers
          6. 18.9.2.6. Testing the Use of GET and POST
          7. 18.9.2.7. Parameter-Tampering Attacks
        3. 18.9.3. SQL Injection Attacks
          1. 18.9.3.1. Cookie Attacks
          2. 18.9.3.2. Testing for Buffer Overflows
          3. 18.9.3.3. Testing for Bad Data
          4. 18.9.3.4. Reliance on Client-Side Scripting
          5. 18.9.3.5. When Input Becomes Output
        4. 18.9.4. Testing Third-Party Code
          1. 18.9.4.1. Known Vulnerabilities
          2. 18.9.4.2. Race Conditions
        5. 18.9.5. Testing the Deployment
          1. 18.9.5.1. Installation Defaults
          2. 18.9.5.2. Default Passwords
          3. 18.9.5.3. Internationalization
          4. 18.9.5.4. Program Forensics
          5. 18.9.5.5. Working with Customer Support Folks
        6. 18.9.6. Penetration Testing
        7. 18.9.7. Testing with User Protection via Browser Settings
          1. 18.9.7.1. Testing with Firewalls
        8. 18.9.8. The Challenges Testers Face
      10. 18.10. Other Testing Considerations
      11. 18.11. Bibliography and Additional Resources
        1. 18.11.1. Bibliography
        2. 18.11.2. Additional Resources
        3. 18.11.3. Useful Net Resources
        4. 18.11.4. Tools
    10. 19. Performance Testing
      1. 19.1. Why Read This Chapter?
      2. 19.2. Introduction
      3. 19.3. Performance Testing Concepts
        1. 19.3.1. Determining Acceptable Response Time or Acceptable User Experience
        2. 19.3.2. Response Time Definition
        3. 19.3.3. Performance and Load Stress Testing Definitions
        4. 19.3.4. Searching for Answers
        5. 19.3.5. A Simple Example
      4. 19.4. Performance Testing Key Factors
        1. 19.4.1. Workload
        2. 19.4.2. System Environment and Available Resources
        3. 19.4.3. Response Time
        4. 19.4.4. Key Factors Affecting Response Time or Performance
      5. 19.5. Three Phases of Performance Testing
      6. 19.6. Setting Goals and Expectations and Defining Deliverables
      7. 19.7. Gathering Requirements
        1. 19.7.1. What Are You Up Against?
        2. 19.7.2. What If Written Requirements Don't Exist?
      8. 19.8. Defining the Workload
      9. 19.9. Sizing the Workload
        1. 19.9.1. Server-Based Profile
        2. 19.9.2. User-Based Profile
      10. 19.10. Problems Concerning Workloads
      11. 19.11. Selecting Performance Metrics
        1. 19.11.1. Throughput Calculation Example
      12. 19.12. Which Tests to Run and When to Start
      13. 19.13. Tool Options and Generating Loads
        1. 19.13.1. Tool Options
        2. 19.13.2. Analyzing and Reporting Collected Data
        3. 19.13.3. Generating Loads
      14. 19.14. Writing the Test Plan
        1. 19.14.1. Identifying Baseline Configuration and Performance Requirements
        2. 19.14.2. Determining the Workload
        3. 19.14.3. Determining When to Begin Testing
        4. 19.14.4. Determine Whether the Testing Process Will Be Hardware-Intensive or Software-Intensive
        5. 19.14.5. Developing Test Cases
      15. 19.15. Testing Phase
        1. 19.15.1. Generating Test Data
        2. 19.15.2. Setting Up the Test Bed
        3. 19.15.3. Setting Up the Test Suite Parameters
        4. 19.15.4. Performance Test Run Example
      16. 19.16. Analysis Phase
      17. 19.17. Other Testing Considerations
      18. 19.18. Bibliography
    11. 20. Testing Mobile Web Applications
      1. 20.1. Why Read This Chapter?
      2. 20.2. Introduction
      3. 20.3. Testing Mobile versus Desktop Web Applications
      4. 20.4. Various Types of Tests
        1. 20.4.1. Add-on Installation Tests
        2. 20.4.2. Data Synchronization-Related Tests
        3. 20.4.3. UI Implementation and Limited Usability Tests
          1. 20.4.3.1. UI Guideline References
        4. 20.4.4. Browser-Specific Tests
        5. 20.4.5. Platform-Specific Tests
          1. 20.4.5.1. Platform or Logo Compliance Tests
        6. 20.4.6. Configuration and Compatibility Tests
        7. 20.4.7. Connectivity Tests
          1. 20.4.7.1. Devices with Peripheral Network Connections
          2. 20.4.7.2. Latency
          3. 20.4.7.3. Transmission Errors
          4. 20.4.7.4. Transitions from Coverage to No-Coverage Areas
          5. 20.4.7.5. Transitions between Data and Voice
          6. 20.4.7.6. Data or Message Race Condition
          7. 20.4.7.7. Performance Tests
        8. 20.4.8. Security Tests
        9. 20.4.9. Testing Web Applications Using an Emulation Environment
        10. 20.4.10. Testing Web Applications Using the Physical Environment
      5. 20.5. Survey of Mobile Testing Support Tools
        1. 20.5.1. Device and Browser Emulators
          1. 20.5.1.1. Palm Computing
          2. 20.5.1.2. OpenWave
          3. 20.5.1.3. Nokia
          4. 20.5.1.4. YoSpace
          5. 20.5.1.5. Microsoft
        2. 20.5.2. Web-Based Mobile Phone Emulators and WML Validators
        3. 20.5.3. Desktop WAP Browsers
      6. 20.6. Other Testing Considerations
      7. 20.7. Bibliography and Additional Resources
        1. 20.7.1. Bibliography
        2. 20.7.2. Additional Resources
    12. 21. Web Testing Tools
      1. 21.1. Why Read This Chapter?
      2. 21.2. Introduction
      3. 21.3. Types of Tools
        1. 21.3.1. Rule-Based Analyzers
          1. 21.3.1.1. Sample List of Link Checkers and HTML Validators
          2. 21.3.1.2. Sample List of Rule-Based Analyzers for C/C++, Java, Visual Basic, and Other Programming and Scripting Languages
        2. 21.3.2. Load/Performance Testing Tools
          1. 21.3.2.1. Web Load and Performance Testing Tools
        3. 21.3.3. GUI Capture (Recording/Scripting) and Playback Tools
          1. 21.3.3.1. Sample List of Automated GUI Functional and Regression Testing Tools
        4. 21.3.4. Runtime Error Detectors
          1. 21.3.4.1. Sample List of Runtime Error-Detection Tools
        5. 21.3.5. Sample List of Web Security Testing Tools
        6. 21.3.6. Java-Specific Testing Tools
        7. 21.3.7. Other Types of Useful Tools
        8. 21.3.8. Database Testing Tools
        9. 21.3.9. Defect Management Tool Vendors
          1. 21.3.9.1. QACity.Com Comprehensive List of DEFECT TRACKING Tool Vendors
      4. 21.4. Additional Resources
        1. 21.4.1. On the Internet
        2. 21.4.2. Development and Testing Tool Mail-Order Catalogs
    13. 22. Finding Additional Information
      1. 22.1. Why Read This Chapter?
      2. 22.2. Introduction
      3. 22.3. Textbooks
      4. 22.4. Web Resources
        1. 22.4.1. Useful Links
        2. 22.4.2. Useful Magazines and Newsletters
        3. 22.4.3. Miscellaneous Papers on the Web from Carnegie Mellon University's Software Engineering Institute
      5. 22.5. Professional Societies
    14. A. LogiGear Test Plan Template
      1. A.1. I. Overview
        1. A.1.1. 1. Test Plan Identifier
        2. A.1.2. 2. Introduction
        3. A.1.3. 3. Objective
        4. A.1.4. 4. Approach
      2. A.2. II. Testing Synopsis
        1. A.2.1. 1. Test Items
          1. A.2.1.1. 1.1.1. Main Application Executables
          2. A.2.1.2. 1.2. Software Collateral Items
          3. A.2.1.3. 1.3. Publishing Items
        2. A.2.2. 2. Features to Be Tested
        3. A.2.3. 3. Features Not to Be Tested
        4. A.2.4. 4. System Requirements
        5. A.2.5. 5. Product Entrance/Exit
        6. A.2.6. 6. Standard/Reference
        7. A.2.7. 7. Test Deliverables
        8. A.2.8. 7.1. Test Plan
          1. A.2.8.1. 7.1.1. The Original Approved Development Test Plan
          2. A.2.8.2. 7.1.2. The Executed Development Test Plan
          3. A.2.8.3. 7.1.3. The Original Approved Final Test Plan
          4. A.2.8.4. 7.1.4. The Executed Final Test Plan
        9. A.2.9. 7.2. Bug-Tracking System
          1. A.2.9.1. 7.2.1. Bug Reports
          2. A.2.9.2. 7.2.2. Bug Database
        10. A.2.10. 7.3. Final Release Report
      3. A.3. III. Testing Project Management
        1. A.3.1. 1. The Product Team
        2. A.3.2. 2. Testing Responsibilities
        3. A.3.3. 3. Testing Tasks
        4. A.3.4. 4. Development Plan and Schedule
        5. A.3.5. 5. Milestone Entrance/Exit Criteria
        6. A.3.6. 6. Test Schedule and Resource
          1. A.3.6.1. 6.1. Schedule
          2. A.3.6.2. 6.2. Resource Estimate
        7. A.3.7. 7. Training Needs
        8. A.3.8. 8. Environmental Needs
          1. A.3.8.1. 8.1. Test Components
          2. A.3.8.2. 8.2. Test Tools
          3. A.3.8.3. 8.3. Facilities
        9. A.3.9. 9. Integration Plan
        10. A.3.10. 10. Test Suspension and Resumption
        11. A.3.11. 11. Test Completion Criteria
        12. A.3.12. 12. The Problem-Tracking Process
          1. A.3.12.1. 12.1. The Process
          2. A.3.12.2. 12.2. The Bug-Tracking Tool (database)
          3. A.3.12.3. 12.3. Definition of Bug Severity
            1. A.3.12.3.1. 12.3.1. 1—Critical
            2. A.3.12.3.2. 12.3.2. 2—Serious
            3. A.3.12.3.3. 12.3.3. 3—Noncritical
        13. A.3.13. 13. Status Tracking and Reporting
        14. A.3.14. 14. Risks and Contingencies
        15. A.3.15. 15. The Approval Process
          1. A.3.15.1. 15.1. Test Plan Approval
          2. A.3.15.2. 15.2. Final Release Approval
      4. A.4. Appendix 1: Setup/Installation Test Case
      5. A.5. Appendix 2: Test Case for Application Shell
      6. A.6. Appendix 3: Test Matrix for XXXXX
      7. A.7. Appendix 4: Compatibility Test Systems
    15. B. Weekly Status Report Template
      1. B.1. I. Testing Project Management
        1. B.1.1. 1. Project Schedule*
        2. B.1.2. 2. Progress and Changes Since Last Week
        3. B.1.3. 3. Urgent Items
        4. B.1.4. 4. Issue Bin
        5. B.1.5. 5. To-do Tasks by Next Report
      2. B.2. II. Problem Report Status
        1. B.2.1. 1. Bug Report Tabulation
        2. B.2.2. 2. Summary List of Open Bugs
      3. B.3. III. Trend Analysis Report
        1. B.3.1. Stability Trend Chart
        2. B.3.2. Quality Trend Chart
    16. C. Error Analysis Checklist: Web Error Examples
      1. C.1. Check for the Existence of DNS Entry
        1. C.1.1. Symptoms
        2. C.1.2. Possible Problems
        3. C.1.3. Examples
        4. C.1.4. Tips
      2. C.2. Check for Proper Configuration of the Client Machine and Browser Settings
        1. C.2.1. Symptoms
        2. C.2.2. Possible Problems
        3. C.2.3. Examples
        4. C.2.4. Tips
      3. C.3. Check the HTTP Settings on the Browser
        1. C.3.1. Symptoms
        2. C.3.2. Possible Problems
        3. C.3.3. Examples
        4. C.3.4. Tips
      4. C.4. Check the JIT Compiler-Enabled Setting on the Browser
        1. C.4.1. Symptoms
        2. C.4.2. Possible Problems
        3. C.4.3. Examples
        4. C.4.4. Tip
      5. C.5. Check the Multimedia Settings on the Browser
        1. C.5.1. Symptoms
        2. C.5.2. Possible Problems
        3. C.5.3. Examples
        4. C.5.4. Tip
      6. C.6. Check the Security Settings on the Browser
        1. C.6.1. Symptoms
        2. C.6.2. Possible Problems
        3. C.6.3. Examples
        4. C.6.4. Tip
      7. C.7. Check for a Slow Connection
        1. C.7.1. Symptoms
        2. C.7.2. Possible Problems
        3. C.7.3. Examples
        4. C.7.4. Tips
      8. C.8. Check for Proper Configuration on the Web Server
        1. C.8.1. Symptoms
        2. C.8.2. Possible Problems
        3. C.8.3. Examples
        4. C.8.4. Tips
      9. C.9. Check for the Deletion of Your Browser Cache (Temporary Internet Files)
        1. C.9.1. Symptoms
        2. C.9.2. Possible Problems
        3. C.9.3. Examples
        4. C.9.4. Tip
    17. D. UI Test-Case Design Guideline: Common Keyboard Navigation and Shortcut Matrix
    18. E. UI Test-Case Design Guideline: Mouse Action Matrix
    19. F. Web Test-Case Design Guideline: Input Boundary and Validation Matrix I
    20. G. Display Compatibility Test Matrix
    21. H. Browser OS Configuration Matrix

Product information

  • Title: Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems, Second Edition
  • Author(s): Hung Q. Nguyen, Bob Johnson, Michael Hackett
  • Release date: June 2003
  • Publisher(s): Wiley
  • ISBN: 9780471201007