Practical Fairness

Table of contents

1. Preface
   1. Goals of This Book
   2. Practical Notes on the Book
   3. Conventions Used in This Book
   4. Using Code Examples
   5. O’Reilly Online Learning
   6. How to Contact Us
   7. Acknowledgments
2. 1. Fairness, Technology, and the Real World
   1. Fairness in Engineering Is an Old Problem
   2. Our Fairness Problems Now
      1. Community Norms
      2. Equity and Equality
      3. Security
      4. Privacy
   3. Legal Responses to Fairness in Technology
   4. The Assumptions and Approaches in This Book
   5. What If I’m Skeptical of All This Fairness Talk?
      1. Won’t Fairness Slow Down Innovation?
      2. Are There Any Real-World Consequences for Not Developing Fairness-Aware Practices?
   6. What Is Fairness?
   7. Rules to Code By
3. 2. Understanding Fairness and the Data Science Pipeline
   1. Metrics for Fairness
      1. Measures of Equity
      2. Measures of Privacy
      3. Measures of Security
   2. Connected Concepts
      1. Privacy and Security
      2. Privacy and Equity
      3. Equality and Security
      4. Accuracy and Fairness
   3. Automated Fairness?
   4. Checklist of Points of Entry for Fairness in the Data Science Pipeline
   5. Concluding Remarks
4. 3. Fair Data
   1. Ensuring Data Integrity
      1. True Measurements
      2. Proportionality and Sampling Technique
   2. Choosing Appropriate Data
      1. Equity
      2. Privacy
      3. Security
   3. Case Study: Choosing the Right Question for a Data Set and the Right Data Set for a Question
   4. Quality Assurance for a Data Set: Identifying Potential Discrimination
   5. A Timeline for Fairness Interventions
   6. Comprehensive Data-Acquisition Checklist
   7. Concluding Remarks
5. 4. Fairness Pre-Processing
   1. Simple Pre-Processing Methods
   2. Suppression: The Baseline
   3. Massaging the Data Set: Relabeling
   4. AIF360 Pipeline
      1. Loading the Data
      2. Fairness Metrics
   5. The US Census Data Set
   6. Suppression
   7. Reweighting
      1. How It Works
      2. Code Demonstration
   8. Learning Fair Representations
      1. How It Works
      2. Code Demonstration
   9. Optimized Data Transformations
      1. How It Works
      2. Code Demonstration
   10. Fairness Pre-Processing Checklist
   11. Concluding Remarks
6. 5. Fairness In-Processing
   1. The Basic Idea
   2. The Medical Data Set
   3. Prejudice Remover
      1. How It Works
      2. Code Demonstration
   4. Adversarial Debiasing
      1. How It Works
      2. Code Demonstration
   5. In-Processing Beyond Antidiscrimination
   6. Model Selection
   7. Concluding Remarks
7. 6. Fairness Post-Processing
   1. Post-Processing Versus Black-Box Auditing
   2. The Data Set
   3. Equality of Opportunity
      1. How It Works
      2. Code Demonstration
   4. Calibration-Preserving Equalized Odds
      1. How It Works
      2. Code Demonstration
   5. Concluding Remarks
8. 7. Model Auditing for Fairness and Discrimination
   1. The Parameters of an Audit
   2. Scoping: What Should We Audit?
   3. Black-Box Auditing
      1. Running a Model Through Different Counterfactuals
      2. Model of the Model
      3. Auditing Black-Box Models for Indirect Influence
   4. Concluding Remarks
9. 8. Interpretable Models and Explainability Algorithms
   1. Interpretation Versus Explanation
   2. Interpretable Models
      1. GLRM: How It Works
      2. Code Demonstration
   3. Explainability Methods
      1. SHAP and LIME: The Workhorses for Local Post Hoc Explanations
      2. Data-Driven Explanation
      3. Explainability Metrics
   4. What Interpretation and Explainability Miss
      1. Attacks on Explainable Machine Learning
   5. Interpretation and Explanation Checklist
   6. Concluding Remarks
10. 9. ML Models and Privacy
    1. Membership Attacks
       1. How It Works
       2. Code Demonstration
    2. Other Privacy Problems and Attacks
    3. Important Privacy Techniques
    4. Concluding Remarks
11. 10. ML Models and Security
    1. Evasion Attacks
       1. How It Works
       2. Code Demonstration
       3. Defending Against Adversarial Attacks
       4. Some Evasion Attack Packages
       5. Why Do Evasion Attacks Matter to You?
    2. Poisoning Attacks
       1. How They Work
       2. Defenses Against Poisoning Attacks
       3. Some Poisoning Attack Packages
       4. Why Do Poisoning Attacks Matter to You?
    3. Concluding Remarks
12. 11. Fair Product Design and Deployment
    1. Reasonable Expectations
       1. Expectations of Moving Targets
       2. Clear Communication
    2. Fiduciary Obligations
    3. Respecting Traditional Spheres of Privacy and Private Life
    4. Value Creation
    5. Complex Systems
       1. The Impact of the Product Life Cycle
       2. The Need for Record Keeping
       3. The Need for Experts
    6. Clear Security Promises and Delineated Limitations
       1. Reasonable Expectations of Security
    7. Possibility of Downstream Control and Verification
       1. Verification Systems and Obligations
       2. Product Iteration Timelines
       3. Tracking Downstream Users
    8. Products That Work Better for Privileged People
    9. Dark Patterns
    10. Fair Products Checklist
    11. Concluding Remarks
13. 12. Laws for Machine Learning
    1. Personal Data
       1. GDPR
       2. California Consumer Privacy Act
       3. Data Broker Laws
    2. Algorithmic Decision Making
       1. GDPR
       2. Proposed US Laws for Algorithms
    3. Security
       1. HIPAA
       2. FTC Guidance on Cybersecurity
       3. Tort Law
    4. Logical Processes
       1. Right to an Explanation
       2. Freedom of Information Laws
       3. Due Process
    5. Some Application-Specific Laws
       1. Biometrics
       2. Local Ordinances on Facial Recognition
       3. Chat Bots
    6. Concluding Remarks
14. Index