Practical Fairness

Preface

Welcome to Practical Fairness. I wrote this book because data scientists and machine learning engineers are increasingly aware of the fairness implications of their work but are not adequately empowered to do anything about their concerns. Academic research on mathematical solutions to fairness concerns is flourishing, and myriad open source options are available thanks to both academic researchers and technology companies sharing resources. However, delving into the topic in a practical and concrete way remains difficult for the beginner, and best practices have not yet emerged in most industries to address even the most basic concerns. This book’s aim is to be an accessible overview for beginners in this field with actionable fairness advice.

Goals of This Book

This book will help practicing data scientists and technologists get their feet wet in the world of fairness. The goal is that after reading this book, you can actively pursue fairness in your own work. Fairness doesn’t have a one-size-fits-all solution, but you should be able to:

• Identify potential fairness problems in your organization’s workflow and data practices
• Break fairness concerns into conceptually distinct categories and target appropriate areas of research literature accordingly
• Offer accessible guidance to nontechnical colleagues on potential fairness problems and solutions in whatever machine learning products you are developing

In this book I exclusively use Python examples and focus on the easiest interfaces available via open source options for implementing relevant fairness methods. I chose this approach because Python benefits from a large degree of open source work in the fairness domain. However, a good deal of work in other languages unfortunately had to be ignored, including in Java, R, and MATLAB code. Also, code bases with organizational sponsors and with a larger breadth of tools were favored over smaller code bases or code bases maintained by just a few individuals. This also meant that some very interesting and high-quality work was omitted. You should be aware that the goal of this book is far more to help you get conceptually organized and that the packages highlighted here are just one set of tools, not necessarily the best or only tools. The fairness toolbox continues to grow rapidly, and there is every reason to expect more tools and code bases to develop over time. The selections in this book represent just one snapshot of convenient APIs.

Fairness in machine learning and in the technology sector remains an active struggle, an ongoing social concern, and an interesting engineering problem. We need legal, economic, and social solutions as well as technical ones. Toward that end, 50% of the royalties earned on this book will be donated to the American Civil Liberties Union, an organization that has relentlessly pursued fairness for a hundred years. The ACLU is actively working to secure fundamental rights to privacy and fairness in the era of algorithms. Readers should be aware that the ACLU played no part in the writing of this book, has not reviewed it, and has not endorsed it. Nevertheless my hope is that this book will reinforce and support the ACLU’s important work in more ways than one.

The other 50% of royalties earned on this book will be donated to Mercy for Animals, an organization dedicated to radically expanding our definitions of fairness to include the well-being of animals and Earth. Mercy for Animals takes a practical approach to addressing devastating cruelty and unfairness practiced in the fundamental industries that bring food to our table. Thus, my hope is that this book’s royalties can contribute to our society’s ongoing development of a fairer and more inclusive outlook.

Practical Notes on the Book

This book is partly about concepts and partly about coding. With respect to the coding, I found that some of the APIs I used were quite verbose. To cut down on the volume of code presented, I did not always include full running code examples in the text, but the full examples are available in the associated GitHub repository.

The topic of fairness is necessarily tinged with a political or ideological flavor, but I have tried to avoid politics and ideology. Also, I have written this book as a practical manual, not an academic treatise on the topic of fairness in machine learning. For this reason, many notions are taken for granted or defined at the level relevant for action rather than for philosophical discussion. So if I said anything controversial here, it was unintentional. For those interested in more academic or combative debates, you can find many places for that.

I want focus on the pragmatic task of increasing fairness in cases of obvious injustice, a simple and hopefully uncontroversial task. This book starts with a few premises—namely, that we would all like a world where everyone has equal opportunities and receives fair treatment. We would all like a world where machines and digital interfaces do not undermine our autonomy or our privacy. We would all like a world where our increasing use of digital products and algorithmic decision-making tools results in better, more robust, and more secure outcomes rather than the opposite. This book provides key concepts to get you on your way to building the fairer digital products that will bring us closer to this desirable world.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Using Code Examples

If you have a technical question or a problem using the code examples, please email [e-mail].

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Practical Fairness by Aileen Nielsen (O’Reilly). Copyright 2021 Aileen Nielsen, 978-1-492-07573-8.”

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

O’Reilly Online Learning

Note

For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.

Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book where we list errata, examples, and additional information. You can access it at https://oreil.ly/practical_fairness.

Email [e-mail] to comment or ask technical questions about this book.

For news and information about our books and courses, visit http://oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

A wide variety of individuals helped make this book project happen. I wish to thank my editors at O’Reilly: Jon Hassell, who green-lighted this book despite it being a novel and difficult topic, and Corbin Collins, who read everything carefully, gave great feedback, and worked tirelessly with me through the complications of writing during the coronavirus pandemic and resulting work-from-home madness. I also wish to thank my production editor, Katherine Tozer, for her relentless and reliable attention to detail and commitment to coordinating a careful review of the book. I wish to thank the copyeditor, proofreader, and sensitivity reader of the book, Josh Olejarz, Sharon Wilkey, and Abby Wheeler, as well as the illustrator, Kate Dullea, who converted all the snippings I had from various sources into beautiful and precise graphics. Thanks to my technical reviewers, Sray Agarwal, Shashin Mishra, and Niall Murphy, for reading the manuscript carefully and providing both detail-oriented and high-level feedback.

Many members of the technical research fairness community also offered help and feedback, as well as generous permission to cite their work and reproduce or repurpose some of their graphics, which are attributed where used. The technical fairness community has been extremely generous in providing guidance to me as I developed this content. I do not include specific names here to avoid implying that they reviewed or endorsed this work.

My work also benefited greatly from time I have spent as a member of ETH Zurich’s Center for Law and Economics. I connected with researchers from a wide variety of disciplines who think about the ramifications of algorithmic decision making and the options for alternative design and deployment for all things algorithmic. I also benefited tremendously from the students in my Law and Tech course, who provided a variety of helpful questions and robust feedback on lectures on AI Fairness from a legal and technical perspective.

Finally, I wish to thank my family, as always, for their support in giving me the time I needed to write this book and periodic boosts to morale to make sure I got the job done, particularly my son, Eddie and my husband, Ivan.

All mistakes are mine. If you spot any, please report them in O’Reilly’s errata portal or to me personally at [e-mail]. I also welcome feedback regarding how to make the book more useful to you in your real-world fairness tasks. Toward that end, I hope to hear from you. This first stab at a practical guide is just a start, and feedback from the community will enable me (and other authors) to develop better resources as the field matures.