Practical Artificial Intelligence with Swift

Preface

Welcome to Practical Artificial Intelligence with Swift.

This book takes a task-based approach to practical implementations of artificial intelligence (AI) using Swift. We do this because we think that you shouldn’t need to dive deep into complicated mathematics and algorithms in order to have clever AI- and machine learning-driven features in your iOS application. AI shouldn’t be a specialist area that’s available only to AI experts. AI should be accessible.

We live in a world where these techniques are becoming more than commonplace. They’re becoming part of the fabric of how we interact with computers. Because of the incredible power of machine learning, and its ability to impact, benefit, influence, and control human beings, it’s as important for people to be able to have knowledge over how to build and understand artificial intelligence as it was to know how to operate a computer.

To that end, this book is designed to give you a practical understanding of common machine learning tasks. With these, you’ll be able to build better tools, and understand the behaviour of the tools used by others in the wider world. We’re glad you’re here. Let’s build the machine.

Resources Used in This Book

We recommend following along with the book by writing code yourself as you progress through each chapter. If you become stuck, or just want to archive a copy of the code, you can find what you need via our website.

Audience and Approach

This book is written for people who already know how to program in Swift, and want to learn about the specific features and techniques that power machine learning. This isn’t a book that will teach you the basics of programming, which means that the examples will expect a reasonable familiarity with the language.

In addition to using Swift, we’ll occasionally dip into Python from time to time. Python is incredibly common in the world of machine learning and artificial intelligence. The meat of the book is in Swift, though, and we explain why whenever we’re using Python. We’re going to remind you about this a few times.

Finally, we also assume that you’re fairly comfortable navigating macOS and iOS as a user, and that you have access to a Mac to develop on, as well as an iOS device to test on, for the tasks that require certain sensors, like motion tracking or analysing the camera.

Organization of This Book

This book is divided into three parts.

In Part I, Fundamentals and Tools, we introduce the fundamental ideas behind machine learning and artificial intelligence. We set you up with the languages and tools you’ll be using, and get you ready to build useful things.

In Part II, Tasks, we visit a wide range of interesting topics, ranging from computer vision, audio, motion, and language. For each topic, we’ll present and build an app that highlights various techniques and APIs; at the end of each chapter, you’ll have a complete working demo app that shows off a practical task that your code can now accomplish.

In Part III, Beyond, we take a look behind the curtain, and examine the techniques that power the apps in Part II in detail. We’ll look closely at the theory of machine learning, and provide a deeper understanding of what it is that your device is actually doing.

Using This Book

We want this book to be deeply founded in the practical. Because of this, we’ve structured it around the practical tasks that you might want to do when dealing with artificial intelligence and Swift. We’ve split the book into three parts.

Part I discusses Swift and AI and the task-based approach to AI that we’re taking for the book (you’re reading a chapter in that part right now), the tools you can use for Swift and AI (that’s Chapter 2), and how and why to approach the datasets you use for your practical artificial intelligences (that’s the third chapter in this part).

Part II, chapter by chapter, explores the different domains for which AI can be integrated into your Swift apps from a thematic point of view: vision-related tasks (Chapter 4), audio tasks (Chapter 5), language and text tasks (Chapter 6), motion and gestural tasks (Chapter 7), augmentation tasks (Chapter 8) for generating and recommending things, and beyond (Chapter 9) to explore the more advanced tools and frameworks we touched on at the end of the first part. We outline what each of the tasks entails in the next section.

Part III explores how the AI methods actually work (Chapter 10), how the practical tasks we made in Part II work under the hood (Chapter 11), and finally how you can implement simple versions of these things yourself, the hard way (Chapter 12).

Our Tasks

In this book, we cover the following tasks in the following domains:

• Vision

  • Face detection

  • Barcode detection

  • Saliency detection

  • Image similarity

  • Image classification

  • Drawing recognition

  • Style classification

• Audio

  • Speech recognition

  • Sound classification

• Text and Language

  • Language identification

  • Named entity recognition

  • Lemmatization, tagging, tokenization

  • Sentiment analysis

  • Custom text classifiers

• Motion and gestures

  • Activity recognition

  • Gesture classification for drawings

  • Activity classification

  • Using augmented reality with AI

• Augmentation

  • Image style transfer

  • Sentence generation

  • Image generation

  • Movie recommendation

  • Regression

• Beyond

  • Installing Swift for TensorFlow

  • Using Python with Swift

  • Training a classifier using Swift for TensorFlow

  • Using the CoreML Community Tools

  • On-device model updates

  • Downloading models on device

The bulk of this book is dedicated to exploring how you can implement AI-driven things in your apps, (mostly) using Swift. Because we take a top-down, task-focused perspective to AI, we decided to front load our discussion of the tools that you might encounter. We do this in Chapter 2.

There, we explore some of the tools that you can use to create models for machine learning and AI tasks with Swift as well as some of the tools for manipulating models, working with data, and generally doing practical AI things with Swift.

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Also used for commands and command-line output.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Caution

This element indicates a warning or caution.

Using Code Examples

Supplemental material (code examples, exercises, errata, etc.) is available for download at our website.

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.

We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Practical Artificial Intelligence with Swift, 1st Edition, by Mars Geldard, Jonathon Manning, Paris Buttfield-Addison, and Tim Nugent. Copyright 2020 Secret Lab, 978-1-492-04481-9.

If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [e-mail].

O’Reilly Online Learning

Note

For over 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.

Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

• O’Reilly Media, Inc.
• 1005 Gravenstein Highway North
• Sebastopol, CA 95472
• 800-998-9938 (in the United States or Canada)
• 707-829-0515 (international or local)
• 707-829-0104 (fax)

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/practical-ai-swift.

To comment or ask technical questions about this book, send email to [e-mail].

For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.

Find us on Facebook: http://facebook.com/oreilly

Follow us on Twitter: http://twitter.com/oreillymedia

Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments

Mars would like to thank her family and coauthors for being supportive even when/though she’s intolerable, and the University of Tasmania for all the opportunities it has afforded her.

She wishes to particularly acknowledge educators Dr. Julian Dermoudy for making her fall in love with programming when she first started only three years ago, and Nicole Herbert for providing—in her role as a highly effective degree coordinator—the support and confidence she needed to do something with this new passion, along with the Australian Apple developer community and the AUC that gifted her their knowledge and interest in Swift, and her parents who continue to be her primary role models of character.

Jon thanks his mother, father, and the rest of his crazily extended family for their tremendous support.

Paris thanks his mother, without whom he wouldn’t be doing anything nearly as interesting, let alone writing books, and his partner (and lead author) Mars, as well as all his friends (several of whom he is lucky enough to have written this book with!).

Tim thanks his parents and family for putting up with his rather lackluster approach to life.

We’d all like to thank Michele Cronin and Rachel Roumeliotis, whose skills and advice were invaluable to completing the book. We’re really excited to work on more projects with both of you in the future! Likewise, all the O’Reilly Media staff we’ve interacted with over the course of writing the book have been the absolute gurus of their fields.

A huge thank you to Tony Gray and the Apple University Consortium (AUC) for the monumental boost they gave us and others listed on this page. We wouldn’t be writing this book if it weren’t for them. And now you’re writing books, too, Tony—sorry about that!

Thanks also to Neal Goldstein, who deserves full credit and/or blame for getting us into the whole book-writing racket.

We’re thankful for the support of the goons at MacLab (who know who they are and continue to stand watch for Admiral Dolphin’s inevitable apotheosis), as well as professor Christopher Lueg, Dr. Leonie Ellis, and the rest of the current and former staff at the University of Tasmania for putting up with us.

Additional thanks to Dave J., Jason I., Nic W., Andrew B., Jess L., and everyone else who inspires us and helps us. And very special thanks to the team of hard-working engineers, writers, artists, and other workers at Apple, without whom this book (and many others like it) would not have reason to exist.

Thanks also to our tech reviewers, with special thanks to Chris Devers, Dominic Monn, and Nik Saers for their thoroughness and professionalism.

Finally, thank you very much for buying our book—we appreciate it! And if you have any feedback, please let us know.