What’s different about user experience design for the Internet of Things?

FUNCTIONALITY CAN BE DISTRIBUTED ACROSS MULTIPLE DEVICES WITH DIFFERENT CAPABILITIES

IoT devices come in a wide variety of form factors with varying input and output capabilities. Some may have screens, such as heating controllers or washing machines (see Figure 1-3). Some may have other ways of communicating with us, such as flashing LEDs or sounds (see Figure 1-4).

Some may have no input or output capabilities at all and are unable to tell us directly what they are doing. Interactions may be handled by web or smartphone apps. Despite the differences in form factors, users need to feel as if they are using a coherent service rather than a set of disjointed UIs. It’s important to consider not just the usability of individual UIs but interusability: distributed user experience across multiple devices (see Figure 1-5). This is explained further in Chapter 9.

THE FOCUS OF THE USER EXPERIENCE MAY BE IN THE SERVICE

When we talk about IoT, we tend to focus on the devices, particularly those with striking or novel forms. But the behavior of the device might be generated by a program that lives on another device on the network (i.e., a server). We call this the Internet (or “cloud”) service.

This means that the service around a connected device is often just as critical in delivering the user experience, if not more so, than the device itself. For example, the smart travelcards such as the London Oyster and Hong Kong Octopus are often thought of as the focus of the payment service. But the services can be used without a card at all via an NFC-enabled smartphone or bank card (Figure 1-6). The card is just an “avatar” for the service (to borrow a phrase from the UX expert Mike Kuniavsky).³ For more on service business models and product–service ecosystems, see Chapter 4.

WE DON’T EXPECT INTERNET-LIKE GLITCHES FROM THE REAL WORLD

It’s frustrating when a web page is slow to download or a Skype call fails. But we accept that these irritations are just part of using the Internet. By contrast, real-world objects respond to us immediately and reliably.

When we interact with a physical device over the Internet, that interaction is subject to the same latency and reliability issues as any other Internet communication. So there’s the potential for delays in response and for our requests and commands to go missing altogether. This could make the real world start to feel very broken. Imagine if you turned your lights on and they took two minutes to respond, or failed to come on at all.

In theory, there could be other unexpected consequences of things adopting Internet-like behaviors. In the Warren Ellis story “The Lich House”⁴ a woman is unable to shoot an intruder in her home: her gun cannot contact the Internet for the authentication that would allow her to fire it. This might seem far-fetched, but we already have objects that require authentication, such as Zipcars (Figure 1-7).

For more information on networking and its impact on UX, see Chapter 3.

IOT IS LARGELY ASYNCHRONOUS

When we design for desktops, mobiles, and tablets, we tend to assume that they will have constant connectivity. Well-designed mobile apps handle network outages gracefully, but tend to treat them as exceptions to normal functioning. We assume that the flow of interactions will be reasonably smooth, even across devices. If we make a change on one device (such as deleting an email), it will quickly propagate across any other devices we use with the same service.

That will not always happen in IoT systems. Many connected devices run on batteries, and need to conserve electricity. Maintaining network connections uses a lot of power, so they only connect intermittently. This means that parts of the system can be out of sync with one another, creating discontinuities in the user experience. For example, if your heating is set to 19° C, and you use the heating app on your phone to turn it up to 21° C, it will take a couple of minutes for your battery-powered heating controller to go online to check for new instructions. During this time, the phone says 21° C, and the controller says 19° C (Figure 1-8).

These discontinuities won’t always be noticed: sometimes the delays will be very short, and sometimes users won’t be around to notice them—for example, when they are turning on a remote device. So the UX may feel synchronous, even if the service technically isn’t. But when we do notice, the UX may feel quite disjointed.

For more on the technical background see Chapters 2 and 3. For more on the design impact, see Chapter 9.

CODE CAN RUN IN MANY MORE PLACES

The configuration of devices and code that makes a system work is called the system model. In an ideal world, users should not have to care about this. We don’t need to understand the technical architecture of a conventional Internet service, like Amazon, in order to use it successfully. Instead, we form a conceptual model of what Amazon does and how it works that’s good enough to help us understand what to do. We know we can search or browse products, that we need to add them to a basket, set up or log into a user account, pay, and then we’ll get a delivery. We don’t need to understand the different machines involved in making this system function.

But as a consumer of an IoT service right now, you can’t always get away from some of this technical detail.

A typical IoT service is composed of:

• One or more embedded devices (the things in IoT, described in Chapter 2)
• An Internet service
• Perhaps a gateway device (a separate device needed to connect some embedded devices to the Internet, described in Chapter 3)
• One or more mobile or web apps for the user to interact with the service via a mobile, tablet, or PC

Compared to a conventional web service, there are more places where code can run. There are more parts of the system that can, at any point, be offline. Depending on what code is running on which device, some functionality may at any point be unavailable.

For example, imagine you have a connected lighting system in your home. It has controllable bulbs or fittings, perhaps a gateway that these connect to, an Internet service, and a smartphone app to control them all (see Figure 1-9). You have an automated rule set up to turn on some of your lights at dusk if there’s no one home. If your home Internet connection goes down, does that rule still work? If the rule runs on the Internet service or your smartphone, it won’t. If it runs on the gateway, it will. As a user, you want to know whether your security lights are running or not. You need to understand a little about the system model to understand which devices are responsible for which functionality, and how the system may fail.

It would be nice if we could guarantee no devices would ever lose connectivity, but that’s not realistic. And IoT is not yet a mature set of technologies in the way that ecommerce is, so failures are likely to be more frequent. System designers have to ensure that important functions (such as home security alarms) continue to work as well as possible when parts go offline and make it as easy as possible for users to understand what’s happening, and recover from any problems.

For more on matching system models to users’ conceptual models, see Chapter 6.

DEVICES ARE DISTRIBUTED IN THE REAL WORLD

The shift from desktop to mobile computing means that we now use computers in a wide variety of situations. Hence, mobile design requires a far greater emphasis on understanding the user’s needs in a particular context of use. IoT pushes this even further: computing power and networking is embedded in more and more of the objects and environments around us. For example, a connected security system can track not just whether the home is occupied, but who is inside, and potentially video record them. Hence, the social and physical contexts in which connected devices and services can be used are even more complex and varied.

REMOTE CONTROL AND AUTOMATION ARE PROGRAMMING-LIKE ACTIVITIES

In 1982, the HCI researcher Ben Shneiderman defined the concept of direct manipulation. User interfaces based on direct manipulation “depend on visual representation of the objects and actions of interest, physical actions or pointing instead of complex syntax, and rapid incremental reversible operations whose effect on the object of interest is immediately visible. This strategy can lead to user interfaces that are comprehensible, predictable and controllable.”⁵ Ever since, this has been the prevailing trend in consumer UX design (see Figure 1-10). Direct manipulation is successful because interface actions are aligned with the user’s understanding of the task. They receive immediate feedback on the consequences of their actions, which can be undone.

IoT creates the potential for interactions that are displaced in time and space: configuring things to happen in the future, or remotely. For example, you might set up a home automation rule to turn on a video camera and raise the alarm when the house is unoccupied and a motion sensor is disturbed. Or you might unlock your porch door from your work computer to allow a courier to drop off a parcel.

Both of these break the principles of direct manipulation. To control things that happen in the future, you must anticipate your future needs and abstract the desired behavior into a set of logical conditions and actions. As the HCI researcher Alan Blackwell points out, this is basically programming.⁶ It is a much harder cognitive task than a simple, direct interaction. That’s not necessarily a bad thing, but it may not be appropriate for all users or all situations. It impacts usability and accessibility.

Unlocking the door remotely is an easier action to comprehend. But we are distanced from the consequences of our actions, and this poses other challenges. Can we be sure the door was locked again once the parcel had been left? A good system should send a confirmation, but if our smartphone (or the lock) lost connectivity, we might not receive this.

For a deeper discussion of programming-like experiences, see Chapter 15.

COMPLEX SERVICES CAN HAVE MANY USERS, MULTIPLE UIS, MANY DEVICES, MANY RULES AND APPLICATIONS

A simple IoT service might serve only one or two devices (e.g., a couple of connected lights). You could control these with a very simple app. But as you add more devices, there are more ways for them to coordinate with one another. If you add a security system with motion sensors and a camera, you may wish to turn on one of your lights when the alarm goes off. So the light effectively belongs to two functions or services: security and lighting. Then add in a connected heating system that uses information from the security system to know when the house is empty. And assume that there are several people in the house with slightly different access privileges to each system. For example, some can change the heating schedule, some can only adjust the current temperature. Some have admin rights to the security system, some can only set and unset the alarm. What started out as a straightforward system has become a complex web of interrelationships.⁷

For a user, understanding how this system works will become more challenging as more devices and services are added. It will also become more time consuming to manage.

For more information, see Chapter 15.

MANY DIFFERING TECHNICAL STANDARDS MAKE INTEROPERABILITY HARD

The Internet is an amazing feat of open operating standards, but, before embedded devices were connected, there was no need for appliance manufacturers to share common standards. As we begin to connect these devices together, this lack of common technology standards is causing headaches. Just getting devices talking to one another is a big enough challenge, as there are many different network standards. Being able to get them to coordinate in sensible ways is vastly more complicated still.

The consumer experience right now is of a selection of mostly closed, manufacturer-specific ecosystems. Devices within the same manufacturer’s ecosystem, such as Withings (see Figure 1-11), will work together. But this is the only given. In the case of Withings, this means that devices share data with a common Internet service, which the user accesses via a smartphone app. Apple’s Airplay is an example of a proprietary ecosystem in which devices talk directly to one another.

We’re starting to see manufacturers collaborating with other manufacturers too. So your Nest Protect smoke detector can tell your LIFX lightbulbs to flash red when smoke is detected. (This is done by connecting the two manufacturer’s Internet services rather than connecting the devices.)

There are also some emerging platforms that seek to aggregate devices from a number of manufacturers and enable them to interoperate. The connected home platform SmartThings supports a range of network types and devices from manufacturers such as Schlage and Kwikset (door locks); GE and Honeywell (lighting and power sockets); Sonos (home audio); and Philips Hue, Belkin, and Withings (connected home products); see Figure 1-12. But the platform has been specifically configured to work with each of these. You cannot yet buy any device and assume it will work well with SmartThings.

For the near future, the onus will be largely on the consumer to research which devices work with their existing devices before purchasing them. Options may be limited. In addition, aggregating different types of device across different types of network tends to result in a lowest common denominator set of basic features. The service that promises to unify all your connected devices may not support some of their more advanced or unique functions: you might be able to turn all the lights on and off but only dim some of them, for example. It will be a while before consumers can trust that things will work together with minimal hassle.

For more information, see Chapter 15.

IOT IS ALL ABOUT DATA

Networked, embedded devices allow us to capture data from the world that we didn’t have before, and use it to deliver better services to users. For example, drivers looking for parking spaces cause an estimated 30% of traffic congestion in US cities. Smart parking applications such as Streetline’s Parker use sensors in parking spaces to track where spaces are free, for drivers to find via a mobile app (see Figure 1-13). The software company Opower analyzes data from smart meters to suggest ways in which utility customers could save energy and money (see Figure 1-14).

Networked devices with onboard computation are also able to use data, and in some cases act on it autonomously. For example, a smart energy meter can easily detect when electrical activity is being used above baseload. This is a good indicator that someone is in the house and up and about. This data could be used by a heating system to adjust the temperature or schedule timing.

To paraphrase another quote from Mike Kuniavsky, “information is now a design material.”⁸

For more information, see Chapter 13.

UI/VISUAL DESIGN

UI/visual design refers to screen layout, visual styling, and look and feel on a device. This is the form that a device interface takes. The outputs of UI/visual design are typically high-fidelity screen mockups (Figure 1-18). Not all UIs are visual, of course: for a gestural or audio interface, the equivalent function might be defining the aesthetics of the gestures or voice.

See Chapter 8 for more details on designing UIs for embedded devices.

INTERACTION DESIGN

Interaction design is the design of device behaviors. Interaction designers shape the sequences of actions between the user and the device needed to achieve particular goals or activity. They also determine how to organize the user-facing functions of the device. For example, a heating controller might have several modes, such as schedule on/off or frost protection, and some hierarchical functions, such as schedule setting. The organization of these functions defines how easy or otherwise it may be for users to find their way around them.

Interaction design is closely aligned to UI design in the sense that the two are usually done in tandem and often by the same people. But interaction design is primarily concerned with behaviors and actions, whereas UI/visual design is concerned with layout and aesthetics. (Just to confuse matters, some people use UI design as a shorthand term to include both interaction design and visual design.) Typical outputs for interaction design might include user flows, low-medium fidelity interactive prototypes, and for a visual UI, screen wireframes (Figure 1-19).

You sometimes hear the term “information architecture” used to describe organization schemes for functionality, but technically this refers to the equivalent activity for content-driven systems, such as content-based websites.

See Chapter 8 for more details.

INTERUSABILITY

Interusability is a relatively new term. It refers to the additional considerations of designing interactions that span multiple devices. The goal is to make the overall experience feel like a coherent service, even when the devices involved may have quite different form factors and input/output capabilities.

Interusability isn’t a separate set of design activities. It’s an extra set of considerations to be addressed in tandem with interaction and UI design. The key differences to a single device UX design process would typically be:

• Specifying which functionality belongs on each device
• Creating design guidelines that span multiple device types
• Designing cross-device user flows for key interactions
• Designing multiple device UIs in parallel

See Chapter 9 for more details.

INDUSTRIAL DESIGN

Industrial design refers to the aesthetic and functional design of the physical hardware in the service: the choice of form, materials, and capabilities it may have (see Figure 1-20). Connected devices contain electronic circuitry and radio antennae, which impose particular requirements on the industrial design. Devices can also have input and output capabilities, which require collaboration between industrial designers and UI/interaction design/interusability.

See Chapter 7 for more details.

SERVICE DESIGN

A connected device is rarely a one-off purchase. It comes with the expectation of an ongoing service, at the very least the provision of the Internet service that keeps it running, and customer support. All of this forms part of the user’s overall experience with the product.

Service design is an emerging discipline that addresses this holistic view of user experience. It looks at the whole lifespan of a user’s experience with a service, provides a view of all the components of the user experience, and specifies how these function together as a coherent whole (see Figure 1-21).

In addition to device interactions, it might include:

• Customer support interactions
• Instructional guides
• Marketing or sales materials
• In-store experiences
• Email communications and notifications
• The UX of software updates and rolling out new functionality

The service experience is critical to the success of an IoT product, but in this book, we have not addressed service design as a separate activity or chapter. We encourage you to take a holistic approach to designing the user experience that includes interactions not just with devices but also the wider service context. Service design methods may be extremely useful here, but you would not apply them differently on an IoT project than you would on any other project, so we have not discussed them separately.

See Chapters 4 and 6 for guidance on scoping the service experience. The suggested methods in Chapter 14 include guidance on designing the service experience as part of overall UX design.

CONCEPTUAL MODEL

The conceptual model is the understanding and expectations you want the user to have of the system. What components does it have, how does it work, and how can they interact with it? It’s the mental scaffolding that enables users to figure out how to interact with your service. Regardless of whether a conceptual model is designed or not, users will form one. If they get it wrong, they’ll struggle to use the system. IoT services are often inherently complex systems. You can create a clear conceptual model through careful system and interaction design and supporting documentation. You want users to feel in control from the start—they should feel confident that they will be able to use the system, even if they don’t understand all the details yet.

See Chapters 6 and 9 for more details.

PRODUCTIZATION

Productization is the activity of defining a compelling product proposition. It addresses the audience, proposition, objectives, and overall functionality of service (and often its business model). Does your product solve a real problem for a real audience? Is it presented so that they understand that? Does it appeal to them? This isn’t always the domain of the UX designer on a project, but it’s the underpinnings of good UX. All the frontend design in the world won’t make a killer product unless it does something of value for people, in a way that appeals to them and makes sense.

See Chapter 4 for more details.

PLATFORM DESIGN

A platform is a software framework. It takes care of low-level details to help developers build applications more easily. At their most basic, IoT platforms make it easy to put data from connected devices onto the Internet. Slightly more advanced platforms may provide frameworks to enable different types of devices to interoperate.

A software platform will aim to solve many technical issues, many of which may not directly have an impact on the UX. But some more advanced platform functionality is very much to do with UX.

For example, a platform like Hue or Withings may provide standard ways to:

• Discover new devices and applications
• Add devices and applications onto the system
• Manage devices and users
• Manage how devices share data

These are basic building blocks for the UX. If they don’t work well for your users, your UI and interaction design will be full of awkward workarounds.

A more complex platform might also provide ways of organizing and coordinating multiple devices. For example, if a user adds a light to an existing home system, they might reasonably expect the system to know that it should control it along with their other lights and/or offer it as part of the security system. It’s not important to make it talk to the toaster. That may be common sense to a human. But the system won’t know that unless this kind of logic is encoded in the platform.

If you’re building a very simple system of a single device, you might not need to do much platform-level UX to start with. For example, you will need to consider how the user gets the device onto the network. But you don’t necessarily need to design a standardized approach that would work for other types of device as well. But once your system has multiple, interconnected devices, there will be design challenges that will require platform logic to solve. Designers should get involved in shaping platforms to ensure they support good higher-level UX.

There is no commonly understood set of activities for this yet. Chapter 15 discusses what these challenges may look like and some of the approaches we might take to solving them.