Understanding the rules of critique

Register now for the O’Reilly Design Conference, which will be held January 19-22, 2016, in San Francisco. Adam Connor and Aaron Irizarry will present the session, “Discuss design without losing your mind.” Editor’s note: this is an excerpt from our recent book “Discussing Design,” by Adam Connor and Aaron Irizarry.

There are four key rules of critique that we need to keep in mind. Paying attention to these rules will help to ensure that our discussion remains focused, efficient, and effective. It’s the facilitator’s job to make certain these rules are adhered to, but it’s a good idea to share them with the team.

Especially in formal critiques, we want to confirm that all participants know about and understand these rules. Don’t hesitate to review them quickly at the beginning of the discussion or post them in the room where the critique is being conducted.

Everyone is equal

Organizational hierarchy has an uncanny ability to make people feel like their perspectives and opinions carry more or less weight than others. Although it’s true that an organization might make decisions based on what its leaders think as opposed to other employees, it isn’t inherently true that their opinions are more accurate just because they’re executives.

It is important in a critique that we remember this and that everyone’s observations and perspectives are listened to equally. More attention should not be paid to those of a higher position just because of that position. You may be familiar with the acronym HiPPO (Highest Paid Person’s Opinion); it’s a killer when it comes to effective critique.

Moreover, the contributors themselves should feel like, and see others as, equal contributors. They should recognize that their feedback is just as valued as that of everyone else, regardless of their job title. Useful insights can come from anyone and, if participants are sensing an inequality, they might be hesitant to share their feedback. Or, they might decide to just go along with the feedback expressed by someone else instead of voicing their own perspectives. When this happens, we can miss out on valuable information.

Think about your team or the people you’ll be pulling together for the critique and see if you can, from past experience, sense how much of an issue this will be. Consider talking to the group about this rule and others beforehand. We’ve even had success in some situations talking directly with the team members and executives whose opinions are being catered to about the issue and gotten their help in addressing the issue. Remember, it isn’t the case that everyone in this position wants this kind of behavior. Many of the executives we’ve worked with have recognized the negative effect it can have on a team and their work, and have taken steps to address it.

During a session, if you notice that people are being quiet because of a “superior’s” participation, ask for feedback directly from those people. Or, if someone is throwing their weight around with their opinions, consider talking to them about it, perhaps after the meeting, to avoid it happening again in the future. To address the inequality seen in critiques he’s run, Adam has even called breaks during which he’s raised concerns to the organization leaders in attendance.

If we begin with a level playing field, when it comes time to make decisions on how to iterate and develop our designs, we can ensure that we are able to hear perspectives from across the team. And later, as we review the feedback collected, we can take into consideration an individual’s expertise and experience, and not focus solely on her position in the hierarchy.

Eliminating the influence of organizational hierarchy can be a tough rule to implement, depending on the culture of a team or organization. But for critique to be successful, it is a key element. Remember that critique is not direction. Even though it’s true that in many organizations, a director’s, executive’s, or other leader’s decisions will take priority over the thoughts of others, we want to keep that separate, and hopefully after a critique. Part of what we want to reveal in the critique is a variety of perspectives from different areas of expertise so that any decisions made later are as well informed as they can be.

Everyone is a critic

Everyone in the critique session should participate. Some individuals might be intimidated or won’t feel like they have anything of value to contribute because they aren’t a designer. Or, they might just not feel comfortable saying things that they fear will make the designer feel bad. But as we mentioned in the previous rule, great insights can come from anyone, and we can miss out on helpful information when people do not participate in critiques.

Ensuring that everyone participates also helps prevent situations in which an individual might not say much during the critique but ends up expressing any concerns he had after the fact, when it is often too late to act on those concerns.

Avoid problem solving

This can be the hardest of all the rules. Avoiding problem solving can be tough. We do it instinctively. It is a natural inclination for many of us to come up with solutions as soon as we perceive a problem or inefficiency.

To understand why this is problematic in critique, we need to remember that the human brain does not consciously think both analytically (taking in and comparing information) and creatively (generating possible solutions by combining ideas — otherwise known as problem solving) simultaneously. This process of multitasking instead forces our brain to divide resources between the two processes and then actively switch between them.

In earlier chapters, we described the toggle our brains make between creative and analytical thinking while we’re working. It happens so frequently that we often aren’t even aware it’s happening. It can take a lot of practice and skill to keep our brains from bouncing back and forth, and to remain focused on just one process.

When an individual in a critique begins proposing a new solution or a change, it means that she’s switched mindsets. Her brain has toggled from analytical thinking to creative thinking.

Now, think about this toggle happening independently within every participant in a critique. You can begin to see why it’s problematic. When new ideas start being proposed, you have some folks in the critique who are in a creative mental process, some in an analytical one (still analyzing the original design), some analyzing the new idea, and some in a third process: comprehension (trying to just understand the new idea).

More than likely, some people’s brains are jumping from one process to another. A big challenge in facilitating any discussion, not just critique, is keeping everyone focused on the same objective. In the case of a critique, that objective is analysis of the design being presented. When problem solving begins to happen, the coordination becomes lost and everyone heads in different mental directions.

On top of that, problem solving can derail the session by not allowing session goals to be met due to the focus switching to solving a specific issue seen in the design rather than analyzing the aspects for which we were looking to collect insights. If we have five items to review and critique in the meeting and we veer off to problem solving while looking at item number two, the chances are that we’ll never make it to item number three, never mind five.

But it’s too damned difficult to not solve problems.

The tendency to generate ideas for solutions can be instinctual and unconscious. You’ve probably seen this yourself in one form or another. It’s not uncommon during a usability test, or in any environment really, to hear someone encountering an issue say something like, “they really should…” or “it would be better if they…” before ever articulating the problem she’s trying to solve.

If that’s the case, how can we expect to prevent a room full of people from doing it when we ask them to analyze our designs?

Well, we can’t.

What’s important here is to understand why the problem solving is happening and the impact it can have on the remainder of the discussion.

If someone has jumped to a solution without describing the issue in the design that she’s trying to solve, ask questions that work toward identifying what that issue is and how it relates to the objectives of the design.

If someone is spending a lot of time trying to describe her new solution to you, and you understand the issue or insight she’s trying to raise, work toward getting her to hold onto that idea for a future discussion that will be focused on exploration so that you can continue with the analysis as planned.

Here’s another possibility: if everyone gets really hung up on a specific issue with the design and can’t avoid trying to solve it, or perhaps the impact of how it might be solved is so huge that it brings further critique of the current into question, try pausing or postponing the remainder of the critique and switch right then to exploring solutions.

What is paramount is to keep everyone in the same mental process.

Don’t rush to make decisions on the changes to be made

The output of a critique is not a list of specific changes for the designer to go back and make. Instead, the desired output is a set of new observations and understandings about the design and whether it is meeting the objectives set for it. This goes back to the previous rule. If a list of changes is coming out of a critique session, problem solving is taking the place of analysis.

After a critique should come a period of exploration, using the new insights to frame challenges and generate possible solutions. For some of the insights uncovered during a critique, the changes to be made will be small and obvious, but for many, there is the potential for numerous solutions to be thought of, and we should give ourselves time to do this. If we rush to make all of the decisions on changes to be made while we’re still trying to analyze things in our critique, it’s likely that we’ll miss opportunities for stronger ideas or make premature decisions that cause more problems down the road. This is another rule that might raise some eyebrows when sharing it with your team.

Waiting to make decisions affords the designer time to digest all of the feedback she received during the session. She can examine it in light of the goals, principles, scenarios, and personas of the project. The designer can then follow up with individuals or the entire group about specific points of feedback and continue the discussion, idea generation, and exploration.