How Experiment-Driven Design perfectly balances progress and process
We’d send in a small team for a lightning-fast, low-cost blitzkrieg.That’s the strategy we settled on when a major fashion retailer approached us about building a conversational user interface (UI) experience. With a short timeline and a small budget, we knew we had to execute quickly.
By the end of the first week, we’d put together a splashy demo to conquer the client’s task. Or so we thought.
What went wrong? We forgot about process in our quest for progress. Instead of spending time upfront on user research and ideation, we ran with the client’s existing code and wound up introducing a colony of bugs. Those bugs became new foes — time-consuming and expensive-to-fix foes, at that.
Had we let experiments drive our design like we usually do, we’d be telling a different story. But as it was, we had a mess to clean up.
Experimentation as a Process
In the worlds of client service and design, it’s easy to give either progress or process the spotlight. Nobody is going to argue with progress, yet pursuing it recklessly is bound to backfire. Process is important for repeatability and team learning, but slow and steady doesn’t win the business race.
For a balanced approach, we turn to experiments. Their fractal nature means that they work across an enormous range of design projects, from identifying a new line of business to choosing a new icon. While experimentation can be used to find the “ideal” icon, it’s especially important for the former, which is the type of project we typically work on.
Often, our clients come to us with a problem or opportunity statement. Given that we work on new products and business initiatives, we want to discover “unknown unknowns” and adapt before they turn into problems down the road.
Through experimentation, for example, we might learn that we need to slow down and learn more about a user’s particular pain point. Or we might realize that we’ve got the user’s needs down pat and it’s time to start iterating. Although this might sound like an indecisive way to design, it’s actually a time- and money-saving process.
Experimentation helps us design into the unknown by creating a series of guardrails. Each time we run an experiment, we discover a boundary — a reason why we shouldn’t continue spending time and money on that idea. By repeatedly bumping into those rails, we avoid going all in on a design until we’ve discovered the one that best meets users’ and stakeholders’ needs.
Design is all about finding the most appropriate answer to a set of needs. Whether it’s better to build a small figurine beautifully or a huge statue roughly, for instance, is a meaningless question without knowing what the sculpture commissioner wants, what his budget is, where he’s going to display it, and more.
We begin to understand those needs when our clients bring us a problem statement. But to move forward, we have to check our assumptions. For example, if a client thinks the target user cares more about a financial app’s security than its transfer speed, we can ask whether that’s a hypothesis or the client has validated the idea with an experiment.
Not only is this a low-pressure way to get up to speed on the client’s research, but it also gives us a reason to build. Experiments are action-oriented, and the best way to move forward is often to start building.
By viewing everything we build through the lens of experimentation, nothing becomes sacred. Need to throw something away? No problem. Did that latest iteration send us down the wrong path? We’ll change course. Instead of assuming we have all the answers at the beginning — and, as a result, constructing an ending that fits our assumptions — we prime ourselves through experimentation to wind up somewhere different than we thought we would on day one.
Perhaps counterintuitively, experimentation creates progress because it puts learning first. Tinkering isn’t a waste of time; it’s how designers unlock doors. Who do you think is more likely to come up with a solution: a designer who spends her time building out different options or a designer who waits to create something until she’s thought of the “perfect” answer? Even if the designers were to come up with the same solution at the same time, the experimenter would have a leg up because she can pull components from her prototypes.
The Truth About Experimentation
True, the designer who prototypes and experiments isn’t going to come up with a solution on her first try. Experimentation might even take her in the wrong direction at first.
But consider the alternatives. Her peer who blindly builds out a stakeholder’s suggestion might finish first, but her design isn’t likely to survive the market’s trials. A third designer who uses the waterfall method might present a more refined design, but the deluge of process means she’ll need months or millions of dollars more in resources to do it.
Experimentation is just enough process that it leads us to a soft landing on a wide range of projects, and it’s action-centric enough to produce progress under any circumstances. That’s why it’s the centerpiece of our innovation process.