Design without Design Research is Pseudo-Design
I coined the phrase “Innovation Theater” some time ago to describe doing design thinking with only the workshopping—standing around with Sharpies® and Post-it Notes® reflecting and not first observing (doing design research) and then also not making (designing and developing). I regularly tell people if that’s what they’re going to do then to not do it because it won’t be effective, will waste time and money, and will just give design thinking a bad name.
Doing design without design research is similarly ineffective, wasting time and money, and will give design a bad name. Pseudo-science and pseudo-medicine are terms that describe practices that are not based on rigorous observation and research evidence. I’d like to suggest that we call design that’s done without design research pseudo-design. Any mature Enterprise Design Thinking focused design practice needs to make design research the foundation of their work. Without design research, the team is flying blind. They’re making things up. They’re guessing. And what they design won’t be successful, at least not reliably so. Do we need innovation and creativity in design? Absolutely. However, that creative effort should be focused on and grounded in a deep understanding of the user that will eventually use that design. To be clear, I’m talking here about the design of products, systems, and services that people will use and design that is intentionally created for people. I consider design outside of these parameters more like art. Art is incredibly important but is not what I’m talking about here
Experienced design researchers need to be specifying what design research needs to be carried out, whether or not the design researchers carry it out themselves or with help from others on the team. Just like you wouldn’t start your day without brushing your teeth, it should be considered basic hygiene to do design research before starting your design and using design research throughout to further hone that design with user input.
I did research some years ago (published in SigCHI and Communications of the ACM) looking at the use of various design practices at more than 100 companies. The findings showed that while executives at those companies assumed that their designers were using the practices that they thought were best, most design practitioners weren’t using the best methods. In fact, the practitioners could identify the most effective methods like contextual inquiry and hands-on user evaluation but they mostly did so-called discount methods like simple surveys and heuristic evaluation. I made the case then and I am again now that we as professional designers and design leaders need to be responsible to our profession and to those who pay our salaries in doing design using the most effective methods which always need to start and end with design research. I’d go as far as to suggest that we adopt an oath much like the Hippocratic Oat of “do no harm” that physicians regularly take which for design might be “do no design without design research”.
As an example, the car I drive just had it’s entire user interface redesigned and I’m convinced that it was pseudo-design because it can’t have been based on user input given the nature of the changes. While there are lots of visually appealing features and some actual improvements, the majority of the release convinces me that either no design research was done or that the design research team had no or little influence on the design. Key frequently used car actions are now several taps away and a customization option doesn’t include any actions critical to driving a car. Just like pseudo-science and pseudo-medicine can be dangerous, so can pseudo-design and especially on a car and other important systems. And, most enterprise design involves the design of important systems.
So, if we’ve established that design research is absolutely required to do good and responsible design then what advice do I have with regard to doing it optimally. Let me share a few thoughts that I have on that.
Hire Design Researchers
In the early years of design, like 10 to 15 years go, it was a practice by many companies that a single designer in a company, or for larger companies a single designer on a project, was all that was needed and those designers would need to do the design research, the user experience design, and even the visual design themselves. I would ask audience members at workshops I ran at the major design conferences during that time with a show of hands how many were the only designer at their company. The majority of attendees would put up their hands. I made the point then how bizarre that was given that the companies that those attendees worked for likely had more than one developer or engineer so design was simply not seen as being as important as the other disciplines. We all needed more designers and design specializations too.
We’ve come a long way in recognizing the need to staff projects with designers much like we do the other disciplines. IBM’s design reboot nine years ago is often credited with leading this trend by specifying that a designer to developer ratio of 1 to 8-12 should be the target and then hiring more than 3,000 designers to staff projects with that ratio. Our rationale for the ratio was that if there were too few designers on a project, the developers would out of necessity do the design themselves with less than optimal outcomes.
The question then arises, how many design researchers do we need? The rationale that I just mentioned applies here too. If we don’t have sufficient design researchers on a project, designers will be flying blind and will need to just make things up. They’ll have to guess. They will also be giving designs to developers without the basic hygiene of user input and evaluation. I suggest that a ratio of design researcher to designer should be in the range of 1 to 5-8 based on industry data.
While visual designers, user experience designers, and industrial designers are able to be recruited from design schools and design programs in universities and even bootcamps, it’s often not as straightforward recruiting design researchers. Very few design schools and design programs in universities teach design research as a discipline. Most are back in the old model of simply ensuring that user experience designers just need a few design research skills. And often, students who aspire to be user experience or visual designers don’t have the attributes to be great design researchers. Where can we then recruit design researchers from? In my experience, people who have education in psychology, sociology, anthropology, and cognitive science tend to have the personal attributes necessary for design research and even have the foundations of many of the design research methods. However, they will still need to learn the full range of actual methods of design research typically on the job. Incidentally, there is a similar problem in recruiting content designers which is also a discipline that isn’t taught in most design schools and design programs. Journalism students tend to have the requisite attributes and many of the skills needed for content design.
Once you have the requisite design researchers, what are some of the other best practices that should be adopted? Let’s explore a few.
Working with customers
It feels like stating the obvious but in order for design researchers to do their jobs, they have to have access to customers so they can work with the users working in those customer organizations. And not just any customers, customers who are representative of the market the product or service is targeting. It’s also a best practice to recruit long-time customers and more recent ones as well as users who may be early adopters and others we are more resistant to change. Some call the former types of users, extreme users. It is also a good practice to do some work with what might be considered future users and customers who don’t currently use your product or service. Satisfying them is an excellent way of ensuring growth in marketshare of your product or service. We recommend that any project should have a carefully selected set of what we call sponsor users with the criteria above and to have only a handful so that we can go deep into their experience. However, we also need to guard against being too myopic in designing for a small group of sponsor users so it is advised to also include a fresh set of sponsor users to provide feedback on the evolving design. It’s important to point out that the best work with customers and users is truly collaborative characterized by co-creation.
Understand various lived experiences of users
Much of the design research that is done focuses on the lived experience of one type of user, typically an able-bodied, white, straight, North American man. But, that’s not typically representative of the actual current or targeted future users of the product or service. Design researchers therefore must ensure that they are including people who are more representative of various lived experiences.
Work as a team
Design researchers themselves should work together with other design researchers, especially in large companies, to plan a coverage strategy for their work. It’s often the case that different products from a company are used by the same types of users so it makes sense to not duplicate design research effort but instead plan which design researchers will explore which unique aspects of the user experience they have in common and what work should be done together. It’s also a good practice to have exploratory design research like ethnographic observation and structured interviews conducted independent of particular products or product release schedules in contrast to user evaluation of product prototypes which need to be done just in time for particular releases.
Design research should also be a team sport involving non-researcher members of the team. Having team members personally experience customer insights first hand is a powerful way of getting buy-in and empathy for the customer experience for all team members. It’s also often a good way to leverage additional help in carrying out the research. One of the most effective pieces of design research that I led with a team a few years ago involved having a researcher and a product manager doing observational work and structured interviews with users and stakeholders. The design researcher focused on the user experience while the product manager leaned in on the business value in the information they were collecting. Another similar experience included a visual designer who also sketched what the user was describing and then immediately sought feedback on the sketch to determine whether the user’s intent was accurately understood.
Be maniacally curious
Design researchers and the designers accompanying them should be maniacally curious. In order to truly understand a user and their environment, you need to adopt an almost childlike curiosity and see every situation from a beginner’s perspective. Make sure to not annoy the user but ask a lot of “whys” and “tell me more” type questions. Make sure to open the aperture of the problem space you’re exploring in addition to the primary areas of focus. And, explore what sorts of things delight the users you’re working with and also for enterprise applications learn about what you could do with your design to make the user more successful in their job, more satisfied while they’re at work, and ideally more efficient so they can finish up their work more quickly at the end of the day.
Be innovative
As design researchers, you have a large toolkit of methods to choose from and you should be innovative in your selection of the right methods and tools for the specific situations but also in modifying them for your particular needs. Keep your focus on what you want to learn and use the tools and methods that will best achieve that learning objective. Just like medicine needs epidemiologic and diagnostic methods to track big picture trends and individual problems so too does design research.
Use Qual & Quant
In my experience, individual design researchers major on either qualitative or quantitative methods. I think all design researchers should have mastery of both types of methods. And, importantly, design researchers shouldn’t shy away from the methods and tools that require statistics. In fact, I think we should be using data science methods more in design research like causal modeling to bring clarity to the relationships of measurements to one another in the entire system and to determine the statistical and substantive significance of our findings.
Look beyond users and use
Design researchers should be working with current users but they should also be working with future users and include important stakeholders in their work. The latter is especially important in enterprise products and services. We’ve had success in the past with including a user and the decision-maker at their company together in a user evaluation session with users giving feedback on their use of the product design and the decision-maker providing an evaluation of their intention to purchase. It’s also important to realize that the entire what we call universal experiences a customer or user has from first impressions in an ad or on a website of a product all the way to upgrading the product should be considered to be within the purview of the design researcher. When thinking about looking beyond users and use, it’s also a good practice to consider one other critically important stakeholder, the earth. In order to design products and services that are more sustainable, design researchers need to factor the earth and its environment as a key stakeholder in their work.
Understand competitors
It’s also critically important for researchers to understand gaps between our product or service and that of the key competitors from actual users. Make sure to do head-to-head hands-on user evaluation sessions to glean insights into the areas that your product or service is better and areas that are opportunities for your product or service to further improve. Consider recruiting users of your competitors’ products and services for your user evaluation studies of your product or service as well. If you can satisfy them with your design, you’ll be in great shape to gain marketshare. If your product or service doesn’t have an obvious competitor, understand how users current do what your product or service is planning to improve. We had a product idea in the past that involved incorporating the capabilities of several existing products from different companies into a single new product that we were considering designing and developing. However, the user research showed that users were perfectly happy and in fact preferred their current use of the combination of products to our idea of a new integrated one. Your product or service always as a competitor, the way people do things today. I’m reminded of the design inspired CEO of Intuit, Scott Cook, who challenged his design team to make the company’s first product, Quicken, better than a person using a pen and the check-writing process.
Make to learn
It’s tempting to always try to do incredibly thorough and comprehensive pieces of exploratory research. However, that can take a long time and lead to analysis paralysis. It’s often better to be more lean and time-limited in your research work and then to quickly get into making to learn which means gathering feedback on very early ideally paper and pencil or foam core prototypes that can be easily modified and thrown away.
Understand the future
It’s not enough to understand what users need today, we also need to give the design team insight into alternative futures by using Strategic Foresight methods. There are many different methods and models but they all essentially involve scanning for strong or weak signals of change and using these to create alternative future narratives to inform potential alternative designs of your product or service.
Share Research
Especially in larger companies with more design researchers, it’s important to share research work and even more importantly research insights. And, design researchers should first seek to understand what other researchers have done before doing their work and then afterwards to share what they’ve done with the other researchers as well.
Design Researchers as Designers
After any piece of design research has been completed, the communication of the work and the key insights from it needs to be designed. Yes, design researchers in this way need to be designers. Design researchers have users: the rest of the design, product management, and development team. Those colleagues are busy and they don’t know what you know nor do they have the skills that you have. So, don’t assume they have a lot of time to consume the output of your design research or the knowledge or skills to understand it the way you do. Spend time identifying the most important, relevant, and insightful findings and then design the communication of them in the most consumable way.
Optimize
Design research is probably the most process and tools dependent design specialization. As such, it has the greatest opportunity to be optimized and streamlined with the aid of DesignOps and ResearchOps specialists and a streamlined toolchain that is tailored for the kind of company and the types of users that it serves.
Implement
The best design research in the world is entirely useless if it doesn’t inform design and be implemented in the target product or system. Product management needs to incorporate design research into plans and ensure that key insights inform the product or service strategy, design, and implementation. All the design research and design effort will be wasted if that work doesn’t get implemented into the product or service.
Executive alignment
Design research work, the incorporation of it into design, and the design’s implementation in the product need to be made transparent to executive management. Like the study I mentioned earlier, executive management assumes that the right work is being done by all the professionals on the team. Their role is also to adjust priorities, balance staffing, and modify schedules. They can’t do their jobs properly governing the organization without visibility to the key customer insights gleaned from design research and the degree to which those insights have informed the design and in turn the implementation in the product or service. We should as design researchers, designers, and design leaders provide them that visibility.
It’s important to note that senior executive managers often haven’t had any design or design management education. Incidentally, the graduates of the EMBA program that I’m an Industry Professor in do have this knowledge but most don’t. As a result, design researchers have to use their design skills to make customer insights able to be understood by senior executive management. This can take the form of scorecards using a format that senior executives would find familiar, like quadrant charts. Keep the focus on showing results that are actionable for senior executives. Some years ago, I was responsible for the design of IBM’s application development products. In that role, I worked with my design researchers to identify the top 10 user problems for the entire suite of products. I simply showed a brief description of the problem and included a brief video showing users experiencing the problem. I also invited the senior executives to try that aspect of the product suite themselves. That led to executives on the team being assigned the job of getting the problem fixed by a particular date and every subsequent status meeting included reports on the status of fixing those problems. I gave the senior executives the information that they could do their jobs with in driving solutions to the problems. Partnering with senior executives especially on topics having to do with design which they often know little about is crucially important.
_______
Now, you may be thinking that I’m being too extreme. That I’m making too big a deal about the importance of design research. I actually think it’s the role of design researchers to not only do the work to improve design but to also use their tools to provide evidence of that improvement. I did a piece work some years ago with a Masters student at a business school in the UK. I collected data on all aspects of the design process. It was IBM’s User-Centered Design process at the time. The data included the number of customers recruited, the number of hours of user input and evaluation sessions, the number of designers on the team, the number of iterations of the design, and other variables like this all regressed in a hierarchical regression model using product revenue and customer satisfaction as the dependent variables. The model was able to calculate what percentage of the variance in the dependent measures of revenue and customer satisfaction the various predictor variables accounted for. The results showed that the amount and effectiveness of design research had the greatest impact on revenue and customer satisfaction. I believe we need design research organizations to do this type of higher level design research on design research itself to assess its efficacy and even the efficacy of various design research methods. In fact, there is no other discipline better equipped to do this work.
In sum, I truly believe that you can have the absolutely best visual, user experience, content, and industrial designers and the very best developers and engineers but you still won’t be successful, or at not least consistently, if you don’t have great design researchers and a practice of incorporating their customer insights and user feedback into the product or service. In fact, the number one reason for 90 percent of startups failing is a lack of appropriate design research. Essentially, they’re doing pseudo-design. And just like pseudo-science and pseudo-medicine, they’ll occasionally be successful purely by chance. Most companies can’t afford to have those kinds of odds so they should do design research and in turn do real design.
