user feedbck

The State of Design Practice

I wrote a book some ten years ago with a couple of colleagues (Scott Isensee and Carol Righi) called "User-Centered Design: An Integrated Approach" which outlined a comprehensive system for introducing UCD to an organization and specified the key methods that should be carried out. This work was an elaboration and extension of the pioneering work of Don Norman and colleagues. I followed the book up some years later with an article in the Communications of the ACM entitled "The State of User-Centered Design Practice" which reported the results of some research that some colleagues (Ji-Ye Mao, Paul Smith, and Tom Carey) and I had conducted getting information from UXD practitioners at more than 100 companies and investigating, among other things, the differences between what is known to be best practice and what actually is carried out inside companies. Even though the importance of design was increasing, practitioners in most companies reported a significant difference between the known best practice and their day-to-day experience.

Design is seen as even more important now, as pointed out elsewhere in the blog thanks largely to Steve Jobs and Apple. I thought it would be good to have another look at the state of actual practice so I turned to the LinkedIn "User Experience" discussion group and asked the question shown here.

Although I posted the question some time ago, the responses picked up significantly recently and have been fairly extensive and from a variety of different companies. The discussion is ongoing but I thought it might be good to summarize the key themes at this point here. There were five key themes among the comments which each were mentioned by a substantial number of practitioners.

Management Understanding/Buy-in: An overriding theme mentioned by many concerned the lack of understanding regarding the need for, execution of, and requisite resources required for User Experience Design. This resulted in insufficient importance given to design and inadequate resources being applied to it. 

Missing or Vague Requirements: Many practitioners mentioned the challenges they faced in attempting to carry out User Experience Design activities when the projects hadn't been given appropriate business requirements from Project Management. The requirements were either missing entirely or sufficiently vague as to have little value.   

Resources to do User Research: Practitioners overwhelmingly reported dissatisfaction with not having the resources to carry out one of the most powerful design methods: user research. This finding is consistent with the results I reported in the Comunications of the ACM article mentioned above. The combination of missing or vague requirements and little nor no user research leads to designers essentially working in the dark.

Everyone Thinks They're a Designer: Another common theme reported by many respondents dealt with the problem of project managers, executives, and developers all thinking that they're capable of doing design themselves. One respondent also astutely observed that designers are often guilty of assuming the developers know nothing about design which is also problem.

Suboptimal Day-to-Day Practices: The above themes were reported about equally by respondents as being substantial major problems. A fifth theme emerged that dealt with a variety of day-to-day frustrations or suboptimal practices. These included the following. 

  • Being handed a design by development and being asked to "make it usable" or by a project manager and asked to "clean it up". 
  • Presenting a wireframe when the expectation was the finished, polished design. 
  • Not being able to iterate on the design based on user feedback and simply being expected to come up with the completed design in one iteration. 
  • Being told that it is too costly or will take too long to build the design or that it will be addressed in the next release. 
  • Being expected to adopt designs that are perceived to be industry leading. 
  • With no audience definition being told to "design for everyone". 
  • Not addressing the findings from user research when it is carried out.
  • Having to deal with everyone on the team thinking that they're the user. 
  • The difference between UI and UX not being understood. 
  • Losing sight of the primary objectives of the project and getting distracted by other tangible results.
  • Unrealistic deadlines and evolving scope creep. 

The experiences of the User Experience Design practitioners that responded to my question clearly articulated the challenges that remain to be addressed in many companies. As I pointed out in my Communications of the ACM article, companies may have the mistaken notion that design is being carried out effectively simply by having hired designers. However, these results reinforce the need for companies to more deeply understand User Experience Design and to address the issues summarized here.

Although some methods and technologies have evolved, the essence of the integrated approach that was outlined in our book ten years ago still applies and my experiences with leadership teams validates it as well. The integrated approach that my coauthors and I published stresses that the commitment to design excellence and the investment in it has to start at the top of the company, clear business objectives must be set by Product Management, user research needs to be carried out to provide the foundation for design informing personas and scenarios, design needs to be carried out by those trained in the disciplines of design and start with low fidelity prototypes evolving through iterations involving regular user feedback and innovation explorations into high fidelity prototypes and code. Product management, design, and development need to work collaboratively and iteratively leveraging each other's skills. The entire team has to share the commitment to achieving the established business objectives with the effective leveraging of User Experience Design. While this isn't easy, it does yield significant dividends when deployed and executed optimally.  

I'd like to thank the members of the LinkedIn User Experience discussion group for their experiences and insights. 

Designing in Collaboration with 100,000 Users

MySpace is widely recognized as a user interface design mess. Facebook started out life as the clean and simple design alternative. However, with the addition of its development platform that created numerous 3rd party apps, the Facebook profile user interface has become more cluttered, unwieldy, and disorganized. Interestingly, recent stats from Nielsen indicate that Facebook traffic to the site is declining too.

The User Experience and Design team at Facebook has launched a major redesign. They're considering organizing the profile content using tabs, adding a new publisher feature for more simply adding content, and making navigation to applications easier via a drop-down menu. They don't plan on changing the site's visual signature, color palette, or branding. The design changes aren't radical but the approach to getting user input is. The team created a Facebook group within which they've been previewing proposed designs and then, using the commenting feature of Facebook, getting user feedback on them. There are 104,629 users in that group as of this writing and lots of them are providing valuable feedback. I think its brilliant that the team is using their own site to design the site.

The Facebook organization appears to be maturing given the way they're approaching this redesign. Previous changes to the site apparently didn't involve iterative design with user input and the results were disastrous.

I look forward to seeing and experiencing their redesign as an observer of their design as well as a daily user of Facebook itself.

For an in-depth report on this project, checkout BusinessWeek's recent story on Facebook's Big Facelift.

Web 2.0's Built-in Customer Input & Collaboration

During Web 1.0 and before that, companies had to go out of their way to collect feedback from their customers and when they did, it was in the form of customers communicating directly with the company. Web 2.0 changes all of that. Now customer feedback is an integral component of most social computing sites. And, the feedback is public. This provides great benefits to companies by getting more direct and regular feedback but it also changes the nature of the feedback process in that customers can also use the feedback from other customers directly as well. An article by Vicky Burger on Web 2.0 transformation addresses this topic.

However, things are not quite the way you may think they are in the world of social computing. Very few people are aware of a recent study which shows that less than one percent of visitors to social computing sites contribute in any way including providing feedback whether comments or ratings. So, the challenge here is that companies are now able to get additional feedback directly from customers but they may run in to two problems. The first is that unless the site gets a huge number of hits, the amount of feedback the site may collect will likely be very small (remember less than 1% contribute). The second, and perhaps even more difficult to deal with, problem concerns the likelihood of the sample being unrepresentative in some way. I haven't seen any research on characteristics of users who tend to contribute to sites versus those who don't and without that type of information it is difficult to determine whether the feedback a company gets in this way is representative or biased in some way.

Companies that are serious about making strategic decisions on the bases of direct customer feedback coming from Web 2.0 sites would be wise to adopt the best practice from market research of directly testing the non-response bias. The way that's done is to solicit input on the same questions from people who didn't contribute the information unsolicited and then comparing the results to test for a systematic bias. Without a specific focus on this aspect of Web 2.0 user feedback, companies could find themselves honing the designs of their sites based on the feedback of the small group of people who contribute feedback and these changes may not be optimal for the entire population of users who the company is trying to satisfy.

Active Involvement in Web 2.0 Still Low

New results show that a small percentage of users of so-called social computing or Web 2.0 sites get actively involved in directly contributing to the site. An article on vnunet.com summarizes the results. A key point that is made is that most of these sites, like YouTube, Flickr, and Wikipedia have huge numbers of people actually viewing them so if a relatively small percentage of them contribute, that still accounts for some significant contribution numbers. However, it is important for those building Web 2.0 capabilities into the projects and products to keep these percentages in mind when tracking both passive and active interactions.