Paradigmatic Change in UIs

In the first wave of computing there was virtually no user interface, comprising little more than instructions written on punch cards which were loaded into a hopper then read by the computer with the results given to the user via a printout. The second wave introduced what we now know to be a user interface, a display with characters and graphics along with a keyboard and mouse. The display evolved from simple monochromatic characters to full color graphics with ever increasing resolution over time and the keyboard and mouse technologies evolved to be smaller and integrated with trackpoints and trackpads. However, the basic elements of a small TV like display, with a keyboard and mouse beneath it, have remained constant for a remarkably long time. What appeared on that display and how a user interacts with it has remained surprisingly constant as well, especially from the time that the concept of programs running in separate windows was introduced.

While there were minor predecessors, the major shift into an entirely new form factor came with the introduction of the iPhone. We're now so used to smartphone UIs that many people forget that we hadn't ever experienced one until Apple introduced its game-changing device. The smartphone form factor existed before the iPhone, but Apple totally redefined it. Once users were used to the iPhone user interface, the adoption of the larger form factor iPad was incredibly easy because it was virtually identical. Like the smartphone, the tablet form factor also existed prior to the iPad but again Apple redefined it dramatically. Key to that redefinition was the perfected use of multi-touch. Interacting via multi-touch is so pervasive now that it isn't uncommon to see people walking up to screens in places like hotels, airports, and stores expecting to be able to interact with them with touch only to be really surprised and disappointed when they turn out not to support touch. That's when you know that we've experienced a paradigm shift as a society. Touch has been in university labs for decades but it took Apple's dedication to design excellence of the entire user experience to perfect the technology to create this paradigm shift.

Another paradigm shift in interaction modality has just started. This one involves the use of speech. Again, speech technology has been around for decades and has been used commercially successfully as well but mostly in niche markets like voice response systems and dictation systems. Apple's Siri is still in beta, a product designation Apple very rarely uses, but promises to do for speech technology what the iPhone did for touch technology - make it a pervasive and paradigmatic change in society.

There are two major insights to glean from these fairly recent advances. The first concerns how these changes took place. In each case, the basic research and foundational technologies as well as even some commercial applications existed for decades prior to the paradigm shift. It was Apple's approach to design that made the difference. The design of everything, from the industrial design of the physical elements of the device (glass, case, bezel, etc.), the visual and interaction design of the operating system and key apps, the engineering design of the internals (processors, memory, battery, GPS, etc.), the manufacturing design of the production line, the design of the website and app store, the design of the content review process, the design of the payment and app download system, the design of the stores, the design of product support, all the way to the design of the product secrecy and product announce/launch systems. Many people like to jump to simple conclusions that these paradigm changes were brought about by this or that individual element but I believe that it was Apple's focus on the total customer experience and all the elements that impact it that was critical. Designers planning a product that they hope will transform an industry need to focus on all of these aspects of design.

The second major insight to glean from these paradigm shifts is the need to rethink how all future products in any market should fit into these major paradigmatic changes in form factor, device, and interaction modality. Designers now need to understand deeply how users are using these technologies in order to design products optimally leveraging them. This is a challenge for many because, as pointed out above, form factor, device, and interaction modality hadn't changed virtually at all for decades. However, these recent changes are so profound that it really does require designers in any market to sit up and take note and consider how users in their markets may be changing.

Having explored paradigm changes we've already witnessed, let's give some thought to where these may go in the future and what other paradigms we may witness in the future. While the computer, smartphone, and tablet form factors and the touch and speech interaction modalities have mostly developed independently, the emerging trend is for them to become more consistent and a future paradigmatic change may involve them integrating deeply. We're seeing the beginnings of changes being made largely to computer operating systems like Apple's OSX to make them more similar to device operating systems like iOS. Apple is making the change gradually with each update to the OS, which is a wise approach that minimizes the magnitude of the change but still moves drives consistency. We're also seeing the very beginnings of a move to integrate form factors, devices, and interaction modalities. Responsive design is part of this trend, as is the enablement of touch and speech pervasively across devices. We're also seeing some degree of cloud based seamless access integrating content and data across devices. We're also seeing that integration spread to even larger form factors like TVs and digitally enabled physical window panes. Some call this a post-PC era, my sense is that we're witnessing a plethora of form factors which in PCs, ensuring that each of these can suit the wide characteristics and contexts of use into the future.  

We're living in exciting times that require designers to be fully aware of, intimately knowledgeable about, and be able to leverage the benefits of these incredible paradigm shifts in technology and people's use of them.

Apple's Post-PC Era

I was just setting up a new iPad for a family member who wanted to use the new device she received as a gift as a stand alone device without needing to connect to a PC. Regular readers of this blog will know that I'm usually pretty positive about Apple's designs and, of course, I'm certainly not alone in that view. Apple is incredibly good at hardware design and the integration of software with its hardware. However, the company to date has had some difficulty with the design of its cloud capabilities.

Apple has recently used the term "post-PC era" as a basis for its mobile and cloud strategies and communication. Recent updates to iTunes and iOS have provided some independence of mobile devices from computers, but only partially. You can now push updates from a PC-based iTunes to an iPod Touch, iPhone, or iPad without using a cable but you still have to use a computer in the mix.

There are other use cases that have absolutely no support on iOS devices. The most annoying and concerning one for me is the inability to subscribe to podcasts on Apple mobile devices. You can download individual episodes but there is no way to actually subscribe within mobile iTunes. With the increasing popularity and use of mobile devices (see last blog post), the inability of podcast listeners to subscribe via mobile devices is not only inconvenient for users, it is also a major problem for those creating podcasts, like me. It is really strange that Apple hasn't provided the capability to subscribe. All it would require is to include the same "subscribe" button to the right of the podcast name and artwork on mobile iTunes as is included on the computer-based version of iTunes (see visual on the right). It was pointed out to me by a friend on Facebook that there are separate apps that provide this capability with one called Downcast that is particularly good and one that he recommends. That's a temporary fix but I still think that this missing function needs to be included in the base mobile version of iTunes. I know that it is a herculean task to develop mobile device operating systems that never need to be connected to a PC and to design that really well. However, leaving off the subscribe button seems to be a rather strange oversight on Apple's part. 

To the listeners of my podcast who may be reading this, I'd suggest that you use the iTunes on your PC to subscribe to the podcast or download an app like Downcast to essentially replace the podcast part of mobile iTunes on your iOS device. I do hope that Apple addresses this problem soon in an update to iOS so that we won't see bifurcation of the podast audience on Apple devices and also no longer have a single reliable place to determine podcast popularity and feedback.

Mobile Design Innovation

I heard the buzz about a new iPad app called Flipboard a few days ago so downloaded it to see what all the talk was about. Due to its popularity, the company could only make some of the capability of the app available upon download and the remainder, Twitter and Facebook integration, after a day or two. I knew immediately when the app launched that I was experiencing something entirely different. The splash screen shows full screen photos with a slow animation cross the screen of highlights from the various content sources you've selected. This approach to splash screens is novel and immediately grabs the user's attention with directly relevant content using often stunningly good photographs. Please note we're not even really using the app yet but have been drawn into it in an amazingly engaging way.

After you swipe to the left across the screen, you see the main Flipboard content channels. There is a starter set of content sources but you can select whichever ones you'd like from a fairly long list of candidates, including Wired, The New York Times, Fast Company, The Economist, Nature, Engadget, and The Onion.

Two special sources are Facebook and Twitter. These are the ones that you need to request access to during the early days of the launch of the app. Once you get your copy of Flipboard activated you now get content directly from your Facebook and Twitter streams into Flipboard. 

So, what's so new you say, other than the cool photo splashscreen other than the fact that this app aggregates content from a number of different sources into one place?  There are other apps that do that already and I've used most if not all of them.

What's truly unique about Flipboard is its design. It seamlessly integrates and visually renders content into in a way that Steve Jobs would call magical. Take a look at the screen shot above. Looks like a magazine layout, doesn't it? A layout that a designer would have manually crafted with photos included for appropriate emphasis of certain aspects of stories, etc. What you're actually looking at in that screen shot is my Twitter stream. Twitter is usually quite stark with textual tweets, links, and the occasional image. Flipboard has designed and coded algorithms to handle the layout and rendering of content dynamically and automatically. And the results are amazing. The user also has the option to tap on a story (actually a Tweet but it feels like a story) to see more detail as well as to be able to tap on any images to see them in full screen mode. You've never seen the images in your Twitter stream, Facebook updates, or from your online news sources look better! The subtle animation effects are also very effective. 

It is often said that the spreadsheet program Lotus 1-2-3 was the "killer app" that drove the early success of the PC industry. I believe that Flipboard will do the same for the iPad and like devices. It shows was it possible with the device and its user interface patterns will likely (and should) be used widely in mobile apps like this. The app isn't without controversy, however, due to the way it pulls content from sources but I hope that this won't hamper the success of this amazing app. I should give you one caution, though. The app is so engaging to use that you'll find yourself spending much more time with it than you planned. It's like the experience you have with games except that in this case you're actually learning a ton during that extra time.       

Going Digital for Free

Many aspects of our lives have been going digital over the past few years. A typical knee-jerk response to a question is to simply Google it.  I consider Google an appliance for my brain given how often I use it in this way. The majority of people now listen to music in the form of MP3s using an iPod, iPhone, or some other personal audio device. Many also watch digital movies on their computer screens or increasingly on their TV screens too. News is increasingly consumed in digital form via websites, blogs, Twitter, Facebook, or podcasts. Books are now often read on dedicated devices such as a Kindle or directly on devices like iPhones and iPads and alternatively listened to as audio books in the way that music is listened to. 

I'm an early adopter of most things digital and have consumed content almost exclusively in digital form for the past few years. I've been paying for much of this digital content though whether through iTunes for music and movies, audible.com for audio books, and amazon.com for eBooks. I was reflecting the other day on the fact that I used to borrow books, music, and movies from the public library, the physical versions of these things. I thought it was an amazing tax-supported way of simply borrowing media for the time period you needed it. I remember checking periodically with the library to see if they were moving into the digital world and was routinely disappointed to learn that they weren't. In fact, I found it extremely frustrating that I couldn't even access the online catalog because all the computers were being used by people accessing their email!

You can imagine my delight when I recently checked with my public library website and found that they now provide digital media and the ability to download and transfer it directly to your media device (an iPhone in my case). The media stays on your device for the loan period and automatically disappears after that leaving you extra space on your device. They offer music, movies, ebooks, and audio books. Since finding this new free resource about a couple of weeks ago, I've already listened to four audio books. I'd recommend that you check out your local public library website to see if yours has digital content for download as well.  

Apple's New iOS4

I just upgraded my iPhone to the new version of the operating system. Apple renamed what used to be the iPhone OS to the more generic iOS because it is the system that powers the iPhone, iPad, and iPod Touch. The latest version, iOS4, comes with the new iPhone 4 but it is also being made available for free for previous iPhone models. I have an iPhone GS so the upgrade was a really nice surprise, especially for free!

The major enhancements that come with iOS4 include multitasking, folders, integrated and threaded mail, and camera zooming. Let's explore each of these in turn.

Multitasking is a welcomed addition to the iPhone. The method of switching between apps once you know that you have to double-press the button is pretty intuitive and convenient. What isn't so intuitive is what this control actually does. The tray of running apps is obvious enough but what is confusing is the fact that that tray just keeps filling up with apps even if you closed them down by single pressing the button. The only way to remove apps from the actively running tray appears to be to explicitly delete them, which is weird. Apparently, only a few apps are able to actually keep running whereas others simply suspend their state while not in focus. Any difference between these two types of apps in terms of their multitasking ability isn't apparent in the user interface itself.

Folders are not only a nice to have but a necessity if you have a reasonably large collection of apps. The user interface action in this case isn't intuitive but once you've done it once, it does feel natural. You have to simply drag one app icon onto another and a folder is created with the default name of the category of the apps. You can then add up to 12 apps per folder. Rather than having to scroll to the right and left through many pages of apps, the new folders allow you to simply tap on a folder which reveals all the apps within it without having to do any scrolling. I've found this a perfect way of cleaning up the apps on my iPhone and being able to access them in a much more efficient manner.

Mail has been enhanced with the addition of an integrated inbox and threaded conversations. I'm not crazy about the integrated inbox because I personally don't want my work and personal email in a single inbox. I find it jarring. However, others may think it is a great feature. As for the threaded conversations, I can't find them. I don't know if you have to somehow turn this feature on through another hidden user interface action but I don't see any threading in my inboxes. I'm assuming that the feature is essentially the approach that Google introduced into its GMail system where all replies to an initial email are nested together. I use that all the time in GMail and in Lotus Notes but I don't see it in my iPhone inboxes. When I find it, I'll report back here what I think of it.

Camera zooming is an incredibly important addition to the iPhone. I take pretty well all my pictures with the iPhone and the other day I was taking pictures of my daughter and her friends at a pre-prom party we hosted at our house. For some of the pictures, we had the group of friends all line up on the other side of the swimming pool from the parents taking pictures. The parents with actual cameras were able to zoom in to capture the group of friends across the pool but I wasn't able to zoom at all and, as a result, missed some of the detail. I now can with the updates made to the camera which allows you to move a slider on the camera view to set the right level of zoom. Pretty sweet!

While my very early experiences with the enhancements made to iOS4 weren't unqualifiedly positive, I am still very pleased with the update and especially so given the price - free.