Multi-Device Interactions – Part 2 : The Models

  • Flavio Loetscher

In our last blog post we started off with John's story to show the everyday encounter of multiple devices and screens, and outlined the emergence of the second screen business. The classical second screen solution is a companion app for mobile devices that delivers additional information to TV content, e.g. a quiz or sport statistics on your smartphone or tablet. With all the possibilities in a multi-device world, it's crucial to focus on the conductor of all these instruments – the user! In the following sections we dive into some theoretical models on multi-device interaction.

Why would the user choose one device over another, pursue one activity over another? How important is the context and what other factors contribute to a specific user-behavior pattern?

Let's have a look at the context which in many cases influences the choice of device.

Context of use

Digital devices are being used in different social environments, as our story of John tries to transcribe. The context of use is key to the selection of a device. It makes indeed a big difference whether you sit in a coffee shop, chill out at home on the couch, or are with friends at a bar watching a football game.

What are the key factors which drive the preference for a device over others? Google identifies four of them:

  • Time : How much time is available and needed for a given task?
  • Goal : What is the defined intent, goal or task?
  • Location : Where is the user and which devices are physically close?
  • Attitude : How does the user feel and what is going on in his mind?

In our example scenario, John is on the go, riding the bus from his office to the shop during rush hour. This doesn't allow him to engage in longer tasks as many distractors and possibility of disruptions are present. Here, the smartphone is the right device. But later at night, relaxing with Michelle at home, it's a much different setting with respect to the four factors above: time is available, John and Michelle are in a relaxed mindset, and start discussing vacation ideas. Their goal is to find more inspiration for their trip, the casual browsing task that can easily be completed from the couch, at home. A tablet is a perfect match for this purpose.

The right device

Google attributes different characteristics to each device. Whereas PCs and laptops are the primary tools to be “productive and informed” and mostly used at home or at the office with the idea to pursue goals that demand time and focus, smartphones are the connecting devices. Usually used on-the-go as well as at home, and best for communication and connecting with others. Often, smartphones are of course used when time is sparse or information needs to be accessed quickly e.g. small tasks. And tablets? Entertainers, clearly. 70% of the time, Tablets used is at home and mostly for browsing and entertainment, when time is largely available.

A Consumer in the report sums up as following:

“My phone… I consider it my personal device, my go-to device. It's close to me, if I need that quick, precise feedback.
When I need to be more in depth, that's when I start using my tablet. The other part of it is where I disconnect from my work life and kind of go into where I want to be at the moment… I'm totally removed from today's reality. I can't get a phone call, I don't check my email it's my dream world.
And then moving to the laptop, well, for me that's business. That's work. I feel like I've got to be crunching numbers or doing something.”

Bradely, Google Report 2012.

Microsoft defines different archetypes to understand marketers how users relate to their devices. It's a form of labeling six diverse user types.

The Everyman: TV as one of the most popular devices in our multi-screen world that delivers passive entertainment and comfort

The Sage: The laptop informs, empowers and teaches – clearly key to productivity

The Jester: The Gaming Console immerses consumers in another world

The Dreamer: E-Readers help us to escape in the world of books and are mainly used for this purpose only (despite the fact that some of them provide internet browsing too)

The Explorer: The Tablet facilitates discovery and investigation and is a great device on the go and rich in media and video

The Lover: The Mobile phone is the most personal device and evokes intimacy, commitment and trust – however, its downside is the constant demand of the user's attention

Now that we might understand how a user comes to choose one device over another and how he relates to these devices, we may think about how to interact with a multiple of devices.

Modes of multi-screening

Screens can either be used sequentially or simultaneously (Google, 2012).

Sequential device usage occurs when one task is initiated on device A and finishes on device B. One common example is browsing the web for shoes and bookmarking the interesting products on a smartphone, to later purchase the item on a laptop. In John's story, many sequential actions happen. For instance the vacation trip planning is an activity that goes through different devices throughout the day. Based on numbers from the Google report, over 90% of users indeed engage in a sequential use of devices to accomplish a given task the same day. It is thus not astonishing that Google launched a new Adwords tracking measurement for marketers in early October 2013. With the “Estimated Total Conversions” cross-device conversions can be calculated.

Parrallel device usage. Opposed to the sequential mode is the parallel use mode. Google speaks of simultaneous use when multiple screens are active and information on the second screen is either related or unrelated to a main screen. The report distinguishes between multi-tasking (unrelated activity) and complementary usage (related activity). Clearly, multi-screening with a smartphone and TV is ranked as most frequent activity among the users in their study, followed by smartphone & laptop. According to their research, emailing, browsing and social networking are the most performed task during simultaneous screen usage.

Even though multi-tasking and juggling different activities at the same time has been stated to mainly induce negative effects on performance and accuracy of a given task (Rachel et al, 2011), 78% of the participants in the Google Multi-Screen World study perform multi-tasking (simultaneous usage). Complimentary usage was thus only 22%. It turns out that 77% of TV viewers use another device at the same time while watching Television. Often TV ignites search; at least for 1/4 of search inquiries occasions that are prompted by television (Google, 2012).

There are other approaches to defining modes of use in a multi-device world. Here is another one.

The four paths of engagement

Microsoft on the other hand defines the mode of use with four “paths of engagement” with devices that partially overlap with Google's definitions of sequential and simultaneous use.

  1. Content Grazing

  2. Investigative Spider-Webbing

  3. Social Spider-Webbing

  4. Quantum

Content Grazing is the classical distractive behavior when using multiple screens. It can be either related or unrelated to the content on the primary screen, similar to Google's definition of simultaneous use. It's often habit driven about small tasks running in the background. Think of Michelle texting back and forth with Sarah during the movie.

The Investigative and Social Spider-Webbing paths of engagement are about consuming content that is clearly related to a primary screen. Microsoft's distinction between investigative and social are straightforward: Investigative Spider-Webbing happens when moments of curiosity or knowledge seeking trigger a search action. Michelle's interest to find additional information about the movie star is a good example. Social Spider-Webbing on the other hand, is about social engagement in forms of conversations and connecting to like-minded individuals. Let's say a tweet or online discussion about content on a primary screen (Microsoft, 2013).   

Quantum Tasking is the equivalent of Google's sequential use. Here, intended tasks travel over space and time from screen to screen. Meaning big tasks are often divided in subtasks. What the report also states is that, when it comes to shopping, spontaneity plays an important role that mostly is present while using a smartphone. In our story it's when John purchases the flight tickets, remember? At home, clearly PC or Laptop is the leading device. Nevertheless, 67% of the studied users started shopping on a smartphone and accomplished the goal on a PC/Laptop (Google, 2012), just like John did.

With all the different modes of use mentioned, users and solution providers started to care a lot about information and interaction orchestration. No wonder some theoretical concepts for screen coordination have thus been developed.

Screen coordination

In Ecosystems of screens, PRECIOUS DESIGN STUDIO documents six patterns for screen coordination

Coherence (appearance and functionality is coherent across different devices)

Syncronization (data gets syncronized)

Screen sharing (multiple screens share a single source)

Device shifting (possibility to actively shift content from one device to another)

Complementarity (the classical TV companion app)

Simultaneity (devices display similar content simultaneously)

The strategies defined, should help to understand and describe our multi-screen world. Here is an example: When John is working on his report, his data is synchronized and displayed coherently on different screens. While watching TV, the companion app Michelle uses is most likely simultaneous and complementary. Device shifting occurs, when John preselected a song on one device and later streamed his music library in the gym.


The trend towards a multi-screen world is emerging and is just about to become mainstream. How we interact with multiple screens is an art for itself. Different modes of use, contextual factors and screen coordination strategies have been outlined and are crucial but must always be considered together, in a holistic approach. The user acts as the “conductor of an orchestra of devices” that plays a hopefully “harmonious experience tune” that some user experience architects designed.

To to do so, we can cut down to four main factors that contribute to the understanding: 




Screen and Interactions

In a next post we'll introduce the Multi-Device Interaction Canvas – a tool to model and think multi-device interaction scenarios in a simple and convenient way.

Tell us what you think