Product Design Lead | Phone & TV | Design a new feature
Led design for this proposed feature that enabled users to set up a TV streaming device using a mobile app instead of their remote control. Our work uncovered that we were solving the wrong problem.
Mobile Enhanced Setup (MES) was a proposed feature that enabled users to set up their AT&T TV streaming device using their smartphone instead of their remote control - enabling them to complete setup faster and begin watching TV.
I led a team that was meant to design and deliver this feature but as it turned out, our immersion into the problem space and synthesis led to a design lens that we were solving the wrong problem. This led to working sessions with product stakeholders to frame the right problem space and align on an approach to begin working on it.
What is the business ask?
Build a mobile app feature that allows users to set up their AT&T TV streaming device using their phone instead of their remote control.
AT&T TV is a live TV streaming service meant to replace traditional cable or satellite TV subscriptions. Users who sign up for the service receive a TV streaming device (similar to a Roku or Apple TV).
The AT&T TV device is the primary viewing device for the vast majority of users who sign up for the AT&T TV service (accounts for ~90% of all viewing sessions). As such, it is crucial that users are able to successfully set up the device and begin watching.
Data showed that the device setup process was one of the biggest pain points that users had with the service. Adding to this, many users were purchasing multiple AT&T TV devices to use in their homes. And were being made to repeat the entire setup process for each box.
Giving users the ability to setup their device using their phones was proposed as a possible way to reduce some setup issues and simplify the process of setting up multiple boxes. The design team was tasked with defining and designing this solution (aka MES).
Who are we designing for?
More than 1 in 2 are older users who are moving from traditional TV services and are less confident with new technology.
The AT&T TV user base skews older. These users are in most cases moving from traditional TV services.
They are less confident in learning new technology and more likely to resist adopting new technology.
They do not use many other video-entertainment services, so expectations for their TV service are not influenced by other streaming services and capabilities.
What do we know?
While there is a user desire for the feature, there are multiple issues that design will need to solve to turn desire into reality.
Do older users want to self-install?
Given that many AT&T TV users were switching from traditional TV services that had installers come to the house to set up the devices, were they open to self-installing the device at all?
Research showed that half of likely AT&T TV users expressed a strong desire to self-install the service. This preference was due to the inconvenience of open-ended installation windows that left them waiting for a technician to show up.
Known Pain Points around Setup
Older users were significantly less satisfied with setup. Tracking data showed that ~40% of them needed assistance to complete setup (most often this meant calling customer support).
Biggest areas of user frustration were around -
Issues logging into the service. A large percentage of this is driven by users who do not what their credentials are.
Setup duration and a lack of context of where they were in the setup process
Some steps do not match user expectations (i.e. why is this step a part of the setup process?)
Screens are displayed that create a false sense of completion when setup is not nearing completion. This is especially true for the device software update that happens in the middle of the setup process.
POC Testing and Feedback
The development team built a POC to demonstrate communicating between the phone and streaming device. We did some light design work to transfer this to a prototype to test some of the assumptions the product and development stakeholders were making about the feature.
The testing results revealed some key issues -
Almost all users preferred using the remote control over the mobile phone. They were unclear on what using the phone would mean and/or what benefit it could have.
User expectations and behavior meant that the remote control already in their hand when they saw this screen. So, not only would there need to be a clear benefit to setting up using the phone but it would also need be strong enough to overcome the inertia of just continuing with the remote control.
Downloading the correct app was a major hurdle for several users. This was on account of AT&T having several apps in the App Stores and user confusion around which was the right app to download
Unless we were able to quickly get users to the correct app, they would abandon MES. And MES would fail before it even began.
Confusion with Google Assistant and Android TV onboarding screens. A significant part of the device setup requires users to navigate a set of contractually required Google onboarding screens introducing Google Assistant and other Android TV features that are part of the device feature set. AT&T does not own or control these screens. Users found this section confusing - they did not understand why Google was part of an AT&T TV setup. This was compounded by the technical constraint that the phone was limited to displaying a d-pad since we did not know what could appear on these Google screens. Without knowing what's on these screens, MES is unlikely to have any impact on solving the user issues in this section.
Hold on...
As we gathered info, a couple of things were emerging that made it seem like we were focusing on the wrong problem.
The users we were designing for did not seem like the right target for a phone based solution.
Regardless of what device was used to complete setup, the setup process itself looked it like had some major usability concerns especially around confusing step, duration and lack of context.
We started raising these red flags to other stakeholders, walking them through the info we had gathered and the picture that was emerging. While stakeholders generally in agreement that these would be large obstacles, they were not convinced this meant we were likely focusing in the wrong problem space.
Design Ask & Goals
We decided to dive in a little more and explore in design if the benefits emerging from the MES would solve for the user mismatch and existing setup issues.
Design Ask
How might we create an experience that provides easy access to a phone app that simplifies the current device setup?
Design Goals
Clearly communicate the benefit of MES to enable users to decide if it is the right option for them
Provide better context and set clear expectations across the setup flow
Reduce perception of length of time needed to complete setup
Set and maintain a clear mental model for the user of the relation during setup between phone and AT&T TV streaming device
Measures of Success
MES must have a setup completion success rate on par or better than the rate for remote control setup
Users who chose MES as their setup path, must have less than n% customer care calls related to setup items
Design Directions
With the baseline info we had gathered, when we flowed out what an MES solution could look like, it only made it even clearer that MES was not going to solve the actual issues that users were running into with setup.
Some of the issues that became even clearer in the design exploration included -
Need a better solution for users to get and retain their login information. Also, login screens need to be updated to provide users with how to get help if they do run into login issues.
A significant portion of the setup duration is taken by a software update that happens in the middle of the setup process. This creates multiple issues ranging from user frustration due to a false sense of completion to what does the user do on the phone while the update is in progress.
Remote control pairing and any resulting fail/error cases could only be done with the remote control itself, so there was no solution where MES could be used to complete setup end to end.
Without a change in how the Google screens were handled, MES could not solve any issues here. In addition, Google had their own mobile device setup as part of their screens. Aside from the user confusion this created, it was unknown how Google could hand back to MES once their section was complete.
And in addition to not solving for the core issues, we were likely introducing additional obstacles to users who were already unsure of their skills using a phone app.
We also explored an alternative direction that focused on using the phone only in the sections where it had a clear benefit over the remote control - setting up an internet connection and logging in. This was better from a MES solution standpoint but meant that a user would have to use the remote control to complete most setup sections AND did not solve for the issues they faced in these sections.
This is not a drill...
MES does not match or solve for
Who our users are and their lack of comfort with a phone based setup
Lack of context and/or setting the wrong context (false completion) in different setup sections
Technical issues that are causing setup errors and user frustration
Contractual constraints that are adding significantly to duration and causing user confusion
These issues and constraints are linked to the setup flow and implementation on the streaming device itself and are not going to be addressed in any meaningful way by offering the phone as an alternative means of setup.
We had made sure that our product, business and dev partners were in the MES working sessions and reviews. So, the outcome above was not a surprise to any of them and most were already in alignment that we needed to step back and redefine what the actual problem was.
Recommendation
Fix the underlying setup issues. That will do more to solve user frustration than adding support for the phone as an alternative means of setup.
An example of what can help
The current setup flow and has no screen that sets context that a setup is about to begin or the info the user might need to successfully complete setup. The current flow just launches into the first setup section.
We added this screen as the first screen that users would see before starting the setup flow. The hypothesis was that it would give users a feeling of control and also set context of what info they would need to successfully complete the setup. In testing, 88% of users rated this screen as helpful.
What Next
Specific setup improvements added to product roadmap.
We presented our findings and recommendations in executive review. They were in agreement that the work needed was on the underlying setup issues on the streaming device. Using our findings as a starting point, specific setup improvements were added to the product roadmap. We also kicked off a project to baseline the setup experience so we could track the impact each specific setup improvement had.
What did I learn?
Always question the problem.
Not so much something new, more a re-affirmation of a long-held belief. Always question the problem. Most often, info gathering and synthesis will help validate that the problem is real. In these cases, the synthesis helps add clarity or nuance. But every now and then, something like MES comes along. And the value of the synthesis is magnificent.
Comments