Hulu UX teardown: 5 user experience fails and how to fix them

Hulu is the first major streaming platform to offer a social watching experience. And with most major league sports now being allowed to resume behind closed doors, Hulu’s combined proposition with ESPN will likely help entertain the service’s 30+ million users over the winter months.

But users have a surplus in choice of streaming services right now, so how will Hulu stay competitive?

With the help of UX expert Peter Ramsey from Built for Mars, we’re going to give Hulu an Extra Crunch UX teardown, demonstrating five ways it could improve its overall user experience. These include easy product comparisons, consistent widths, proportionate progress bars and other suggestions.

Comparing features inside packages

If your product/service has different tiers/versions, ensure that the differences between these options are obvious and easy to compare.

The fail: Hulu has four different packages, but the listed features are inconsistent between options, making it incredibly difficult to compare. Instead of using bullet points, they’ve buried the benefits within paragraphs.

The fix: Break the paragraphs down into bullet points. Then, make sure that the bullet points are worded consistently between options.

 

Steve O’Hear: I’m really surprised this one got past the marketing department. Not a lot to say except that I would argue that when UX, including layout and copywriting decisions, become decoupled from business goals and customer wants, a company is in trouble. Would you agree that’s what has happened here?

Peter Ramsey: Honestly, this happens all the time. I think it’s just a symptom of the designers building things that look nice, not things that work nicely. I probably raise this issue on about one-third of the private audits I do — it’s that common.

Keep a consistent width

Try to maintain a consistent page width throughout a single journey — unless there’s a major benefit to changing the width.

The fail: During the Hulu sign-up process, the page width doubles at a totally unnecessary point. This is disorienting for the user, with no obvious rationale.

The fix: Hulu has a pretty consistent first-half of their journey and then it drops the ball. I’d redesign these “extra-wide” pages to be the default width.

Disney+ UX teardown: Wins, fails and fixes

Disney announced earlier this month that it’s going all-in on streaming media.

As part of this new strategy, the company is undergoing a major reorganisation of its media and entertainment business that will focus on developing productions that will debut on its streaming and broadcast services.

This will include merging the company’s media businesses, ads and distribution, and Disney+ divisions so that they’ll now operate under the same business unit.

As TechCrunch’s Jonathan Shieber reports, Disney’s announcement follows a significant change to its release schedule to address new realities, including a collapsing theatrical release business; production issues; and the runaway success of its Disney+ streaming service — all caused or accelerated by the national failure to effectively address the COVID-19 pandemic.

So what better time than now to give Disney+ the Extra Crunch user experience teardown treatment. With the help of Built for Mars founder and UX expert Peter Ramsey, we highlight some of the things Disney+ gets right and things that should be fixed. They include zero distractions while signing up, “the power of percentages,” and the importance of designing for trackpad, mouse and touch outside of native applications.

Zero distractions while signing up

If the user is trying to complete a very specific task — such as making a payment — don’t distract them. They’re experiencing event-driven behaviour.

The win: Disney have almost entirely removed any kind of distractions when signing up. This includes the header and footer. They want you to stay on-task.

Image Credits: Disney+

Steve O’Hear: This seems like a very easy win but one we don’t see as often as perhaps we should. Am I right that most sign-up flows aren’t this distraction-free and why do you think that is?

Peter Ramsey: Yeah, it’s such an easy win. Sometimes you see sign-up screens that have Google Adwords on it, and I think, “You’re risking the user getting distracted and leaving for what, half a penny?” If I had to guess why more companies don’t utilise this technique, it’s probably just because they don’t want to deal with the technical hassle of hiding a bunch of elements.

The power of percentages

Only use percentages when it makes sense. 80% off sounds like a lot, but 3% doesn’t. Percentages can be a great way of making a discount seem larger than it actually is, but sometimes it can have the reverse effect. This is because people are generally bad at accurately estimating discounts. “What’s 13% off £78?”

The fail: If you sign up to a year of Disney+, then you’re offered 16% free. But 16% isn’t easy to calculate in your head — so people guess. And sometimes, their guesses may be less than the actual value of the discount.

The fix: In this instance, it would be far more compelling (and require less mental arithmetic), if it was marketed as “60 days free.” Sixty days is both easy to understand and easy to assign value to.

Image Credits: Disney+

Percentages may be harder to process or evaluate in isolation as an end user but they are easy to compare with each other i.e., we all know 25% off is better than 10% off. Aren’t you advocating obscuring the actual saving in favour of what sounds better on a case-by-case basis and therefore actually working against the end user? Of course I’m playing devils advocate a little here.

So, it’s actually a really complex dilemma, and there’s no “easy” answer — this would probably make a great dinner time conversation. Yes, if you’re offering two discounts, then a percentage may be the easiest way for people to compare them.

Conversational analytics are about to change customer experiences forever

Companies have long relied on web analytics data like click rates, page views and session lengths to gain customer behavior insights.This method looks at how customers react to what is presented to them, reactions driven by design and copy. But traditional web analytics fail to capture customers’ desires accurately. While marketers are pushing into predictive analytics, what about the way companies foster broader customer experience (CX)?

Leaders are increasingly adopting conversational analytics, a new paradigm for CX data. No longer will the emphasis be on how users react to what is presented to them, but rather what “intent” they convey through natural language. Companies able to capture intent data through conversational interfaces can be proactive in customer interactions, deliver hyper-personalized experiences, and position themselves more optimally in the marketplace.

Direct customer experiences based on customer disposition

Conversational AI, which powers these interfaces and automation systems and feeds data into conversational analytics engines, is a market predicted to grow from $4.2 billion in 2019 to $15.7 billion in 2024. As companies “conversationalize” their brands and open up new interfaces to customers, AI can inform CX decisions not only in how customer journeys are architected–such as curated buying experiences and paths to purchase–but also how to evolve overall product and service offerings. This insights edge could become a game-changer and competitive advantage for early adopters.

Today, there is wide variation in the degree of sophistication between conversational solutions from elementary, single-task chatbots to secure, user-centric, scalable AI. To unlock meaningful conversational analytics, companies need to ensure that they have deployed a few critical ingredients beyond the basics of parsing customer intent with natural language understanding (NLU).

While intent data is valuable, companies will up-level their engagements by collecting sentiment and tone data, including via emoji analysis. Such data can enable automation to adapt to a customer’s disposition, so if anger is detected regarding a bill that is overdue, a fast path to resolution can be provided. If a customer expresses joy after a product purchase, AI can respond with an upsell offer and collect more acute and actionable feedback for future customer journeys.

Tap into a multitude of conversational data points

Five ways to bring a UX lens to your AI project

As AI and machine-learning tools become more pervasive and accessible, product and engineering teams across all types of organizations are developing innovative, AI-powered products and features. AI is particularly well-suited for pattern recognition, prediction and forecasting, and the personalization of user experience, all of which are common in organizations that deal with data.

A precursor to applying AI is data — lots and lots of it! Large data sets are generally required to train an AI model, and any organization that has large data sets will no doubt face challenges that AI can help solve. Alternatively, data collection may be “phase one” of AI product development if data sets don’t yet exist.

Whatever data sets you’re planning to use, it’s highly likely that people were involved in either the capture of that data or will be engaging with your AI feature in some way. Principles for UX design and data visualization should be an early consideration at data capture, and/or in the presentation of data to users.

1. Consider the user experience early

Understanding how users will engage with your AI product at the start of model development can help to put useful guardrails on your AI project and ensure the team is focused on a shared end goal.

If we take the ‘”Recommended for You” section of a movie streaming service, for example, outlining what the user will see in this feature before kicking off data analysis will allow the team to focus only on model outputs that will add value. So if your user research determined the movie title, image, actors and length will be valuable information for the user to see in the recommendation, the engineering team would have important context when deciding which data sets should train the model. Actor and movie length data seem key to ensuring recommendations are accurate.

The user experience can be broken down into three parts:

  • Before — What is the user trying to achieve? How does the user arrive at this experience? Where do they go? What should they expect?
  • During — What should they see to orient themselves? Is it clear what to do next? How are they guided through errors?
  • After — Did the user achieve their goal? Is there a clear “end” to the experience? What are the follow-up steps (if any)?

Knowing what a user should see before, during and after interacting with your model will ensure the engineering team is training the AI model on accurate data from the start, as well as providing an output that is most useful to users.

2. Be transparent about how you’re using data

Will your users know what is happening to the data you’re collecting from them, and why you need it? Would your users need to read pages of your T&Cs to get a hint? Think about adding the rationale into the product itself. A simple “this data will allow us to recommend better content” could remove friction points from the user experience, and add a layer of transparency to the experience.

When users reach out for support from a counselor at The Trevor Project, we make it clear that the information we ask for before connecting them with a counselor will be used to give them better support.

If your model presents outputs to users, go a step further and explain how your model came to its conclusion. Google’s “Why this ad?” option gives you insight into what drives the search results you see. It also lets you disable ad personalization completely, allowing the user to control how their personal information is used. Explaining how your model works or its level of accuracy can increase trust in your user base, and empower users to decide on their own terms whether to engage with the result. Low accuracy levels could also be used as a prompt to collect additional insights from users to improve your model.

3. Collect user insights on how your model performs

Prompting users to give feedback on their experience allows the Product team to make ongoing improvements to the user experience over time. When thinking about feedback collection, consider how the AI engineering team could benefit from ongoing user feedback, too. Sometimes humans can spot obvious errors that AI wouldn’t, and your user base is made up exclusively of humans!

One example of user feedback collection in action is when Google identifies an email as dangerous, but allows the user to use their own logic to flag the email as “Safe.” This ongoing, manual user correction allows the model to continuously learn what dangerous messaging looks like over time.

Image Credits: Google

If your user base also has the contextual knowledge to explain why the AI is incorrect, this context could be crucial to improving the model. If a user notices an anomaly in the results returned by the AI, think of how you could include a way for the user to easily report the anomaly. What question(s) could you ask a user to garner key insights for the engineering team, and to provide useful signals to improve the model? Engineering teams and UX designers can work together during model development to plan for feedback collection early on and set the model up for ongoing iterative improvement.

4. Evaluate accessibility when collecting user data

Accessibility issues result in skewed data collection, and AI that is trained on exclusionary data sets can create AI bias. For instance, facial recognition algorithms that were trained on a data set consisting mostly of white male faces will perform poorly for anyone who is not white or male. For organizations like The Trevor Project that directly support LGBTQ youth, including considerations for sexual orientation and gender identity are extremely important. Looking for inclusive data sets externally is just as important as ensuring the data you bring to the table, or intend to collect, is inclusive.

When collecting user data, consider the platform your users will leverage to interact with your AI, and how you could make it more accessible. If your platform requires payment, does not meet accessibility guidelines or has a particularly cumbersome user experience, you will receive fewer signals from those who cannot afford the subscription, have accessibility needs or are less tech-savvy.

Every product leader and AI engineer has the ability to ensure marginalized and underrepresented groups in society can access the products they’re building. Understanding who you are unconsciously excluding from your data set is the first step in building more inclusive AI products.

5. Consider how you will measure fairness at the start of model development

Fairness goes hand-in-hand with ensuring your training data is inclusive. Measuring fairness in a model requires you to understand how your model may be less fair in certain use cases. For models using people data, looking at how the model performs across different demographics can be a good start. However, if your data set does not include demographic information, this type of fairness analysis could be impossible.

When designing your model, think about how the output could be skewed by your data, or how it could underserve certain people. Ensure the data sets you use to train, and the data you’re collecting from users, are rich enough to measure fairness. Consider how you will monitor fairness as part of regular model maintenance. Set a fairness threshold, and create a plan for how you would adjust or retrain the model if it becomes less fair over time.

As a new or seasoned technology worker developing AI-powered tools, it’s never too early or too late to consider how your tools are perceived by and impact your users. AI technology has the potential to reach millions of users at scale and can be applied in high-stakes use cases. Considering the user experience holistically — including how the AI output will impact people — is not only best-practice but can be an ethical necessity.

This UX specialist opened 12 UK bank accounts and ‘logged everything’

“I’ve got a really high attention to detail, which might sound great, but it’s possibly a curse because I can’t help but spot problems with everything around me,” says Peter Ramsey .

He’s the founder of Built for Mars, a U.K.-based UX advisory, and he has spent the last three months documenting and analyzing the user experience of a dozen leading British banks — both incumbents and challengers — including Barclays, HSBC, Santander, Monzo, Starling and Revolut.

“Quite literally, I opened 12 real bank accounts,” he explains. “You remember the stress of opening one account? I did that 12 times, [and] it was probably a terrible idea. But I really needed to control as many variables as possible, and this was the only way of doing that.”

Next, Ramsey says he “logged everything,” recording every click, screen and action. “I saved every letter, and made a note of when they arrived. I recorded pretty much everything I could,” he recalls. “At one point I even weighed all the debit cards to see if some were heavier. That was a total waste of time though, because they all weighed the same amount. But you see what I mean, I just thought about making it as scientific as possible. Also, UX is really quite subjective, so I wanted to back up my opinions with some more quantifiable metrics.”

The resulting analysis — covering opening an account, making a first payment and freezing your card — supported by individual bank case studies, is being published on the Built for Mars website over the month with a new interactive chapter released weekly.

After being given early access to the first three chapters and an initial series of case studies, I put several questions to Ramsey to understand his motivation, methodology and what he learned. And if you’re wondering which bank came out on top, keep reading.

TechCrunch: Why did you choose to do this on banks?

Peter Ramsey: My background is in fintech, and I think the banks are just in this weird place right now. When they first came out I think consumers were surprised at how much better the apps were. Banking was renowned for having old software, it was almost acceptable for an old bank to be buggy. But now that these challenger banks have been out for five years, I think that perception has changed. So I chose the banks because they represent this industry of “challenger” versus “legacy.” Plus, for billion-dollar companies, you’d expect them all to really care about experience.

Why BMW needs to own its customer experience from start to finish

For the last few years now, BMW has wrestled with the question of what it’ll mean to be a luxury car manufacturer in the age of electric cars, autonomous driving and rapidly changing — and increasing — customer expectations. What, after all, makes something the “ultimate driving machine” when the driver eventually stops driving?

For BMW, the answer is a renewed focus on technology and the in-car experience it enables, without forgetting its heritage in performance cars. To discuss the state of the company’s transformation, not just in terms of its cars but also its business model, I sat down with BMW’s outspoken VP of Digital Products and Services Dieter May shortly after the company unveiled the latest version of its in-car operating system.

“We build digital products and services that are meant to help us differentiate our core product, the car and generate revenue,” May said. “But these digital services also provide us with channels and touch points that allow us to now have a direct relationship with the customer on the sales side and talk to the customer directly.”

In the car industry, however, the sales channel has traditionally been the dealership. That’s where you buy the car and that’s where you get it serviced. It’s the dealer who knows (ideally) who you are and what you want. The manufacturer’s role in this model is to build the car, maybe build a bit of a central online presence with a configurator so customers can get some idea of the car’s price — and get out of the way.

That’s not the future that May envisions, though. And neither is it one where the big tech companies like Apple and Google own the driver and the user experience.

“As we’re building the digital products in the car, we are also building out the car as a channel and touchpoint at the same time,” May noted. “We’ll have our app, a personal assistant etc. and with that, we can create a user profile and provide that to our sales teams. Today, virtually every car manufacturer can’t talk directly to the customers because the customer belongs to the dealer, and because the different business units, like after sales, financial services, etc., aren’t unified and all try to talk to the customer separately.”

So for BMW, digital experiences in the car are one thing — and you can expect to hear a bit more about this in the coming weeks and months — but the company is looking beyond this and how it can use this transformation to also create new business opportunities that go beyond maybe selling an in-car Spotify subscription for a few dollars. But what gets May most excited about this is the prospect of being able to talk to the customer throughout the ownership lifecycle. “That’s the cool part, because it allows me to keep the product ‘car’ fresh throughout the lifecycle and manage it like a device,” he said.

The move that May is hinting at here means that BMW wants to not just focus on selling cars but to create a model where it can extract some revenue from users throughout the car’s life. As an example, May noted that BMW may sell you a Mini with a charging package and, in addition, it’ll sell you a flat-rate subscription to charge it. “There are so many opportunities here, but you have to play it smart, both before somebody buys the car and after the sale.

If BMW wants to own the customer, though, that means dealerships have to change. “The dealers will have to grow into a different role over time,” May acknowledged. “We expect and hope that just as we will share data with the dealer, the dealers will share their data with us. A small piece of a larger cake is still better than nothing.”

Customers will still come to the dealer for their service needs, so BMW isn’t cutting them out completely, but the company definitely wants to own a larger part of the relationship with the customer. And at the end of the day, it’s the dealer who represents the manufacturer, whether that’s taking somebody on a test drive or helping the customer take delivery of a car.

“What’s most important for us — and everybody is talking about autonomous driving and electric vehicles and so on — but if we don’t become a customer-centric company, then we are destined to fail. The number of digital elements in our customers’ lives and in the car continues to increase, and if we don’t understand that, we’ve got a problem.”

As for its current in-car systems, May told me that BMW now has more than three million registered users for its ConnectedDrive system, but what’s maybe more important is that the number of user interactions is increasing significantly faster than that. What’s interesting to hear is that the way BMW thinks about these users is pretty much in line with any consumer internet company. The team tracks monthly, weekly and daily active users, for example, and is working to increase those engagement numbers with every update.

One problem car manufacturers have long suffered from is that cars stick around far longer than smartphones, and that the in-car technology can quickly seem out of date. Because it is betting on a connected car that is always connected to the cloud, BMW (and, to be fair, many of its competitors) is now able to update the in-car software. That’s true for new head units, but not necessarily for older ones, and fragmentation remains an issue — though with a standardized model for both BMW and its Mini brand, that’ll likely be less of a problem for newer cars than for those that launched two or three years ago.

“In the car industry, a lot of people think that everything has to be backward-compatible reaching back 20 years, but my take is that we have to be more like smartphone vendors,” said May.

Taking a page from the software industry, the BMW team often launches new features that are akin to minimal viable products. That’s not necessarily something the luxury car buyer is used to, of course, but it does allow the company to test new features and expand on them as they gain traction.

The next concrete step for BMW in this journey is to feature an interactive personal assistant in the car that knows about the customer. May believes this will drive a lot of usage. Although the exact details remain to be seen, the BMW team hinted that we’ll learn more in the fall.

Salesforce acquires Sequence to build out its UX design services

screen-shot-2017-02-02-at-01-31-39 Salesforce has made another acquisition that underscores how the CRM and cloud software giant is looking to sell more services to its customers that complement the software they are already buying. It has acquired Sequence, a user experience design agency based out of San Francisco and New York that works with brands like Best Buy, Peets, Apple, Google and many more. The news was announced… Read More

Look out for Ultrahaptics haptic feedback in new cars this year

bosch-showcar_ces2017_1008 Since Ultrahaptics was founded in 2013, the company has showed off a string of better and better technical demos. Which has been fun, but obviously where the rubber really hits the road (pun most emphatically intended) is when the products start showing up in the real world. 2017 may just prove to be the year where that happens. Read More

Bad UX kills

A confusing traffic light system with multiple signal heads. It clogs systems, causes accidents, wastes energy and makes people unhappy. It’s more than a bad experience on a website — in cities, bad user experience (UX) design can actually kill. We’re talking about signage, public spaces, civic and emergency communications and other forms of urban design that influence our daily routines and, in some cases, are there expressly for… Read More