Separating the Signal from the Noise
This chapter is from the book
This chapter is from the book
This chapter is from the book
WeChill is a data-driven, streaming content platform that uses a variety of metrics and analytics to make decisions. The company places a strong emphasis on customer focus and has built a product that meets the needs of its users. Data-driven decision-making is a cornerstone of WeChill’s business strategy, and the company uses extensive research and experimentation to inform its decisions.
WeChill has a strong company culture that values transparency, accountability, and innovation, and its employees are empowered to take ownership of their work. The company’s personalized content recommendations and sophisticated algorithms drive engagement and retention.
Their content strategy focuses on creating high-quality original content that keeps users returning for more. With a global reach spanning over 150 countries, WeChill has quickly scaled and achieved significant growth.
Recently, WeChill has run into a rough patch as users leave in droves for one of their rival competitors. This has caused WeChill to do a deep analysis of its user base, including users who have recently left the platform. All data that has been analyzed thus far indicates that customers are happy. Even customers that have left report having a good experience with WeChill.
With a proud company culture, this sudden change from leader to lagger has WeChill spiraling internally. A lot of noise and panic among managers begins as WeChill searches for answers. But no matter how hard the company tries, it cannot understand why so many people are leaving.
As the market shifted on WeChill, the company failed to notice and then had difficulty identifying why it was losing users. The customer satisfaction gap had changed because of competitor offerings; while viewers had been happy with WeChill’s offerings in the past, competitors were now offering better content than. That led to an increased satisfaction gap between what viewers experienced with WeChill compared to what viewers experienced with its competitors.
On the surface, WeChill was doing all the right things, but the market was shifting under its feet. By the time the company realized it, panic had set in, causing distraction instead of focus. It is important to constantly evaluate what signals you are analyzing and whether they still make sense or have become noise.
Identifying Signals
It is not enough to identify measures and goals once and move on. You must constantly evaluate the usefulness of both your goals and what you are measuring. Your goals, what you measure, and how you measure will change and evolve as your products and customers change and evolve.
WeChill, as a content creation platform, looks at the following kinds of signals:
Strategy and goals: WeChill has a clear strategy and goals to guide its decision-making. The company’s goal is to become the world’s leading streaming entertainment service, and it uses this goal to prioritize its decision-making.
User data: WeChill collects vast amounts of user data to identify trends, preferences, and patterns in user behavior. The company uses this information to decide which shows and movies to produce or license, as well as how to market them.
A/B testing: WeChill uses A/B testing extensively to optimize the user experience. It tests various versions of the user interface, content recommendations, and marketing campaigns to determine which perform the best. By comparing and contrasting the results of the tests, the company can arrive at new hypotheses.
Content analysis: WeChill analyzes the performance of its existing content to determine what types of content are most popular with viewers. This helps the company decide which new content to produce or license. WeChill cannot simply ask viewers whether they like the content because that could lead to a blind spot. Viewers might like the content but not as much as something else a competitor is offering. WeChill must measure the satisfaction gap relative to alternatives to understand if it has the right content.
Employee feedback: WeChill encourages employees to give feedback on the company’s strategy and decision-making processes. The company believes this feedback helps it make better decisions by considering multiple perspectives.
WeChill captures and analyzes user data in different ways. Here are some examples:
Viewing data: WeChill collects information on what shows and movies its users are watching and how long and frequently they watch them. This data helps WeChill identify which shows are popular.
Engagement data: WeChill also collects data on how users interact with its platform, such as which shows users add to their watchlists, rate or review, and share on social media. This data can help WeChill understand how engaged users are with a particular show.
Demographic data: WeChill also collects information on its users’ age, gender, location, and other demographic factors. This data can help WeChill understand which shows are popular among different user base segments.
Search data: WeChill tracks what users are searching for on its platform to help identify trends and understand what users are interested in.
Survey data: WeChill also uses surveys to gather feedback from its users on what they like and dislike about its shows. This data can help WeChill understand why users do or don’t enjoy a particular show.
If a show has high viewing and engagement data but low survey scores, WeChill may decide to renew the show but make changes to address the issues raised in the surveys. Alternatively, if a show has low viewing and engagement data, WeChill may decide to cancel it.
As you can see in our example, WeChill is deliberate in how it identifies signals and has elaborate user data to sift through. Yet, it still missed the market shift, and its competitors noticed before WeChill. Too much information can cause as many problems as not enough.
You must frequently ask yourself, “What questions do I need to answer right now?” to decide what you need to look at. That can help you to identify which signals are relevant now.