Product Case Metrics - A List
This is a Work in Progress (WIP) list of metrics to consider when thinking about how to measure a case solution. You typically want 3 to 5 metrics. If it is an execution case and you are working towards metrics for success, you want 5 to 10 metrics.
Not all metrics are on this list and many of these metrics don’t work for cases, but I found putting together the list helps some think through possible metrics and not always rely on DAU or MAU or CSAT scores.
Engagement: Traffic metrics
DAU: Daily Active Users —Number of active users on the selected day. Activity typically a click, comment, post, scroll, application activation, etc.
MAU: Monthly Active User — Determines how many users of our product are active during the month.
New users: The number of users who performed any activity in the product for the first time in the selected period.
Churned users: The number of users who have left the product in the selected period. The number of users who have not performed any activity for the last N days. Usually, N = 28. For subscription products, number who stopped paying.
Reactivated users The number of users who returned to the product after leaving it. (Also returning users and revived users.)
Users balance A metric that shows simply whether the number of active users is increasing or decreasing. Users balance = new users + reactivated users — churned users
Bounce rate: Ratio of Users who did not perform any action after entering the site / all active users in a time period
Engagement: Behavior
Daily user engagement: Total attention budget we receive from the user on one day. It should be reported using the median and selected percentiles (usually 10, 25, 75, and 90). For Meta, it is an ecosystem metric typically Daily engagement in Product X/daily engagement within the ecosystem
Daily user engagement = total sessions duration / DAU,
Daily user engagement = median of total session duration for a single user.Total sessions: A total number of sessions made by all users in the selected period. A sequence of recorded activities, with a pause of no more than 30 minutes between two consecutive activities.
Total sessions duration: Total time of all sessions made by all product users for the selected period. Session time is the interval between the first and last recorded activity during the session.
Average sessions duration: The amount of time a single user spends on average per session. = Total sessions duration / Total sessions
Daily sessions per active user: A number of sessions, i.e., daily interactions with the product. It tells how strong the habit of using the product users is. Note that the minimum value for this metric is 1, not 0. Daily sessions per active user = Total sessions / DAU.
Weekly product adoption: A metric that can be calculated only in the context of a subscription model, calculated for a single customer. Its decline often heralds the client’s departure, which can be used to prevent parts go away. Weekly product adoption = Number of active users/number of paying users.
Product feature usage: It is worth counting it separately for several key functionalities in the product. Usually, it is calculated for days or weeks, less often for months, depending on how often the functionality should be used to bring value to the user. Very useful for analyzing the behavior of new users. Product feature usage = a number of users who used the selected functionality/number of active users who had access to this functionality.
Key actions per session: Key Actions are the most important things a user can do to derive value from a product. Most often used in the analysis of the behavior of new users, it allows for assessment of how useful the product is for them. Key actions per session = Total number of key actions performed by active users / Total number of sessions.
Engagement: Retention
Retention can be hard to measure. I typically try to stay away from it for interviews, but sometimes you need to address it. You may have heard D1, D3, D7, etc. These metrics are more suited to consumer than business products. Keep reading to learn more.
D1 cohort retention: It is all about first impressions: users who return for a second day in a row. What percent of users returned the day after using the product for the first time? D1 cohort retention = Users who used the product the day after first use / all new users the day before.
D3/7/28 cohort retention: A percentage metric measures medium-term (D3 / 7) and long-term (D28) commitment to a product. D3 / 7/28 cohort retention = Users who used the product 3/7/28 days after first use / All new users 3/7/28 days before. Simply put: what% of users returned 3/7/28 days after using the product for the first time?
Health Metrics
Website loading time: Possible to calculate only for website products (e.g., e-commerce store). Median shows typical user experience on page load, high percentile especially bad user experience on page load. Both values are essential when studying the impact of the page load speed on user experience.
Uptime: This is a metric used to better understand a system's overall reliability. It is best described as the time that services are online divided by total active availability.
Utilization: In this instance, we are thinking about a platform utilization rate. It answers how much of your platform’s capacity is being used at a particular point in time or as an average. In the world of data, when we get to 80% utilization we start to think about offloading data to deep storage or increasing our capacity.
Monetization
Transactions - The number of transactions performed in the product by its users. A metric is reported for a given period, usually a day, week, or month.
Average transaction value - Often used to compare different user groups. Average transaction value = Gross revenue from all transactions / total number of transactions.
MAPU: Monthly Active Paying Users — Number of users who paid in the last month. A month is defined as a fixed number of days (28 or 30) to avoid problems with different lengths of months.
Paying users: The number of users who have made any payment on the product during the selected period.
Bookings vs. Revenue A common mistake is to use bookings and revenue interchangeably, but they aren’t the same thing. Bookings is the value of a contract between the company and the customer. It reflects a contractual obligation on the part of the customer to pay the company. Revenue is recognized when the service is actually provided or ratably over the life of the subscription agreement. How and when revenue is recognized is governed by GAAP.
Marketing Metrics
App Installs - The number of users who installed the application for the first time in the selected period. An essential supplement to the New users metric for mobile products that require installation (mobile products and some desktop products). Do not use it as a replacement for New users: installation does not imply the use of the product.
App store rating Average app rating in the app store. Usually given in a week, month, or total product life.
ROI: Return of Investment - Always expressed as a percentage (e.g., 30%, not -0.3), it can apply to any form of investment in a product and its return. ROI =
CPA: Cost Per Acquisition — measures and determines the cost the advertiser must incur for one action the user performs. CPA = total costs (marketing costs) spent / the number of new customers in the same period.
LTV: Lifetime Value — Total revenue generated on average by a single user since the product was first used. LTV = Lifetime Customer Revenue — Lifetime Customer Costs
Revenue per customer (per month) = average order value multiplied by the number of orders.
Contribution margin per customer (per month) = revenue from customer minus variable costs associated with a customer. Variable costs include selling, administrative and any operational costs associated with serving the customer.
Avg. life span of customer (in months) = 1 / by your monthly churn.
LTV = Contribution margin from customer multiplied by the average lifespan of customer.
Financial Metrics
For design cases, I tend to stay away from these as your focus is typically on user engagement and product-market fit based on usage that indicates you are solving that “job to be done.” But if you are interviewing at a start-up, these might come up.
Annual Recurring Revenue (ARR) - think subscriptions and contracts - is a measure of revenue components that are recurring in nature. ARR = Average Annual Profit / Average Investment
ARR per customer - Is this flat or growing? If you are upselling or cross-selling your customers, then it should be growing, which is a positive indicator for a healthy business.
Monthly Recurring Revenue (MRR) - This should be calculated monthly. Some companies will multiply one month’s all-in bookings by 12 to get to ARR (I do not advise it). MRR = a number of customers * average amount of active subscriptions. The metric is helpful in SaaS businesses or other businesses with a subscription model. They are used interchangeably with the ARR.
Gross revenue: (before deducting VAT, other taxes, and fees) was generated by the product in the selected period. The formulas: - gross revenue = (number of goods sold) x (price per item) - gross revenue = (number of customers) x (price of service)give you the gross revenues for both product-based and service-based income.
Gross Profit: in general all costs associated with the manufacturing, delivery, and support of a product/service should be included.
Conversion rate: Conversion rate = Paying users / Active users.
ARPU / ARPA - Average Revenue per User / Account. Usually counted over the last month (28 or 30 days). Monthly Revenue or MRR, shared by active users or accounts for that month. ARPU = Revenue Metric / Active Users, ARPA = Revenue Metric / Active Accounts.
ARPPU / ARPPA: Average Revenue per Paying User / Account. Usually counted in a month. Monthly Revenue or MRR, shared by paying users or accounts that month. ARPPU = Preferred Revenue Metric / Paying Users, ARPPA = Preferred Revenue Metric / Paying Accounts.
Incremental Monthly Recurring Revenue (iMRR) — Counted year-on-year, quarter-on-quarter, and month-on-month. A negative value indicates a shrinkage of the product, a positive value indicates an increase in the product.
Contract Metrics
Total Contract Value (TCV) is the total value of the contract, and can be shorter or longer in duration. Make sure TCV also includes the value from one-time charges, professional service fees, and recurring charges.
Annual Contract Value (ACV), on the other hand, measures the value of the contract over a 12-month period. Questions to ask about ACV:
Gross Merchandise Value (GMV) vs. Revenue
In marketplace businesses, these are frequently used interchangeably. But GMV does not equal revenue!
GMV (gross merchandise volume) is the total sales dollar volume of merchandise transacting through the marketplace in a specific period. It’s the real top line, what the consumer side of the marketplace is spending. It is a useful measure of the size of the marketplace and can be useful as a “current run rate” measure based on annualizing the most recent month or quarter.
Revenue is the portion of GMV that the marketplace “takes”. Revenue consists of the various fees that the marketplace gets for providing its services; most typically these are transaction fees based on GMV successfully transacted on the marketplace, but can also include ad revenue, sponsorships, etc. These fees are usually a fraction of GMV.
Resources:
UX Planet - Product Metrics - A Complete List
A16Z - 16 Startup Metrics
