Product Metrics: Self-Study Tool

This is meant as a quick exercise to help you start thinking about product metrics in a more structured manner.

The best way to use this is to grab a piece of paper, or blank document, and try to list 2 to 4 metrics for each category. Don’t list just the name, also describe how it would be calculated (Three Parts of a Metric).

After writing or logging your answers, take a look. (If you feel strongly one is missing, feel free to ping me at intrico.io@gmail.com. It is still a WIP and I am trying to balance brevity with sufficient info to prep.)

If you are not focused on SaaS products, you can skip that one but, if you ask me, it is worth the thought exercise.

• Acquisition

  • Trial-Customer Conversion Rate: # of paying customers / # of trial users (or a given cohort) = % customers converted

  • Retention Rate: # users or customers in period 2 / # of users of customers in period 1 = % retention

  • Churn Rate: # of users or customers lost in period 2 / # of users of customers in period 1 = % churn

• Engagement

  • DAU, WAU, MAU - sum of all users taking action in time period (Action must be defined beyond opening product)

  • Stickiness/Churn: Daily Active Users (DAU) / Monthly Active Users (MAU) = Stickiness Ratio

  • Cohort Retention: Best measured with curve visualization or retention chart (spreadsheet typically with heat map coloring)

  • Time in App: Average time (mins, hours etc) per time period (session, day, week, etc.)

  • Number of User Actions / Session: count of actions per session (or day, week, etc. depending on product)

• Feature Usage

  • Feature Adoption: # of X actions taken per day (per feature)

  • Average time using feature per session: total time spent in product (feature, etc.) /number of engaged users

  • In-Product Feedback Response Rates: # of survey responses / # of surveys delivered or shown = feedback response %

• Search & Discovery

  • # of clicks before spending at least 30 seconds on a listing

  • # of clicks on recommended content per session

  • exit rate - # of searches without a click

  • bounce rate of search results- # of search results pages abandoned without a click in under 30 seconds (some times the summary might be good enough, so click isn’t always required - think Google summary results)

  • exit vs bounce rate - exit rate/bounce reate

  • null result rate - # of searches without a result/all searches per session (or all searches for X time period)

  • Page Visit Time - Average time page visit length from search results page

  • Converstion rate - # of seach visits resulting in a click + 30+ second stay on a page

• Network Effects

  • Organic vs Paid Users:

  • # of users signing up after clicking on ad

  • % of users signing up organically/all new signups

  • Traffic Sources: Internal vs External: # of visits from organic requests/total number of visits

  • Virality: # of users signing up after a referral per month (link info or survey to get data)

  • Multi-Tenanting: % of users also using competitor

  • User Retention Cohorts: MAU Cohort B-Cohort A. Looking for an increase in older cohorts.

  • Overall User Count

  • Core Action Count

  • Paid Users

  • By Location

  • Power User Curves (L7 & L30 Charts): Histograms of User Engagement showing the total number of days users were active in doing a particular action in a given timeframe

  • Match Rate: Vary based on product. Here are some examples:

  • % of the time drivers have a percentage per shift/day/week

  • # of jobs filled/total jobs listed

  • average # of hours frealance works per week

  • Market Depth: Number of items avaliable (again marketplace dependent)

  • # of homes avaliable per search

  • # of drivers avaliable per search per location

  • Time to Find a Match (think search type metrics for how quickly the search is relevant)

  • Average Time to find a match per search/day/month

  • Concentration of Supply - % of GMV the top X sellers or buyers account for 

• Satisfaction

  Note: I hate most of these metrics for product case interviews. I want to see you talk about what you find in the logs.

  • NPS: Involves collecting responses

  • CSAT: Involves collecting responses (# positive responses / # total responses X 100 = CSAT %

  • Support Tickets: # of support tickets created per X time period

• SaaS Metrics

  • MRR - Monthly Recurring Revenue : # of customers * average monthly contract value

  • LTV - Lifetime Value: annual contract value * average customer lifetime (in years) = customer lifetime value

  • CAC (Cost of Customer Aquisition): Marketing cost + sales cost + new customer support cost = Customer Acquisition Cost

  • ARPU (Average Revenue per User) - Total annual recurring revenue / total # of paying customers = ARPU

  • Cash Burn Rate: All period cash in – all period cash out = cash burn rate

• Notes: Time Period

  Most candidates forget to list the time period. And worse, if poked they aren’t sure why week over day, etc.

  The general rule of thumb here is that if 60% of your users leverage your product daily, it is a daily product and you should measure as such.

  That is a general statement, daily usage products also have a number of valid metrics that are measured at the weekly and monthly time period.

  The important thing to remember is that you need to have a POV.

Resources:

19 Product Management Metrics and KPIs to Track in 2022

16 Ways to Measure Network Effects