Design Metric Examples by Product Type
After describing a design solution, candidates will get asked, “What are some metrics you would use to measure success?
Ask a PM interview candidate this question and you often get a “deer-in-the-headlight” response. Included in the typical gut-initiated responses are:
Vanity Metrics - DAU or MAUs without explaining how “Active” is defined
Too High-Level - Engagement, Activation, etc.
Retention - List this without a proxy metric and you sound clueless
When answering success metrics during a design case, it is important to point out what will measure your ultimate goal as stated earlier in the discussion. But it is also important, if the two are not the same, to point out activation metrics.
Why Activation Metrics:
Proxy for Retention
Aha Moment Identification
Success Requirements
Lucky for you, Lenny Rachitsky and Yuriy Timen did a study on activation metrics.
Highlights to Help you during Interviews
Qualities of Activation Metrics
Predictive
Actionable
Groupings by Company Focus
B2C Subscription (e.g. Duolingo, Calm)
B2C Free/Ads Supported (e.g. Instagram, Snap)
B2C Marketplace (e.g. Airbnb, eBay)
B2B SaaS (e.g. Salesforce, Figma, Slack)
B2B Marketplace (e.g. Faire)
eCommerce (e.g. Chewy, Wish)
DTC Subscription (e.g. Seeds, Lumen)
Common Problems
The following problems Lenny and team identified with activation metrics come up time and again for PM interview candidates.
Too early
Simply completing the sign-up flow is not enough.
Unlikely to show a user the value of your product.
Unlikely to be predictive of long-term retention.
Too late:
Marketplace or eCommerce: Multiple purchases
Leading indicator of habit, not prove the habit
Multiple purchases is too much to ask for activation
Not predictive:
Not highly predictive
You want (i.e. at least 2x) of long-term retention and/or monetization.
Not actionable:
Not easily actionable, measurable, or hard to experiment
e.g. using an offline event that digital growth teams can’t impact
e.g. takes three months to get any data
Too complicated:
Selecting a multi-stage activation event with:
multiple different user actions
along with a time bound
if you can do it with a simpler metric, do so.
A Few Examples
First day of food and weight logged
First 5 survey responses collected
First design published and shared
First video created that received a view
First listen to more than 40% of a track
First listen to 3+ sessions in the first week
First time booked listing
First moved meeting
First job application sent
10 posts published in first 2 weeks
An Exercise
Try to guess a few of the metrics before peeking. Pull out an old-fashion pen and paper.
Resources: