Design Metric Examples by Product Type
After describing a design solution, candidates will get asked, “What are some metrics you would use to measure success?
Ask a PM interview candidate this question and you often get a “deer-in-the-headlight” response. Included in the typical gut-initiated responses are:
Vanity Metrics - DAU or MAUs without explaining how “Active” is defined
Too High-Level - Engagement, Activation, etc.
Retention - List this without a proxy metric and you sound clueless
When answering success metrics during a design case, it is important to point out what will measure your ultimate goal as stated earlier in the discussion. But it is also important, if the two are not the same, to point out activation metrics.
Why Activation Metrics:
Proxy for Retention
Aha Moment Identification
Success Requirements
Lucky for you, Lenny Rachitsky and Yuriy Timen did a study on activation metrics.
Highlights to Help you during Interviews
Qualities of Activation Metrics
Predictive
Actionable
Groupings by Company Focus
B2C Subscription (e.g. Duolingo, Calm)
B2C Free/Ads Supported (e.g. Instagram, Snap)
B2C Marketplace (e.g. Airbnb, eBay)
B2B SaaS (e.g. Salesforce, Figma, Slack)
B2B Marketplace (e.g. Faire)
eCommerce (e.g. Chewy, Wish)
DTC Subscription (e.g. Seeds, Lumen)
Common Problems
The following problems Lenny and team identified with activation metrics come up time and again for PM interview candidates.
Too early
Simply completing the sign-up flow is not enough.
Unlikely to show a user the value of your product.
Unlikely to be predictive of long-term retention.
Too late:
Marketplace or eCommerce: Multiple purchases
Leading indicator of habit, not prove the habit
Multiple purchases is too much to ask for activation
Not predictive:
Not highly predictive
You want (i.e. at least 2x) of long-term retention and/or monetization.
Not actionable:
Not easily actionable, measurable, or hard to experiment
e.g. using an offline event that digital growth teams can’t impact
e.g. takes three months to get any data
Too complicated:
Selecting a multi-stage activation event with:
multiple different user actions
along with a time bound
if you can do it with a simpler metric, do so.
A Few Examples
First day of food and weight logged
First 5 survey responses collected
First design published and shared
First video created that received a view
First listen to more than 40% of a track
First listen to 3+ sessions in the first week
First time booked listing
First moved meeting
First job application sent
10 posts published in first 2 weeks
An Exercise
Try to guess a few of the metrics before peeking. Pull out an old-fashion pen and paper.
-
First piece of content created
First upvote on a post
First piece of text highlighted
First piece of content created
Shared their location
First piece of content shared
Booked first AI manager
-
First Ride
First Trade
First completed gig
First job application sent
First credit approval
First time booked listing
First transaction
First purchase
First car license plate checked
Three favorite items on the marketplace
-
First meditation session completed
First workout
First meal logged
First listen to more than 40% of a track
First plant added
First drawing created
Completed a relationship self-assessment
attended first live math class
First workout completed
Lined a financial institution
-
First booking
First order
First purchase
First Transaction
First order delivered and not returned
Two orders in one month
Amazon account connection
First post read in our social commerce app
-
Third order placed
Second therapy session
First meal logged
App download
First purchase
First watch
-
First order
First purchase
First completed job on the platform
Two orders on two separate days
2+ bookings within seven-day time frame
First active offer
First hire
-
First document sent
First design exported
First meeting scheduled
First money deposited
First course enrollment
First QR code generated
10 posts published in first 2 weeks
Two users in workspace edited a doc
Three accounting documents created
Two recorded videos
-
First virtual event created
First message sent through our network
First reconciliation completed
First meeting moved
First starting of a lesson
First API called in production
First automation built
First loan application submitted
Fixed vulnerability in first 30 days
First 5 survey responses collected
Resources:
