Grading Advice No. 7: Product Design/Sense
Perhaps the most intimidating of PM interviews, the product design or product sense cases.
Unless you do it all the time, you’re not gonna be able to grade on everything. I advise that your grade on five sections.
I’ll give you a framework you can use that has links to help candidates who do poorly. They can go read and learn how to improve so you don’t have to worry about giving too many specifics on what they should do to improve. Tell them to read the links from the coach.
The Five Sections
Clarifying Questions. Even if you’re not comfortable with the questions, you can help by telling them if the questions where:
To fishy or begging for help.
Too many questions.
Irrelevant questions
Robotic questions
Strategic Setup. The candidate should quickly state observations on why it’s an important problem to solve. And/or what makes this problem interesting or unique. It is okay for the candidate to skip this section if they want. But if they do, they need to show strategic insights throughout the body of their answer. Things to look for:
Do they Identify the aha moment or the insight that makes this an interesting prompt? (Typically, if they don’t identify it before they get into users they’re not going to get come up with a good Solution.)
Users. Do they come up with at least three segments? (There are some exceptions but most of the time they need to show considering and eliminating other user segments.) Common problems include:
Pure, demographic groupings
MECE for the sake of MECE. Not really thinking about what makes sense. (Typically, what you’ll see there is trying to make the segments so distinctly different that there’s is no way to compare and contrast to prioritize.)
Segments that are too broad. Think: kids vs adults.
Occasionally, someone will get too hyper-specific. It will sound ridiculous but that’s a rarity.
Solutions. There are two methodologies: one big solution with three features OR three distinct solutions. (Unless the interviewer says feature, technically speaking neither is better than the other.) That said, ommon problems include:
Not being creative
Being too creative when the interviewer wants you to be iterative (Think: Meta)
Only giving one solution with one feature, and nothing to prioritize
Not focusing on the pain point they said they would focus on
The exception is when they acknowledge the ideas they came up with are better for another pain point. But they still need to solve for the pain point they selected if the interviewer is picky.
Metrics for Success. In most cases, they need to list three metrics and pick a Northstar. Common problems include:
Not setting a goal before solutions
Not aligning to the goal they stated
Acquisition or Engagement stated as metrics but no measurement guidelines given (for example saying engagement instead of # of users reading at least one article per session)
Links to Help
If the candidate you are helping is struggling with design questions, recommend the following articles:
