Grading Rubric for Product Design
The following are elements I grade on during a Product Design and Insights case interview.
Clarifying
Goal: Does this person ask the right questions to get what they need to answer the question. Do they show curiosity, clarity and insights.
Common Mistakes
Frameworky: Asking from a list that is clearly memorized and not focused on the prompt.
Wrong Questions: Does the candidate ask about the fundamental issues that will help them make good decisions as they go along.
Compound Questions: Do they ask three questions in one and expect me to answer them all?
Fishy vs Strategic Questions: Are they basically begging me to give them the answers?
The Key Elements
Framework Does the candidate share the framework upfront? Is it clear and concise?
Insights - Why is this an important space to consider now? I ask myself, am I hearing interesting observations on the space or the problem? Does the candidate focus on user insights or just business insights? Can the candidate identify a problem to solve?
Users - Can the candidate handle the ambiguity of identifying a target audience? Can the candidate empathize with users? Are the groupings too focused on demographics? Are the user groupings ones that enable the candidate to show an ability to prioritize between more than two options? Is prioritization clear?
Users Journey - I don’t always need to hear a strongly structured user journey. But the candidate does need to show that they are empathizing with the current solutions (or lack thereof) and the user’s POV with these solutions.
User Prioritization - The candidate should provide an easy-to-understand rationale for why they are picking one user over the other as well as why they rejected an initial focus on the other users. I am looking to see if the candidate can pick a user out of the three and not get hung up. (Ambiguity grading coming back.)
Solutions - Here I look for clarity of thought and creativity with user empathy throughout. I prefer low, medium and high complexity/creativity so show your ability to solution now and into the future.
Success Metrics - The most important point here is being able to pick metrics that align with a stated goal. I am also looking for the three parts of a metric as well as something at requires product logs that show the candidate can work with engineering teams to figure out what and how to log data.
Communication - I am looking for clear communication. The problems and habits that frequently fail people are: mumbling, speaking too quickly, not using lists of three, not having a point, etc. It boils down to not knowing what you want the interviewer to put in their notes.
The Grading Scale
Some companies use a 1-to-5 grading scale, but more commonly it is a 4 or 6-point scale. No matter the scale, the general ideas are the same. How strongly or weakly do I believe the person should be hired.
Strong Hire
Hire
Leaning Hire
Leaning No Hire
No Hire
Strong No Hire
Common Criteria by Grade
Strong Hire - They impress on the why, user, and solutions with strong metrics.
Hire - They have strong user empathy and solutions but might be weak on the why or metrics.
Leaning Hire - They probably needed some nudging and/or had weak communication skills but the fundamentals where there.
Leaning No Hire - Something major was missing or communication was impossible to wade through at some point during the interview. They probably needed nudging or made major mistakes without the nudging.
No Hire - Were polite but missed the mark on all (or most) key elements. I couldn’t imagine working with them day-to-day. I might say No Hire at the level being asked but Leaning No Hire/Leaning Hire at the level just below.
Strong No Hire - Typically were arrogant or rude in their way of communicating and completely missed the mark. They may have come across as clueless.
