4 Product Sense/Design Formats Evaluated

TL;DR - Among tech companies that rely on product design case questions, Meta removes the most bias by asking you to think about similar products. They typically select products (or concepts) that are far enough removed from their existing products to remove their own bias but not so abstract so as to throw you off guard completely while also testing your thought process on concepts important to being successful in their organization (think: marketplace dynamics, building community, etc.). Other interviewers should learn from their example.

Caveat: This is not to say that product design/sense questions are the best way to assess candidates. But within the genre of questions, related but not biases by your deep knowledge of the product are best.

Background

Most product interviews, particularly for individual contributor roles, involve a question/prompt that is used to try and get a read on your product sense/design skills. They are testing to see if you know how to think about different users, prioritize one, identify pain points, and come up with some solutions. The way they set these interviews up is with a prompt. Today, I am going to go through the design/product sense prompt subtypes and walk through their pros and cons.

Relevance Note
If you are a candidate, knowing will help you keep an eye out for what is coming. If you are a hiring manager, consider the bias’ addressed in the different prompts discussed below.

Four Types of Design Prompts

There are four general types of Product Sense/Product Design prompts. They involve a product:

1. the candidate knows well

2. the company builds

3. in problem space similar to the company

4. wholly unrelated to the job at hand

Now I will dive into each prompt in detail. It will quickly become apparent that the third item on the list is my favorite because it balances testing relevant skills while attempting to remove a large amount of bias is some of the other types.

1. Candidate Knows

   • Example: What is your favorite product? How would you improve it?

   • Pros: The candidate knows the space and can speak with confidence. When we join a product team, we learn the space and know a lot before we start trying to think about user problems and solve for them.

   • Cons: People can practice, and those without real product sense can game the system by getting help polishing their responses.

   • Food For Thought: This is a good question for junior PMs, if they practice, you know they care and they will have learned to think about products critically, a great skill to have. If the candidate is senior and can’t nail this prompt, this is an obvious fail but probably doesn't help you judge the stronger or more well-practiced candidates.

   • Companies Know for This Prompt: Google, mid-tier and startups.

2. Company Builds

   • Example: Interviewing for Yelp Search Ads. Look at this part of our Search experience, develop a new ad format for it.

   • Pros: This is a decent question if a large part of what the interviewer wants to test is their experience in the specific type of search space or their passion for the role such that they thought about the space deeply.

   • Cons: It is nearly impossible for the interviewer to remove their bias as they know the problem inside and out. Different candidates with different experiences will bring different things to the table. It puts good generalists at a severe disadvantage.

   • Worrying Trends: Less experienced or impatient interviewers tend to find it difficult to let the candidate lead the interview because they are looking for a checklist of things that are top of mind rather than seeing how the candidate thinks. (Not a rule on this last one, but very common in my experience.) This question type is often asked by companies that blend it with behavioral, and so the time allotted to the prompt is limited. Many candidates can’t get a good read on how much they should lead and how much they should ask for the context the interviewer might be hiding unless they ask for it, which feeds the impatience listed above.

   • Food for Thought: If you ask this question, consider giving the candidate a heads up that they can expect a design case related to the product and space. As part of the interview invite, suggest they think about the challenges the company/product/line of business faces. 9 out of 10 times, I have found this is what interviewers who like this prompt are looking for and testing.

   • Companies Know for This Prompt: Mid-Tier companies without standardized process and Startups

3. Similar Problem Space

   • Pros: You can test for the key elements of what the candidate needs to do in your company without dragging in all your unintentional biases from being immersed in the space.

   • Cons: Generally, you might grab a topic the candidate knows well or poorly, advantaging some and disadvantaging others randomly. This is still a false setup; most people need to practice to do well.

   • Food for Thought: This is a reasonable approach, but after a while, to avoid people over-practicing questions, it becomes difficult to come up with new prompts, and interviewers start to drive to more abstract questions. It is better to have a commonly practiced prompt than one that gets too off the wall.

   • Companies Know for This Prompt: Meta, Stripe, Marketplaces in general, Google

   • Note: While I praise Meta for this approach, many others improve on the general concept by getting laser-focused on a smaller group of competitors or adjacent concepts, and it is more consistent.

4. Wholly Unrelated

   • Pros: You are testing creative thinking and abstract understanding of a fundamental framework or approach to product design.

   • Cons: These questions completely throw the candidate off. This means it doesn’t really test the vast majority of candidates appropriately because they are soo thrown off as to feel lost, and by the time they get their bearings, it is too late to win.

   • Food for Thought: How do you balance the shock factor with the ultimate goal? Are you really trying to judge their product sense reasonably? Or are you a fan of an old concept that has more downsides than upsides?

   • Companies Know for This Prompt: Google (think teleportation device), Startups, Mid-Tier companies that let ex-Googlers do what they want).

Hopefully, this has helped you prepare or think through your interview techniques. If you are a hiring manager who likes the example from their product line or the wildly random question, please reconsider the pitfalls and biases inherent in those questions. Or provide a little more direction in your setup so that the candidate can drive the conversation without fear of falling into a trap.

Note: Meta is not perfect and a little too checklist driven at times. They also have the problem mentioned above of being too prone to the luck of the draw with regard to individual experience with a product or space. Still, the point here is that their prompts do a decent job of testing product sense with a significantly reduced number of unknown biases relative to the other question types.