Technical Interview Grading

TL;DR - In May of 2022, Google changed how it handles technical interviews for PMs. Right now, things are in flux. If you are really worried about the technical interview, I still recommend Lewis Lin’s course for Tech Interview Prep for PMs.

That said, in this article I take a high-level approach telling you what they are looking for, not necessarily how to study. The idea being, if you get stuck and feel overwhelmed, remember the main point of the interview is to see how you would work with engineering.

Functional vs Non-Functional Requirements

If you can’t dive into the details, can you at least distinguish functional (what the system does) from non-functional (how the system does it) to help determine what decisions you need to make as a PM to keep the team moving forward and making the right prioritization decisions.

Your interviewer wants to know can you prioritize functional vs non-functional requirements appropriately to focus the team on the highest priorities. Non-functional requirements describe how the system works, while functional requirements describe what the system should do.

A functional requirement is any requirement which specifies What the system should do.

• Business Rules

• Transaction corrections, adjustments and cancellations

• Administrative functions

• Authorization levels

• External Interfaces

• Reporting Requirements

• Legal or Regulatory Requirements

A non-functional requirement is any requirement that specifies How the system performs a certain function.”

• Performance – for example Response Time, Throughput, Utilization, Static Volumetric

• Scalability

• Capacity

• Availability

• Reliability

• Data Integrity

• Interoperability

5 Technical Concepts

You can demonstrate command of these technical 5 tech concepts in the 'tell me about yourself' as well as the meat of the discussion. The interviewer is looking to grade you on a continuum on each concept.  

Baseline vs Excellent Answers

• The Technical Problem

  • Baseline: Able to explain a technical problem

  • Excellent: Identifies key points in complex topics (looking for excellence in explaining the core problem) Think : The 'aha' moment

• Systems

  • Baseline: Understands technical systems

  • Excellent: Can architect a product

• Impact

  • Baseline: Understands impact on eng team

  • Excellent: Makes the right technical trade-offs

• PM/Eng Partnership

  • Baseline: Understands how to work with eng

  • Excellent: Models and leads PM/Eng partnership

• Roadmap

  • Baseline: Can design a feature

  • Excellent: Builds technical roadmap & strategy

Knowledge & Intuition 

The eng interviewers are coached to balance knowledge and tuition judgments. Even someone with very little technical background can have great technical intuition. Intuition is based on knowledge that is developed based on past experience. Knowledge is the basis for good instincts that all one to logically derive solutions to problems..

• Knowledge is specific: "explain what X is"1

• Intuition is a candidate's ability to arrive close to the right answer based on related knowledge, past experience, and "gut feel". Candidates with good intuition just "get it." Even when guessing or discussing an unfamiliar topic, they gravitate toward the good answers.

• The ideal candidate is strong in both knowledge and intuition, but many good candidates will have more of one than the other. Interviewers should allow for both types of candidates to do well.

Grading Overview by Level 

All levels should be able to explain the technical problem to a non-technical person.

L3 (APM)

Demonstrates ability to work through technical considerations and identify impact of product proposals. Can Outlines effective ways to experiment with product proposals.

L4

Demonstrates understanding of how technical systems work. Can map between technical implementation & product constraints. Understands how to implement experiments

L5

Knows how tech systems work eg. client-server, hashing, timestamps. Can trade off between tech debt and time to market, experiment in complex domain to inform strategy, & convey tech aspects of their products & tech's impact on product.Understands impact of changes in plan to eng team

L6

Describes product's system architecture & how that impacts product decisions. Can experiment in complex domain to inform strategy. Accounts for impact of changes in plan to eng team. Works well with eng to solve product issues w/tech constraints.

L7

Knows how to architect a product. Gives model for good eng partnership. Builds architecture decision points into roadmap to minimize impact to eng, uses existing tech arch. to solve issues, knows limitations of different tech, and can experiment in complex domain to inform strategy.

The Questions & Interviewers

The questions should be multi-layered. There is no one right answer. If you get an inexperienced interviewer, reasons not to worry, a good HC (hiring committee will be looking for red flags). 

Common feedback pitfalls in eng feedback. If the engineer focuses feedback on the wrong areas the hiring committee will push back: 

1. Focused on the wrong areas

2. Stuck on one “right” answer

3. No evidence of skills feedback

4. Contradictions and ambiguity

5. Too focused on petty minor details 

So, if you hit it out of the park with product sense and you get a bad eng interview, you might get a shot at redoing the tech interview. Don't bank on it, but it is always a possibility.  

Additional Reading:

Function vs Non-Funcational Requirements