Observations on Home Assignments
This is part of a series on the out-of-control expansion of take home assignments that is hurting product managers and the teams that insist on them.
Like all good product leaders, I always make decisions with imperfect information. As I learn more, I make better decisions. In this case, I used to support take-home assignments. After a months watching exceptional clients being denied roles they would have knocked out of the park, I began self-reflecting on why I support them and the flaws in my assumptions. I no longer agree that take home assignments (outside of the two-page storytelling ask of Amazon) are worth the time for the candidate or hiring manager.
In this article, I will walk you through why I created a take home requirement when interviewing product managers and what I learned from it. I will reflect on current observations that have changed by mind.
TL;DR - The Take Home Assignment reflects more on the team doing the hiring than the candidates in the hiring process.
Why I Asked for Take-Home Assignments
As a product leader looking to hire a team, I once created a take-home requirement for three reasons:
Loss of Trust from Bad Hire
Disagreement With Engineering
Willingness to Hire the Inexperienced
Let me walk you through those reasons.
Loss of Trust from a Bad Hire I once agreed to hire a product manager who displayed red flags. (Read more here.) Ultimately, the responsibility of hiring the wrong PM was mine. But everyone involved no longer trusted the others to make the right decision on the next hire. The compromise: a take-home assignment that could be uniform and traceable.
Disagreement with Engineering After the bad hire, it became clear that product and engineering were grading the PM candidates differently. The take-home assignment put a bit of control in the process. We could document comments and trace differences and inconsistencies in feedback.
Willingness to Hire the Inexperienced My engineering partner basically hated product and was sharing that POV at every chance they got. With each candidate, it became more obvious that anyone who learned product elsewhere wasn’t going to work for them. The compromise: find someone with talent who would agree to their definition of what product should do. At this point, it was about getting someone who could bring harmony to the team. It was a case of a startup that needed one PM, not a huge team of PMs and my responsibility was to find someone who would fit the bill given the dynamics of the existing team. A writing sample was the easiest way to do this.
Anyone who reads the above list should realize that the take-home had much more to do with organizational dynamics of the company than the prospective PM hires. It is fair to ask how often this is the case.
Pattern Matching
I would say the above list is a collection of the extreme. But the themes are common. And after debriefing with a number of candidates recently, it is clear from the questions from different players in the room during many take-home assignments that the grading is not uniform and the reasons the take-home is leveraged are not uniform within any individual company.
In the case I referenced above, when it came time to grade the results, none of the disagreements on hiring changed. The signs of potential were still seen differently by different stakeholders. And so the candidates were subjected to a take-home assignment they slaved over just to have the decision-making criteria be no different, just more traceable.
Now, many advocates of the take-home assignment will claim this is an extreme that doesn't apply to them. That may be the case, but if, after being a bit (naturally) defensive, they look at how people are really graded, can they truly say the take-home is giving the average candidate a better chance to shine? Or is it about helping their organization get on the same page leveraging a piece of work?
Trends and Patterns
Don’t Be the First The first candidate to answer a take-home always gets rejected.
The team still doesn’t know what they want or how to set the candidates up for success and fair evaluation.
Team dynamics determine what is valued (think: execution over strategy).
For the same PM leader, a different engineering partner can mean a different candidate wins.
A product leader who started in engineering and just moved into product is going to look for more execution than strategy, for example. How doest the candidate know to account for this?
Always Takes More People spend 5x to 10x the time estimated by the hiring team. This is a reality and unfair. When the environment gets better, more people will reject take homes.
Reviewed Presentation Most people get others to review their presentation even when the rules say do the work on your own.
But outside of telling people their slides are too busy or they are reading off their slides, it is nearly impossible for people to give advice on what the team really wants.
Not Like Real Life At All
The advocates of the take-home claim it is a great way to test people’s real skills. But that is so far from the truth. Even the top leaders of Amazon who are skilled at writing 6-pagers write their documents and get team feedback before sharing with leadership. And the first version is almost never what wins the argument.
To Hiring Managers Is the take home really about the candidates or about your inability to spot talent with our flawed interview process or get your team on the same page about what you are looking for in your next hire? Yes, the document means you are baselining on the same things, but all those biases exist and groupthink kicks in. One person says something negative, and others have their POVs changed. It is just as flawed as traditional methods.
To Candidates In today’s job market, you don’t have a choice, you have to complete the take-home assignments when asked. But don’t beat yourself up about how it goes. You would never prepare a document for executive review without a ton of reviews from people who truly know what the company culture and team needs. And when they don’t, your leader shares the burden of meeting unexpected expectations. Please, please, please don’t beat yourself up. Take away one learning, brush yourself off and move forward.
Lessons from My Last Take Home:
You come with a different perspective and lack context no matter how the take-home prompt is written or explained.
Presenting with images vs. without is a totally different game.
We need to over-explain when there are no visuals or written documents.
With visuals, you need to change your storytelling techniques.
Don’t read from your slides.
The team you are presenting to is looking for criteria you were never told about.
Sometimes they don’t know what they are looking for until they hear your presentation.
The criteria of grading changes based on the pool of applicants, and that is outside your control.
