Hiring Product Managers at Scale

In this post, I summarize a process that I recommend for hiring product managers at a midsize or growth company, adapted for a distributed hiring environment (most applicable to a company that will hire multiple product managers).

I’ve hired and trained over 40 product managers over the course of my career, and this draws on my experience as a product hiring manager and team lead.

Internal buy in and scope of role

When hiring product managers (PMs) at a mid size company the most important thing is to have internal support from the executives and the design and engineering partners. There should be a strong desire to hire PMs to help build better products in a better way and to bring in more structure and systems to the product development process.

Once there is buy-in from these stakeholders, organize the teams into sensible working groups (e.g. by user journey such as onboarding/growth or by key metrics such as conversion/retention or by product line).

I prefer a matrix structure (although has tradeoffs) where PMs ‘own’ each of these areas in partnership with a design and engineering lead (with around 5-10 engineers per PM, depending on the project). I also suggest that engineers and designers report into their own functional leads and PMs direct the scope and priorities of the projects.

Hiring process

It’s essential to have a clear hiring process and system both for the sake of your internal team and for the candidates. Most companies are incredibly disorganized about hiring, but a little bit of work can save a lot of time in the future, especially when hiring many folks for the same role.

Internal Team 

  • Recruiter: There should be a consistent point of contact for the candidate during their application process – ideally a recruiter. The recruiter communicates with the candidate, lays out the hiring process clearly, and moves them through the process. They act as a liaison between the hiring manager(s) and the candidate. They often do the initial resume screens and have an essential input into hiring because they get to know the candidate so well. 
  • Hiring Manager: The hiring manager is the person that is hiring for the role. They are the person who ultimately makes the decision to recommend the candidate as a ‘hire’ or ’no-hire’. This is typically a senior product leader.
  • Interviewers: Each interviewer should have a clear set of criteria that they use to evaluate the candidate. The interviewers should be excellent at the functional areas that they are evaluating candidates and hold the quality standard for the organization. The best people should be involved in late-stage interviews and this should be a core part of their job description.

Hiring Process

  • Resume screen: Internal and external candidates should submit a Resume / LinkedIn profile which should be screened upfront (recruiter + hiring manager). Candidates who pass this phase should move to a conversation with the recruiter, followed by the hiring manager.
  • Interviews: Interviews should consist of a standard set of, very well calibrated questions that can be asked by a variety of interviewers representing the different development functions (e.g. design, engineering, product, marketing). A structured hiring guide improves consistency and calibration, and can reduce bias from the hiring process
  • Central Tool/ATS: Interview feedback should be stored in a central place/tool (e.g. Greenhouse or Lever) and each interviewer’s feedback captured clearly (with a hiring recommendation). This allows us to both evaluate interviewers and the candidates – e.g. some interviewers bias towards higher or lower scores.
  • Written Exercise: If you are hiring in a distributed environment, try to find candidates with strong communication skills (particularly written skills) and clarity of thought. All candidates should complete a written exercise as part of their recruitment process which could include:
    • Break down a product you love – what you like, what you don’t like, how you would make it better (1 page)?
    • What is your favorite technological shift and why?
    • Write a ‘product spec’ to address a specific problem that the company has (better if it is a real problem).
  • Trial: If possible, ask the candidate if they would be open to a two-way trial (which is compensated) where they try and solve a real problem and collaborate with an internal team. This is time consuming (20-40hrs for the candidate, 5-10 hours internally) so very few candidates should go through this process if you decide to incorporate trials. You may filter out some good candidates because of the time commitment, but candidates who join are more likely to be successful.
  • References: I think that final candidates should be referenced checked by the hiring manager, especially if there are open questions. Backchannel references are the best (but avoid people at their current company) otherwise, ask the candidate for references. Here are some questions that I like:
    • How do you know the person? (gauge depth of relationship)
    • What are their strengths?
    • What are their areas for development?
    • What percentile would you put them in relative to similar folks in their position?
    • Would you hire them again?
  • Decision: For borderline candidates, the panel of interviewers should have a sync or async discussion – e.g. a private recruiting slack channel for hiring. The hiring manager is ultimately the decision maker. From start to finish, try and keep this process fast (e.g. under one month, and track the throughput).

Candidate experience

Candidates should have a great experience, understand how they are being evaluated and have consistent clear communication through the process.

  • Hiring Criteria: Candidates should understand the criteria by which they are being evaluated and the steps in your hiring process – this should be a templated email or a public blog post that you can send to product candidates.
  • Point of Contact: Candidates should have a clear point of contact (ideally the recruiter), to ask any questions about timelines and next steps.
  • Acceleration: If a candidate performs very well in early interviews or comes in through a trusted referral, they should be bumped up to the top of the queue or potentially skip steps so you don’t lose great people because of slow process.

How to Assess

When hiring, it’s important to be explicit about the skills you are looking for, and get a sense for where candidates are truly exceptional.

Here are the dimensions that I think you should use to assess candidates in the interview process:

  • Analytical Ability: AB Testing, Interpreting metrics, Data-informed decision making.
  • Product Judgment: System design, UX design to solve user / business problems.
  • Leadership: Inspiration, Influence, Empathy, Communication.
  • Execution: Prioritization, Getting things done when you say you will.
  • Technical Ability: Earn trust and respect from engineers as partners. Some roles will have a higher technical bar than others.

Each person on the interview team (3-5 people) should be responsible for evaluating the candidate along a subset of the interview criteria to create a balanced view. Ideally interviewers would ask the same questions to each candidate to calibrate their answers. I suggest that each interviewer test at least 2 dimensions of the list.

I suggest looking for candidates with an exceptional ‘A’ level strength, particularly in harder to learn skills like analytical ability and product judgement. I much prefer ABC candidates over BBB candidates because it’s possible to design complementary teams with AAA skills in aggregate.

Candidates should also demonstrate strong domain knowledge, and passion for the product, company and the customer. If they have prepared, it goes a long way (and it’s surprising how many candidates are ill prepared). If a candidate teaches me something new, or helps me challenge my own assumptions, that is wonderful. 


Appendix: Other resources

Google Criteria

  • Product Design: User experience, Design driven problem solving.
  • Analytical ability: Fluency with numbers, Using data to drive product decisions, dashboard design.
  • Technical ability: Understand technology and fundamental computer science principles.
  • Strategy: Go to market, Competitive analysis.
  • Culture: Googliness, Kindness, Leadership, Empathy.

Facebook Criteria

  • Leadership and Drive: Influence, Self-starting, Motivation, Persistence.
  • Execution: Goals, Metrics, Prioritization. Understand, Identify, Execute.
  • Product Sense: A design exercise to solve a specific user or business problem.
  • Engineering fit: Do engineers want to work with you?

Product Manager Articles

Here are a few articles about product management as an appendix, in case they are useful.

Hiring your First Product Manager

When you’re running a small startup, you may ask yourself when to hire your first product manager and what you should look for in the candidate. This is a question I get from founders fairly often.

Startup founders in technology companies usually are great at least one of the following things; making stuff and selling stuff. Few individual founders are great at both, but most successful founding teams are excellent at both.

In the early stages of your startup, when you are trying to find product-market fit, you do not need a product manager. As founders (at least one of) you should be the product people at the company – obsess about the product, spend time with customers, drive the product roadmap, etc.

When you have found product/market fit and you are starting to scale, is when you should hire your first product manager. This person can run the day to day product development and allow you (as the founder) to step back and focus on fundraising, product strategy, hiring, and important partnerships. This person should bring you significant leverage. 

Many founders think they need a ‘Head of Product’ first – I don’t think that is right. I would not hire a super experienced product person who expects high compensation, owning the entire product vision and strategy or brings in ‘this is how we do things’ from their background.

Instead, I would look for an early/mid stage in their career, who is a strong executor and is high potential and can grow with the company. This person should be passionate about your product, users and you should be excited about mentoring and working closely with them (as founder/CEO). 

If this person ends up doing a great job at the execution, they can take on more product strategy from you as you build mutual trust and respect. If they are not able to scale in this role, you can bring in more experienced folks to lead the organization. Replacing a product manager is much easier and less expensive than a head of product.

Product Roadmaps

Over the last few months, I’ve been thinking about how to design a lightweight framework for product development that fits in with different ‘product cultures’ and scales from a few teams of a few people to up to 50 people.

My current ‘best guess’ of this is a modified version of the Now/Next/Later framework that Noah Weiss nicely laid out here. For each product line I’d recommend a separate roadmap of this otherwise it gets pretty unwieldy. You can always roll up specific projects into a higher level company level view, if needed.


Principles

Here are some principles that I suggest for your roadmap:

  1. The product roadmap needs to be publicly visible and easily accessible to folks in the whole organization.
  2. It needs to be easy for folks outside the immediate executing team to add ideas or requests to the ‘New’ section for triaging.
  3. Ideas should be tracked from ideation to completion without duplication of work to update projects by lots of unconnected tools.
  4. The tool you choose is not important, as long as everyone uses it. I’ve enjoyed using Asana in the past (although they don’t integrate super well with other services).

NEW/NOW/NEXT/LATER

New (Inbox) : This is where all new items start to be prioritized. Items could be anything from large feature ideas, bugs or feature modifications. There should be a clear owner for prioritizing these items, usually the product lead, or on call engineer depending on the type of issue. Large projects should be broken up into milestones and prioritized. Try and keep this section at ‘Inbox Zero’.

Now (In Development) : This is what the team is actively working on right now, in order of priority. These tasks should be assigned to individuals on the team and usually correspond the the current sprint. Tasks in ‘Now’ are granular and well defined and have a clearly scoped deliverable and timeline. If you want more granular ordering or filtering in this list, I suggest tagging tasks by priority (e.g. P1, P2, P3).

Next (On Deck): These are the tasks/features that the team will work on in the next 1-2 development sprints. Tasks in this section should be in rough order of priority and have some definition around them, but may be broken up into more specific tasks when they move into the ‘Now’ bucket.

Later (Icebox): These are items that are unprioritized, but you know you want to get to them at some point. They could be bugs, loosely defined features, or feature modifications which should be tagged for clarity. The order or specificity around these is less important (especially for features) as you will prioritize and add more definition when the time comes.

Not Doing (Pit of Despair) : This bucket of tasks is usually hidden but is a list of ideas that you have explicitly decided that you are not doing. I suggest adding in a comment into the task to articulate the reason why you are not doing it, before moving it to this ‘Not Doing’ bucket.

In addition to the execution priority, it’s also useful to include some way to show the relative importance of tasks, so you can clearly see what tasks in the sprint can not drop. I suggest a Priority and Project Size tag like I’ve shown below.

This is an example of what a product roadmap could look in Asana:


In an ideal world, the tool is easily used and accessible by everyone in the organization and tightly integrates with other product development tools you use – e.g. Figma, Github, Google docs. Asana does not do a great job here and it’s hard to get information out of Asana if they whole company does not use it (which probably improves their retention, but makes it a worse tool overall).


Central Teams

These principles don’t just apply to user facing product development teams but would be also useful for ‘central teams’ like data/analytics or platform teams who are working cross functionally with folks across the organization. These teams often have ‘requestors’ of their time and expertise. It allows for better visibility into work and timelines, and also provides a structured system to ask for help or request reprioritization of work which hopefully leads to better visibility and trust between ‘central’ teams and ‘product’ teams.


Every company is different and each team may have different norms, but having a set of common principles, common tools and a shared language can improve visibility and reduce the switching costs for managers who oversee lots of projects and individual contributors moving between teams.

Gamification for Software

I spent 5 years working at Pocket Gems, a free to play mobile gaming company in product management where I helped design, develop, and manage most of our products.

Nail the ‘Core Loop’ first

I often get asked by friends about how they can incorporate gamification techniques into their products. Most of the time, folks are asking about these tactics prematurely before they have a ‘core loop  established and before they’ve reached product market fit. 

The core loop is the set of actions that a user completes over and over again and this must be inherently satisfying and / or useful before applying gamification tactics. 

Some examples of core loops:

  • Instagram: Take Picture, Post/Share/Tag picture, Review / Respond to likes and comments
  • Uber: Request ride, Take ride, Rate driver
  • Candy Crush: Play level (consume life), Complete level, Progress to next level, Request/Buy life

Once you’ve defined your core loop and this is already inherently satisfying (or provides utility) to your users, then tactics from free to play gaming can be very helpful in improving core metrics such as retention and monetization which ultimately drive improved LTV.

Understand user goals and motivation

When designing products always start with the user goals (short, medium and long term goals) and design the meta-experience around these goals.  Once you understand user goals (and ideally map them to business goals) you can then design a set of tactics to help incentivize behaviors that help users achieve these goals.

Here are a few principles of player motivations in games that can also be applied to lots of other products:

  • Purpose: All great games have a meta-objective (e.g. Save Princess Peach in every Mario game) that players can easily understand. This gives players purpose, and these principles also apply to utility products where purpose is already clear and does not need to be manufactured.
  • Progression -> Completion: People enjoy the feeling of progression. The simple act of completing a level, or filling up a progression bar is very satisfying to many players and is a very tangible feeling of making progress towards a meta objective.
  • Mastery: People like improving at the core action in any skill based game. It’s important to communicate to players explicitly when they reach different mastery tiers as these are typically moments of great satisfaction.
  • Status / Peacocking: People like to show off their status to their community – I’m a VIP or important in some way and there are lots of examples of this both online and offline. Some examples include – Yelp Elite, Instagram Verified, League of Legends Platinum and Rank/Awards on military uniforms.
  • Expression / Creativity: People like to create and to express themselves with easy to understand constraints. Lego or Minecraft are both great examples of having some constraints but also allowing users to be incredibly creative within those constraints.
  • Collection: People like to collect things and we’ve been doing it for a very long time – coins, stamps, etc. They like to be able to see what they have collected and admire it, as well as identify what is missing and know how to find it. Loot drop mechanics combined with collection can be very powerful. A game like Hearthstone does this very well.

Not all of these principles or motivations apply to every player, and many of the best games pick a few and execute them very well vs. trying to be all things to all players.

Apply Gamification Tactics

There are a number of effective tactics in gaming that appeal to some of the player motivations described above.

  • Levels: Even a simple leveling up system allow us to hit a lot of player motivations – Progression, Mastery, and Status. It’s a relatively cheap way to reward behaviors in your product and incentivize continued engagement in the core loop.
Candy Crush leveling system is simple and clear
  • Ranks / Tiers: Ranks are quite useful to differentiate between players and allow them to also communicate to others that they have a higher status (either earned or purchased) in their community. In League of Legends, players rank up by playing competitive matches and it’s used to both signal skill as well as find other players with similar skill levels. This can be applied to users outside of gaming with ‘VIP’ Tiers or ‘Elite’ Tiers for customers who are either highly engaged or high spenders.
League of Legends Tiers
  • Rarity: Collectors, Expressionists and Status seekers all enjoy finding items that are exclusive and rare. When combined with randomness this can be a very powerful mechanic. Some of the most successful free to play games like Hearthstone allow rare items to be both earned through skill, purchased (usually through loot drops), or crafted (usually very expensive).
  • Randomness (loot drops): Packs or Boxes which have an unknown set of rewards are very appealing to players. Sometimes just the act of opening these packs as just as satisfying as the rewards. You can integrate mystery / loot drops into your products by running a mystery sale for example where players need to open a box to reveal their custom offer.
Pack opening / Loot drops in Hearthstone
  • Quests: Quests are one of the most useful tools in free to play games to incentivize user behavior. They allow us to guide the user in a specific direction for a clear reward. Quests are quite easily applied to lots of products outside of games – e.g. 3 blog posts in 3 weeks for WordPress.com for $10 of credit towards your next purchase or 10 rides in your first month for Peloton for a badge displayed on your profile.
An example of a quest screen from Clash of Clans by Supercell
  • Badges: Badges are a very quick and easy way to reward ‘good behavior’ from players. In the example below from Peloton, you get badges for beating a record, cycling with a friend, or working out 10 days in a row. I like to look at user behaviors that result in improved retention (or another metric) and then create rewards for those behaviors – badges are one way to do that.
  • Gating: Gating prevents users from accessing parts of your product until they have completed certain tasks. Level gating or item gating are a simple way to do this in games. For example, in Links Awakening, you can’t access any of the water areas of the map until you’ve found the flippers which allow you to swim. You could apply this tactics to many complex products where users need to have complexity exposed to them gradually.

I hope that folks find this useful – it’s not meant to be a playbook but talk about some of the principles and tactics that we use in games. Remember none of this is a substitute for having a product which is inherently satisfying or useful at its core but act as a multiplier instead.

A few Management Frameworks

As a manager or manager of managers of product development teams, it can be hard to focus on the right things and to make sure that you’re making progress on your ever growing list.

To help others overcome the same challenges, I am sharing a few of my personal frameworks that have helped me focus better and be more productive over my career:

1. Understand Product and Team Health

I wrote a separate post about this here but one of the most important things you can do when managing lots of products/teams is to understand the health of both the products and the corresponding teams. I do this by asking the following three questions and tracking this over time.

  • What do the metrics say? Metrics are impartial measures of how the product is performing on an absolute basis and trending. Having valid, high quality data sources is essential.
  • What does the team say? Most of your insight will be from the team lead, but make sure and also talk to team members from time to time so you can further validate (or invalidate) the insight from the lead.
  • What do our customers say? Talk to customers, talk to customer support, get structured data on customer pain points.

Combining the insight from these three sources has helped me improve judgement around what we should build and also help with designing better teams.

2. Segment your work

I segment all of my work into three buckets:

  1. (10-20%) Set of things that only I can do (or want to do) myself
  2. (60-70%) Set of things I can structure and review
  3. (10-20%) Set of things that need zero oversight or someone else can do better

This allows me to spend time on the areas that I can have the most impact while making sure that I don’t drop the ball on all the jobs that need to be done by the organization.

Over time, if your objective is to make yourself redundant you should aim to move more and more tasks from category 2 to category 3. This is also a good sign of a team that both well assembled and performing well.

3. Track your time

Each quarter I write up my personal goals (Primary focus, Secondary focus, Observing) and share them with folks I work with very closely.

As part of this exercise, I reflect back on the previous quarter and break down my allocation of time and highlight anything in my list of goals that did not get done.

I then take all the tasks that I don’t think should be on my plate going forward (not the right priority, or someone can do better) and plan to transfer them to someone else for the next quarter as part of my personal planning process.

This has helped me be more deliberate and focus on the things that matter.

4. Get buy in for projects

People are the most important asset in any product development organization and high performers do not like to be told what to work on. One of the most important things that managers of product development teams have to do is get buy in from their teams on the projects they work on.

In the ideal situation a specific job to be done matches both the interest of a team/person and their capabilities. In other situations you’ll need to get buy in from teams to take on projects, and the best outcomes are always situations the team is motivated to work on the project (ideally it’s even their idea).

Depending on the person or team and their preferences, it’s important to phrase the project in the right terms:

  1. Do it for the Company: This project the most impactful thing you can do for the company’s growth – logic and long term thinking.
  2. Do it for your Team: This is the most impactful thing you can do for your peers or your team – community and selflessness.
  3. Do it for Yourself – This is the most impactful thing you can do for your development/career – drive and growth.
  4. Do it for Me: This is something I need you to do for me – strength of the personal relationship. This line should be used sparingly, because it can be relationship damaging and/or selfish.

These are a small subset of tools that I’ve found personally helpful as I’ve worked with product development teams over the last decade and hope you do as well!

Getting into Product Management

I get asked frequently for advice from folks who are looking to get into product management and often send them slightly customized versions of the same thing. I decided to write something a little more comprehensive and share it broadly. 

There are a few phases of work for folks looking to get into product management:

  1. Start by learning about product management and what product managers do
  2. Figure out where you want to work and make a list of companies that are exciting to you
  3. Prepare for your PM interviews, and learn how you’ll be evaluated

1. Learn about product management

Product management is different from company to company. It’s worth learning about the different perspectives of product people at different companies, and here is a short selection:

2. Figure out where you want to work

An important part of the process is generating a list of companies you may want to work at by evaluating company size, quality of mentors, your connection to the product etc. Product management varies significantly by product, vertical (ecommerce vs. autonomous car PMs do different things) and individual company so it’s useful to spend time upfront here.

I recommend going somewhere where you think you’ll get good mentorship from people who are both experienced and very strong product managers. I also recommend joining a company which is growing, as a lot of opportunities can arise from growth.

Companies like Google and Facebook have very well respected product management practices, but it can be difficult to get an interview or get through their process without prior product management experience (unless you are earlier in your career where they have great rotational Associate Product Manager programs).

Some good resources:

3. Prepare for your interviews

Read and Learn: There are a few foundational books that will help you prepare for your PM interview and generally help you becoming a better product person:

  • Read the lean startup
  • Read design of everyday things to teach you how to think about user experience
  • If you are a ponderer and not a doer I would recommend making things happen
  • If you want to learn how to run a product development process I suggest reading agile product management which can be a bit dogmatic and dry, but it’s useful to know these foundational elements
  • If you don’t have a technical background, I’d spend some time learning how technical systems work and some of the foundational elements of programming. The best way to learn is to read/watch videos (good resource) and take a Codeacademy class (javascript or python)
  • Cracking the PM interview is a really good book by a former Googler, and I’d start by watching her youtube video
  • Practice questions on the PM interview – it autogenerates a bunch of questions and you can go through them

Analyze products: Spend time breaking down products you like/don’t like – most ‘product people’ naturally do this, and enjoy this type of exercise. I like to break down my analysis into 1) Why does this product exist, what user need is it solving? 2) What do I like about the product? 3) What would I change and how would I change it? 

Learn how you’ll be evaluated: Companies hire somewhat differently so make sure that you ask your recruiter or hiring manager about how you’ll be evaluated as part of the interview process. Here are a few dimensions from my experience that I’ve used, and seen used in the past. 

My Interview Criteria: There are a few key skills that, I believe, PMs need to be successful and I use them to assess product management candidates. It’s important to have at least one area where you feel like you are excellent and can get that across during the interview process.

  • Analytical Ability: Run AB Tests, interpret metrics, data informed decisions
  • Product Sense: System design, uX design
  • Leadership: Inspire, influence, build loyalty, have empathy
  • Project management: Prioritize, get things done, make tradeoffs, unblock
  • Technical ability: Ask the right questions, build trust/respect

Google Interview Criteria:

  • Product Design: User experience and design
  • Analytical ability: Fluency with numbers, product metrics
  • Technical ability: System design, algorithms – earn respect from engineers
  • Strategy: Business turnaround, go to market
  • Culture: Googliness, kindness, leadership

Facebook Interview Criteria:

  • Leadership and Drive: Influence, Self starter, influence teams
  • Execution: Goals, metrics, prioritization, process
  • Product Sense: Design, understanding users
  • Engineering fit interview: Not a technical one like Google, more of a fit interview

Being a product manager is fun, challenging and a great fit for people who like to make things, and like making things in a better way.

Best of luck in your journey and thanks for reading!

Personal productivity tools (free)

I love using tools to make myself more productive and let fewer things fall through the cracks. There are so many great (free) tools available and here are a few of the ones that have helped me the most:

Calendly: Scheduling is one of the biggest time sinks for me. Calendly allows me to share my availability (in time slots that I define) and allow people to schedule time with me without the back and forth usually required. It usually saves me 3-5 emails per scheduled call and I like it better than using virtual assistants like Clara or Byron ($200 per month each). I use the free version which I imagine would fit most people’s needs.

Streak: If you manage any kind of pipeline (sales, investments, recruiting) and use gmail, then I highly recommend Streak. I’ve used it to track potential investments, investors and for recruiting. Streak allows you to have a CRM in your inbox and also scales well to multiple users. It allows me to stay organized, have a record of interactions, and make sure that I don’t let to-dos drop. I use the free version as well, but it costs $50 per month if you’re using it with multiple people or need API access.

Gmail canned responses: I realized that I was very frequently writing the same set of emails over and over again: 1) Scheduling time 2) Making a connection 3) Product information 4) Passing on an investment. I use the Gmail canned response feature to add in the re-used content in addition to the personalized note that I send.

Get insight from independent sources

In my opinion, one of the hardest parts of product and general management is drawing insight from the right sources to determine ‘product health’ to identify where to focus, especially when managing multiple product lines.

In my experience I try pull data from three independant, uncorrelated sources to inform where I should focus my effort – the data, the team, and the users:

  • Data: Design dashboards that give you the metrics at the right level of detail on a daily/weekly/monthly/quarterly basis. Be able to translate data into concrete hypotheses and insights. 
  • Team: The general manager (GM) or product lead of the business is your main source of information, but make sure to spend time with team members and other functional leads as well, so you can validate/invalidate what you hear from the GM. The broader team is also an incredibly strong resource for ideas for new features. 
  • Users & Customer Service (CS): It’s important to maintain empathy/understanding of your customers, even when you’re a step removed from the product.
    • On a regular cadence (e.g. weekly), spend time reading user reviews, blogs, forum posts etc.
    • Get quantitative and qualitative information from the CS team about what users are saying about your products over customer service channels, either through a short meeting, or a list of top 5-10 issues each week.
    • Spend time actually interacting with customers, and responding to them (directly or on forums for example).

Making great games

I started working in the games industry in 2010 and joined Pocket Gems as the first product manager to help us create free to play gaming as a new category on mobile. Mobile phones are personalized, portable computers that are carried around everywhere by people, and it made sense to me that for many people in the world, this would be their primary gaming device.

It was a really interesting time to be in mobile games; Apple had just launched in-app purchases, app discovery/advertising was nascent, and almost all causal developers were focused on Facebook/Web vs. Mobile.  In 2010, for some additional perspective, King.com (Candy Crush) which now has an annual revenue of >$2BN (95% mobile) had zero mobile revenues and Supercell (Hay Day and Clash of Clans), which is now valued at >$3BN, did not even exist.

In 2010, in order to succeed we had to create products that had mass appeal and were first to market. People played our games because they were casual and fun at a time where few free to play games existed on mobile. Our design was simple, and often inelegant, and we did an excellent job with merchandising and tactics but often lacked insight into player behavior beyond what our (fairly sophisticated) analytics told us. We lacked empathy for our players and designed products which were inauthentic to us, and over the long term I think this became evident to our players as well.

We have since realized that we will never be a creator of really great products and games if we continue to develop them in this manner. I was responsible for the design and development of one of the simulation games in our last cohort, Animal Voyage, which will end up making a small profit but we don’t consider it a success because we didn’t create a lasting franchise. I was never a player of sim games, and struggled to get into the mindset of the player. It resulted in a product that was inelegantly designed, with too many disjointed mechanics and a lack of attention to player experience. Over time, our players realized this and long term retention was poor despite really strong early metrics (which we used to determine the game’s viability). In the end, lack of empathy for our players and lack of focus on making the game really fun (measured by long term retention) led to the downfall of the product. 

I’m now working on a new title, and the emotions I experience while playing the game remind me of games that I loved growing up. I’m excited to tell my friends about it, I’m excited for our daily throwdowns and I’m really excited about how energetic our team is about the product and the vision. I have come to the realisation that to even have a shot at creating something great you have to have great passion for the product, care greatly for your players and make design decisions that are consistent with your mission and objectives. I have also realized how important it is that your team cares about the product, is deeply invested in the outcome, and makes every decision, no matter how small, with the player in mind. I think that even a team with all the right skills and talents needs this mindset to be able to create something amazing, and I’m personally really excited to be developing new games with this philosophy.

I hope it leads to a game that has many loyal fans and becomes a lasting franchise but even if it does not, we’ll feel much better about the path because it at least gives us a shot at achieving this goal.

Metrics – The Online Advantage

I believe that online businesses have a key edge over offline businesses – they are able to easily gather data on customers, the purchase funnel and conduct iterative A/B testing. Harnessing this data and using it to drive decisions for customer acquisition, product development and generally having excellent management information (MI) is critical to executing a successful internet business.

There are three things that I want to touch on in this post:

Using Metrics to raise financing: While I was working in VC last summer, I encountered many entrepreneurs and it was interesting to see how they all thought about and ran their businesses. It was significantly more impressive and informative when the entrepreneurs understood the right metrics for their business. It allowed them to educate us on the important variables and what the implications were for their business and it also made it easier to compare the business against other models that we were familiar with. I think that it shows professionalism and credibility to be on top of this information and it was definitely something we used to screen entrepreneurs.

Segmenting customer base – yield optimal unit economics: Once the product has been launched and the customer base starts to expand I think it’s really important to start to segment the customer base and understand the motivations and unit economics of each segment. I think you should start by understanding what attributes that you can use to segment customers – demographic information, source of click, etc and then measure these attributes against engagement metrics, revenue per user, social metrics etc. This will allow you to identify different user groups, understand what motivates them (potentially through qualitative studies) and plot their evolution over time. This data would be extremely useful to drive product changes as well as acquire specific types of customers.

For example: I was recently talking to Pasha Sadri at Polyvore (a social fashion site where people create sets or outfits which are shared with the community) and a handful of talented “creators” drive 80% of the traffic to the site (approx. 6M monthly users). If they were able to identify patterns about where these creators come from / demographic it would be easier to acquire more creators and they would drive significantly more traffic to the site.

Product changes – A/B testing: This is a pretty heavily blogged about topic but I think you should use data and metrics to drive and measure incremental product changes. It’s especially efficient when you have a suite of products with similar features and you can leverage learnings from one product change and apply it to the family of products. Zynga are especially good at A/B testing and they have learnt best practice in monetisation/virality and roll out their learnings to new and existing social games very effectively.