Tools to Power Distributed Work

The way we build software and collaborate is going through an accelerated evolution as more organizations adopt distributed and hybrid (partially in person) work. In both cases, tools and processes will be designed for distributed work first, which can then be adapted to in-person more easily than the other way around.

There is a lot of innovation coming in the tool stack to facilitate these new practices, which I’m personally excited about and plan to follow closely.

The principles that I think will become the most important for these tools are:

  • Asynchronous: Tools should enable both asynchronous and synchronous collaboration seamlessly. This should apply to both short and long-form communication (which should also have separate interfaces). Examples of tools that do this well are WhatsApp and Slack, which are both probably more important to me than email now.
  • Transparent: Organizations will move more towards ‘transparent by default’. This will allow more people to have more context and create more trust in organizations. A challenging problem to solve is information overload, allowing teams to better separate signal from noise, and picking most important things to consume and act on.
  • Connected: Organizations will want to collect and record more data on work created in the organization. Tools will need to be more connected (i.e. use APIs to talk to each other better) and information will need to be better organized and accessible across the organization.
  • Consistent: To be effective, a minimum set of collaboration tools need to be accepted and used by the organization, which can be difficult at scale as it may involve both cultural and behavioral change. This is pretty self-explanatory (I don’t go into more detail below).
  • Measurement: As work moves to the cloud, and we collaborate digitally, we will need more tools to measure productivity and organizational effectiveness, culture and morale.
  • Human: There will be more focus on being human and building broad and deep relationships inside and outside organizations. In person time is still great for driving creativity, serendipity, building trusting relationships, and humanizing work. We will find more ways to simulate these experiences virtually.

I’ll explore these in more detail below.

Asynchronous

Building a culture, a set of tools and a set of practices that allow for asynchronous work is the foundation for distributed teams. It’s more inclusive because it allows for people in multiple time zones (or working hours) to process information and give feedback. Systems that work asynchronously that can be adapted seamlessly to work synchronously (e.g. Slack) are great where communication can be organized around projects, teams or topics.

I think this should apply to audio, video and text. I’ve not come across tools for async Audio and Video (possibly Loom) in the workplace. I used WhatsApp personally for sync/async text and audio the most. We recently took a long (1.5hr) townhall video and ‘chunked’ it down to its components and pulled it together in a blog post so that teams in different time zones could consume it asynchronously. It was very well received, especially as we did it very quickly, so it was ready in a few hours after the presentation (but a tool would help do this faster).

There are arguments that asynchronous work can slow down the pace of iteration or reduce ‘riffing’ off each other to create better solutions. I think that these are fair points, but most of this kind of riffing is useful at crossroads or can happen in very small groups and teams can create synchronous time for this if necessary. I think synchronous collaboration (and in person) is especially effective for new projects which require finding product market fit, where rapid iteration and creativity are even more critical.

Transparent

I think that more companies will work more openly in the future. We will trust our employees with more, and they will have more context for their work. This applies to both short form communication (chat) and long form communication.

In order to enable this (particularly for long form communication) we need a set of tools that allow employees to be able to collaborate in the open. At Automattic we use a tool called P2 which allows us to publish anything openly (structured as a blog post) and for our colleagues to like, ask comments and questions and collaborate in the open (Notion also works well). We’re able to pull in information from other tools and embed them into the post or link to other areas of work that are relevant. We need to make improvements on how this information is organized and the ‘state’ of a post (e.g. is it ‘active’ or ‘closed’? which project does it relate to? how high priority is it?). We’ve tinkered with some ideas but not settled on anything just yet as we were worried about increasing the friction to share with colleagues.

In addition to building systems to share openly, we also need to think through the subscription and notification systems to alert people to the right information they need for their work at the right cadence and the right priority. Different people need different levels of breadth vs. depth depending on their role.

A crucial benefit of working openly and making this information easily discoverable (indexed, searchable) is that the work exists regardless of the organization’s mix of people. As employees move to different teams or companies, or when new people join the company the decisions and the reason behind these decisions is always available.

Connected

Organizations want to control more of their own information. As we rely more and more on external SAAS tools to power our work our internal data is more fragmented than ever.

Organizations need to be able to extract and record information from these tools which is currently difficult and requires a lot of custom integration work. Many SAAS tools make this difficult because they know that we are less likely to churn if we are reliant on their platform as a source of record.

Creating connected tools and workflow that enables data to move between these tools and ultimately be owned and stored by the organization is important. Once this (difficult) part is complete, building a UI (and search tool) to discover this information is a natural extension.

A few interesting companies innovating in this space include: Playbook, Onna, Bento, Solve Data and Rock (which is a broader solution).

Measurement and Productivity

We don’t have good systems to track productivity for individuals, teams or projects especially as our teams become more distributed (and tools become more cloud based) and we become more output focused.

Managers need both qualitative and quantitative data to monitor teams more objectively with less bias, but employees also need to feel like they are part of the process and are not being micro-monitored.

Organizations need to log and visualize this data over time and understand that this is an input into productivity and does not tell the full story.

I wrote a detailed post covering this topic here – https://aadil.blog/2020/12/18/measuring-productivity-of-distributed-teams/

Human Relationships

Relationships are important for collaboration and teamwork. Trusting relationships in our immediate teams and around the organization can lead to more productive, more creative work with more candid conversations that get to better outcomes faster.

Even at Automattic, which has been distributed since 2005, we’ve relied on in person team meetups and a global ‘grand meetup’ to help build relationships and foster serendipity in the organization. There is value to seeing colleagues in person, enjoying lingering (less transactional) conversation, and eating and drinking together.

As we move towards more distributed and hybrid models of work, we will need to make sure that we create opportunities to build relationships (deep and broad), humanize our interactions during work, learn from each other and have fun with each other both within and outside our organizations,

A few interesting companies innovating in this space include Remotion, Lunchclub, Start Playing Games, Contra and On Deck.

Hiring and Onboarding (Bonus)

Hiring and onboarding of employees also evolves in a distributed environment. Hiring may take longer and leverage more text based (short and long form) collaboration trials versus traditional in person interviews. There are advantages to this process — it feels more two-way than one way, and helps remove more bias, and is more reflective of our actual work.

Onboarding and training employees needs a clear process which can be less rigid than typical onboarding bootcamps at larger organizations (and asynchronous). To achieve this we need better documentation and need to humanize the process as much as possible (e.g. pairing new folks with buddies, recording welcome videos for context etc) while giving employees the tools to onboard and contribute at their own pace (with guardrails, of course).

A few interesting companies innovating in this space include Rippling, Donut and SkillMagic.

I remain very positive and excited for the future of how we build software and collaborate as humans. The more we can remove physical barriers to build high quality creative products the more we can improve liquidity in the global labor market and find the right people for the right jobs regardless of their location. This is is an exciting future, and better tools will continue to power this movement.

Measuring Productivity of Distributed Teams

Measuring productivity is hard, especially for knowledge workers and craftspeople who are building software products. 

People are typically the most expensive asset for technology organizations, yet it feels like the internal systems for defining and measuring productivity are archaic and don’t match the methodical approach we use for external product development (iterative, data+gut) in customer facing products. 

As more of our work happens in the cloud (and distributed) we are now able to actually record contribution and interaction in a way that was impossible even just a few years ago. 

This post is not meant to be an ‘answer’ but a set of early thoughts (from someone who runs a distributed team at scale). I’m excited to see how our work evolves, and how our tools evolve to power more productive global and distributed work. 

Defining productivity

  • How do we define “productive”?
  • How do we think about this at both an individual and a team level?
  • How do popularity and personal (and cultural) bias factor into this?
  • How frequently / over what time period should we measure productivity? (Creative work happens in phases)

These are hard questions, and I don’t have clear answers, and I think there is room for improvement across the industry for in-person, hybrid, and fully distributed teams to define and measure all of these better.

In my mind, productivity is a function of volume, quality, and impact. It’s easy to say this, but these are all hard to define, track, and attribute which is why this is still an unsolved problem.

The most productive teams can have no outstandingly productive individuals, and the least productive teams could have exceptionally productive individuals. It’s hard to actually parse individual productivity when running a larger organization.

As managers of managers, we don’t have sufficient tools to be able to understand most of this and most of our information is driven by sentiment and infrequent interactions with individuals on our teams.

For in person teams, productivity is often a function of time spent in the office combined with their team’s sentiment/perception of their work (and likeability) which is imprecise, and polluted by biases and proxies for actual productivity.

Measuring Productivity

To measure productivity of teams and individuals, we should look to both quantitative and qualitative measures. I prefer measures that apply to individuals but can also roll up to both teams and projects.

Quantitative metrics

Quantitative metrics to measure productivity can be hard to pick, measure and implement in the organization. These types of metrics may make employees feel ‘watched’, don’t tell the full story, and could encourage suboptimal behavior to gamify metrics.

Defining quantitative metrics that can roll up from individuals to team level and project level metrics are best, because it allows us to have the most flexible view of contribution and productivity, but these metrics are hard to select.

Here are a few ideas:

  • Volume: A measure of activity across different tools (e.g. Slack messages, GitHub commits, words on internal Wikis). These metrics will vary by team and role, and need to be measurable across tools. I don’t think more activity necessarily implies more productivity but an absence of activity likely implies a lack of productivity.
  • Quality: Quality is hard to measure. There are some heuristics we could use like rejected pull requests, bugs, live issues, comments/likes on internal memos. I don’t have a clear view if these would actually work at an atomic level without running some experiments to figure out if they are actually predictive of quality.
  • Impact: First we need to define impact (e.g. OKRs, trackable metrics). What business or user goal are we trying to achieve? How do we attribute work directly to these goals, especially at the individual level? It’s easier to do at a project or team level, but still difficult to map work to impact accurately.
  • Micro-contracts: I generally work using a series of micro contracts. I commit to a specific scope and timeframe and it’s usually an agreement between me and one other person. It would be valuable to track these micro contracts in a transparent, two player view (vs. separate todo lists) that integrates nicely with my existing workflow (e.g. a Slackbot). This is a feature, not a product, but it would be useful to capture data (e.g. missed, delayed, complete) on this over time across the organization.
  • Engagement: We have an internal tool called P2 at Automattic (replaces email) which we use to write and store long form content and share it transparently with our colleagues. Much of my work as a manager is engaging with these P2 posts, leaving likes, comments and asking questions. We track some of this data internally, but don’t have a great view for managers to look at aggregate views for individuals, teams or projects (which would be helpful). 
  • Project Iteration: I strongly believe that speed of iteration is a sustainable advantage for companies, and many companies lose this speed as they grow. Something which captures progress towards stated project goals, as well as the project state over time, would be very helpful especially when combined with some of the other individual metrics listed above.

I’d start by tracking all these metrics individually (and creating a time series for each) and then worry about creating compound metrics later, for the metrics that are most accepted by the organization.

Qualitative Metrics

Qualitative feedback is important both as a source of positive and developmental feedback for individuals and teams. This feedback contains the most bias, but is the most widely used and accepted form of assessment across our industry.

  • Peer Review: This provides a good view of the person’s capabilities as a team player, and a measure of their ‘popularity’ with their peers. It’s also a good measure of team fit as different teams can have different microcultures.
  • Manager Review: Managers typically write regular (e.g. annual) reviews which are a useful point in time synthesis but have recency bias (something of which I’m certainly guilty). Managers don’t record all the ‘micro feedback’ and ‘micro improvements’ in a transparent way to the organization (e.g. logged in a database) which I think could be interesting to try.
  • Direct Report Review: This is similar to manager reviews, except it’s many to one for the manager. This is useful to check for consistency of feedback, which can be a stronger signal than an isolated piece of feedback which we could over-react to if particularly positive or negative.
  • Self Review: This is an opportunity for individuals to synthesize what they think is most important and impactful retroactively. I personally find it useful, and think it’s also useful to kick off a conversation with a direct report (when there is discrepancy in perception). It’s also a good measure of self awareness (if self view is very different from the others).

In general, storing micro feedback (e.g. positive/developmental plus a string), and periodic summaries (discrepancies, sentiment by group, change over time) would be a good place to start, and then HR teams could perform analysis on the usefulness and predictiveness of each of these inputs over time.

Both this qualitative and quantitative data would be helpful to both the individual and the manager to track their development and journey during their tenure at the organization. It would also make manager handoffs (team switches or people leaving the company) cleaner and easier without losing valuable organizational data.

Open Questions

A couple of additional open questions that I did not have good answers to:

  • Meetings: I almost added meeting time/number of meetings to the quantitative metrics section, but I’m in two minds about them. I think recurring meetings are a ‘lazy’ way to have individuals and groups get together and make up an agenda. Large meetings with lots of passive meetings are also very ‘expensive’ for the organization, and likely better handled via written, asynchronous alternatives. I do think there are a few exceptions (1x1s for direct reports, project kick off, strategy/direction changes) which have their place to drive alignment and build relationships. I personally like short interactions about specific problems spun up in real time but these are hard to coordinate if teams are over-scheduled.
  • Real Time vs. Synthesized: For all of these metrics, how much should available in real time vs. point in time? Real time plus history allows for better trend analysis, but point in time allows for synthesis. Point in time reviews suffer from recency bias and most creative work happens in phases (with natural spikes and troughs).
  • Transparency: For all the quantitative and qualitative metrics how transparent should this be to the individuals affected and the organization? If totally transparent, I worry about gamification but if totally hidden it feels like spying. I think it should be run as a contained experiment to see the impact on the culture.
  • Popularity vs. Objective Metrics: As humans, how do we eliminate bias? We are often biased towards people who we perceive to be similar to ourselves. We are more likely to excuse poor performance for someone we like v.s. someone we don’t. Measuring and accounting for bias is extremely difficult.

After writing this, I was left with more questions than answers, but feel confident that most organizations (distributed or hybrid) are not good at measuring and storing performance over time at the individual, team, or project level. I also think that the appropriate solution(s) will vary by company and its culture. Building tools to track performance and creating a culture of internal experimentation is necessary for organizations to get to better solutions across all levels. 

I’m personally excited about innovation in this space as a manager, and an increased focus on output over facetime and popularity as companies work in a more distributed or hybrid work environment.

As a side note, this was an interesting view on how the folks at GitLab think about this — I appreciate them being open with sharing a lot of their practices in general.

Decentralized Game Development

There has been a movement towards decentralization of content creation in many industries (Youtube for video, WordPress for writing, Podcasting for radio). These creators and storytellers now have the tools to deliver high quality experiences (without massive budgets) and have access to distribution platforms to find and grow audiences, which was very hard to do in the past. I think there will be a movement towards the decentralization of game development next.

The power of games is in the mechanics, the stories and the world. Even for large game studios, the visionary is usually one or two people (also true for Pixar Movies – see Creativity Inc for more). However, most of the cost and the time for games is spent in the ‘production phase’ for AAA studios which means many independent game makers cannot compete with large franchises.

If small, independent teams had access to free/cheap and high quality game engines, reusable off the shelf content (entire rule based worlds + logic), asset libraries (textures, photogrammetry, user generated) and common game mechanic libraries (leaderboard, ELO) then their focus can be on the story, the world, the core gameplay.

Flexible, cross platform game engines like Unreal and Unity are not quite good enough yet to realize this vision although I think we will get there very soon. I spent some time in 2017 making a VR film entirely in Unity and was really impressed by the power and flexibility of the platform.

My mind was blown when I learned that The Mandalorian was made in a single room and all the worlds were created in Unreal and rendered in real time during filming on massive LED screens (short video below).

Game distribution platforms like Steam, Google and Apple App Stores (and communities like Discord) are going to become even more powerful and influential for creators to find players and engage with them (and each other). Franchises will still be very powerful, but independents will be able to access (niche) audiences much more easily than ever before. I think there will be a lot of pressure on app stores to reduce their take rate as 30% feels much too high. Epic and Unreal have the most developer friendly agreement I know (free to use and then 5% after $1M in sales)

I’m particularly excited for young people (even children) to be able to have access to the tools that will allow them to conceive, create and publish games the same way we publish a blog, podcast or youtube video today. Roblox is a great example of a game creation tool that embodies these principles (30M DAU, 7M Active Developers & $600M Revenue) and has exploded in popularity over the last few years, especially with young people. It still lacks the power of Unity and Lua is not that easy to learn for non-technical folks.

There is a lot of innovation on both creation and distribution that will continue to empower creators. Combined with the general trend in ‘no code development’, this will democratize game development which, I hope, will continue to become more mainstream. The game engines and distribution platforms are very well placed to both create and capture value over the next decade if they build for the long tail of creators.

Finally, more of our social lives are now lived online and combined with lasting effects of physical distancing (from Covid-19) this will accelerate the development of games where people can have meaningful, deeper interaction online. For example, I play Fortnite with my nephew in Paris (he’s 9), from America and it’s a really nice way for us to have fun together and hang out.

My family and friends live all over the world, and I’m rarely physically present with them. If we had more options to socialize over games (both simple as well as immersive) maybe even made by us together, that would be pretty dope.

Creating Liquidity in Private Markets

Two of the most significant issues with private market investing are their inherent illiquidity and the unpredictable nature of exits. It means that you need to ‘invest and forget’ when investing in private markets, and is particularly true for angel investing. This problem also applies to equity based compensation for employees at startups, where employees are unable to realize value even when the business has increased in value.

I believed this problem will be solved over the next few years. There will be a number of new options for liquidity in the private markets and early stage investing is going to become even more attractive to even more people. I also think that the US government will make further relaxations to the accreditation rules that restrict investors from making certain investments based on their net worth or income (originally intended for their protection).

For late stage (pre-IPO) private companies there are a number of platforms (e.g. Forge, EquityZen) working on the liquidity problem, but very little is automated. Typically ‘blocks’ of equity become available either in secondary offerings (existing holders, like an early employee selling their stake) or when early investors have pro rata rights to a future fundraising round and can’t fill it themselves. I’ve used these platforms from a buyer’s perspective (and explored as a seller) and it’s all fairly manual right now, and the products are mostly a nice user interface. There are also more traditional alternatives like Setter Capital who don’t claim to be a technology platform.

In real estate, companies like Cadre are both providing access to large real estate projects to small investors (breaking up allocations into smaller chunks), and providing “windows” where investors on the platform can sell portions of their positions (every 6 months or so). This both increases the number of investors who can invest in the asset class (lower minimums) and improves the exit options for investors who would typically have to hold their equity position for 5-10 years.

In the future, I expect platforms like Carta and AngelList to lead the way in creating a true ‘marketplace’ for private securities. Carta just announced their new liquidity platform CartaX, a private stock exchange which is an awesome innovation launching in January 2021. It’s a natural extension of their business model, and I’m excited to try it out. Angellist will also continue to innovate, especially because of their SPV and fund offerings, as they hold a lot of private company stock (and have access to a lot of data).

There are some problems that need to be solved such as restrictions on employee stock, rights of first refusal and companies at the early stages desiring control of their cap table. I think these are all surmountable, and most of the trading will start with late stage private companies first, where these concerns are less relevant.

Overall, these developments make me even more excited to continue angel investing; as private company stock becomes more liquid, it’s value will increase (in addition to normal value creation through growth). Ultimately, it means that more investors will invest in private companies and they can have more of their capital ‘working’ (and need less of a cash buffer) because they know that there will be a liquid market in case they need the capital.

Democratizing VC Investing in Africa

Most people don’t have access to investment opportunities in either emerging markets or private markets. Access to early-stage investing (venture capital), in particular, requires prohibitively high minimum amounts of capital, and emerging markets investing requires specific knowledge and access. While it would be rational (both for diversification and long term gain) for many investors to have part of their capital allocated to these segments, most investors are over-exposed to both traditional asset classes (public equities/bonds) and their home markets and lose out on the benefits of diversification. This needs to change. 

I strongly believe that there is a tremendous market opportunity in African entrepreneurship and technology over the next decade. I’ve been an active angel investor in Africa for six years (and a global angel investor for ten years). I’ve set up a rolling fund focused on Africa to offer more investors in my extended network access to early stage investing in Africa. 

This new fund is my small contribution towards democratizing access to global, private markets which have long been difficult to access. It has some additional benefits, including over 10x lower minimum commitment amounts and 10x lower management fees, and I am personally one of the largest investors, which is all atypical in the industry. 

Access to private and global investments

Over the last decade, technology has enabled many notable trends towards democratization and decentralization — in publishing (blogging), television (YouTube), and radio (podcasting) to name a few. As platforms and tools continue to evolve, I believe this trend will extend to more industries (including private investing), and I’m excited to contribute towards this movement.

Access to private investments is becoming increasingly relevant and important; public equities are now more concentrated than ever, and actively managed public funds have been replaced by ETFs. This Morgan Stanley report summarizes the increased capital allocations over the last decade towards private investments driven by technology investing — and these investments have historically been impossible to access for all but the very wealthy, or large institutions (pension funds, endowments, etc).

Source: Morgan Stanley

Most individual investors have a more limited set of assets that they can access and are excluded. This results in individuals being overexposed to the most liquid, tradable assets (e.g. public equities in the USA). 

In addition, individual investors are typically over concentrated in their home markets (real estate, stocks, etc) relative to their net worth – foreign markets are harder to understand, and can be legally complex. I believe that there will be significant value created in global, and particularly emerging markets over the next decade. The data point to more and more entrepreneurs building businesses in their home countries (even after a tier 1 US education) versus trying to build their companies in entrepreneurial hubs like Silicon Valley and the USA. Over 40% of Y Combinator founders are now international, and the vast majority want to build their businesses in their home markets. This trend has been accelerated by Covid 19, and the rise of distributed work which allows for much better labor mobility regardless of physical location. 

Source: Vanguard for public equities 

AngelList (Rolling Funds) have built tools for angels to accept small amounts of capital from external investors, and invest this capital globally which was very hard and expensive previously. My rolling fund was developed to democratize access to investing in private markets (venture capital) in Africa; more details in the next section. 

Investing in VC in Africa

There is significant market potential in Africa – many young people ready to work (median age of 19), increased urban mobilization (45% living in cities by 2025), high smartphone penetration (50% and growing fast) with digital finance access, and increasing capital and talent flows into African technology hubs (e.g. Nairobi, Cape Town, Lagos).

This year we finally saw the start of technology company exists, where seed investors realized over 30x their capital. Stripe bought Paystack ($200M), WorldRemit bought SendWave ($500M) and GoDaddy bought Over.

I am passionate about advancing the technology ecosystem in Africa. I was born and raised in Mombasa, Kenya where my family has lived for five generations. I have a strong network of co-investors and local entrepreneurs, several of whom are investors in this fund. I’m a Kenyan, a product manager, and an entrepreneur, and my experience (including building technology products in Africa as an operator) allows me to have empathy for founders on their journey. 

I’ve been personally investing in technology companies in Africa since 2014 through Musha Ventures, and am now excited to allow others to participate in these deals. Over the last six years I have backed 35+ African companies in 8 countries with an IRR of over 36% (based on future fundraising rounds) and MOI of 1.7x. The portfolio includes companies like Flutterwave, mPharma, Sokowatch, Branch, Twiga and Kobo360.

Here are some of the important details:

  • 10x+ Lower Investment Minimums: Investors are able to invest in this fund with as little as $2.5k per quarter whereas most VC funds have a $250k+ minimums for LPs. Investors will need to meet US-Accredited Investor requirements to participate.
  • 10x Lower Management Fees: The fund has a management fee of 0.2%, to cover basic running costs, which is 10x lower than the industry standard. This further helps to align incentives; I earn carry when investors make positive returns. 
  • Alignment of Incentives: I’m personally one of the largest investors in the rolling fund, as this is an extension of my existing angel investing. This is not funded through deferred management fees, it is capital that I wire into the fund just like every other investor.
  • Consistent Investment: I intend to invest conservatively and consistently into companies across Africa over many years. It’s very hard to ‘time the market’ and so we will instead focus on factors we can control like amazing entrepreneurs, evidence of traction, product quality and delighted customers.
  • Investing in B2B: We are focused mainly on startups that serve other businesses — particularly fintech, marketplaces and software as a service technology companies. We may make the occasional consumer investment, but think that business is the foundation that comes first.
  • African Entrepreneurs’ Fund: For entrepreneurs building businesses focused on Africa (and particularly portfolio company CEOs), they are able to invest in the fund with no fees or carry. This is my attempt to pay it forward, and also get even better deal flow from my network due to further aligned incentives.

I believe in the power of being transparent, which I hope will allow me to build new relationships and deepen my current relationships – it’s why I’m publishing this openly. 

Want to learn more? 

If you’d like to get in touch please fill out this short form and I’ll reach out to you so we can get to know each other better. 

If you’d like to see more detail on the fund (market opportunity, detailed investment history and fund terms) please check out the rolling fund page (https://angel.co/v/back/musha-ventures). 

Making a VR Film

In 2017, I made a virtual reality (VR), live action series with a friend (who I worked with at Pocket Gems) who was the creative visionary and conceived the idea. I was new to both VR and filmmaking and this was a cool opportunity to learn a lot in a time boxed project (8 months) with a concrete deliverable.

We made a 4-part series called ‘Playback‘, which was fully funded by Oculus Studios (Facebook). Over 15k users downloaded and watched the series and it was one of Oculus’s featured projects in the Oculus App Store. We were also selected for the Google Jump Start Program and which gave us access to a Odyssey 360 Camera.

Thesis

Our thesis had a number of components, and we were trying to figure out if we could drastically improve the quality to cost ratio for VR film production by introducing constraints. Ultimately we were looking to create a platform and process for profitable VR content creation (‘Netflix for VR’).

  • First Person: The series is shot primarily from the perspective of the protagonist, which allows the viewer to experience the story from their perspective. We thought that stereo (both eyes have a different video) was important for realism as it mimics our own eyes.
  • Narrative Stories: Focus on storytelling and include some light interaction and choice to give the user agency and improve immersion. Black mirror tried this approach with their Bandersnatch Series (which was produced after this).
  • Freemium: Start with episodic content with the first episode free, together with an upgrade path to unlock the rest of the experience (learned from our time working on Episode)
  • Technology: Build software to create proprietary production techniques that increase the quality and reduce the cost for even low end Android devices (increase the size of the audience).

Pitch

This was our short pitch for the production, called Playback:

“Playback is a revolutionary VR miniseries that pushes the boundaries of narrative storytelling and viewer interaction. It’s a cautionary tale about life-changing technological advancement and the personal and societal implications of its adoption.

You experience life through Alex’s POV as his company is about to launch their new, groundbreaking product: a device that records moments and allows people to re-live those moments in all five senses

As the story progresses, the boundary between Alex’s technology and his own reality begins to fall away, pressing the viewer to question what their own reality will mean as our world becomes increasingly virtual.”

Process

The entire process took around eight months and was broken up as follows:

Month 1: Pitch and Funding

  • High Level Pitch: We wrote up a high level pitch and deck for the Oculus team who then approved a $50k budget for Playback (over another Zombie concept).
  • Budget: Prepare high level budget to make sure that we could actually produce the experience with the $50k.

Month 2-3: Writing and Production Process

We split up the creative part (writing and character development) with the production part (technology, team, production process) and worked on each in parallel:

  • Overall Workflow: How would our overall process work from the point of filming to getting it in a users hands. We went through this entire flow, including creating an app and deploying it to the app store.
  • Vertical Slice: We picked a single scene and tried to make it look ‘production ready’ with a very short test clip. This set the quality bar for the production:
  • Story arc and characters: The pitch needed to be broken down into a story, and the characters needed to be developed.
  • Detailed Script: After the story, a detailed script with dialogue was prepared for 4 episodes, including choice/branching narrative.
  • Core Team and Contractors: We needed to assemble a small team (which was mostly unpaid) and the core team included our Directors of Photography (Sensorium) an NYU film student, and a visual designer who worked with us at nights and weekends.
  • Technology experiments: We ran lots of experiments with both hardware (e.g. cameras, rigs, lighting etc) and software (e.g. Depthkit) to figure out some of the tools that we could use during production.

Month 4: Pre-Production

  • Casting: We decided to hire SAG Actors (under New Media) which meant we had access to better actors, but had to follow certain guidelines. We looked at 100+ audition tapes and 3 finalists for each major role.
  • Prepare for shooting: This required a lot of work – we needed to pick venues, rehearse with actors, design the sets, and create very detailed shot lists to make sure we got all the footage we needed.
  • Shoot week: We planned to shoot the whole series in one week. This was a very structured and organized period starting early in the morning and ending late at night. Our “team” included over 30 people working at different points during the week and this was the most “expensive” part of the production process.

Month 5-8: Post Production

Post production took almost half the total time, and we underestimated how long this part of the process would actually take by a fair bit.

  • Shot selection: We reviewed all the raw footage and picked the shots that we wanted to use for the series.
  • Stitching: For stereoscopic VR, we needed to ‘stitch’ the video and image files together and make sure that each eye had an image that did not make it look like the user was seeing double (very off putting) which was difficult and time consuming.
  • Engineering: We added components like a ‘cyber world’, user AR interaction, branching narrative, and a number of performance improvements which required focused engineering and design time.
  • Film festivals: We chose a few festivals and created pitch decks and submission entries for them.
  • Launch and review metrics: Once launched we needed to analyze the metrics and user behaviour and compare to our original hypotheses.

Bugdet

We had a strict budget, and part of our thesis was to see how high we could push the quality bar with a fixed, restricted budget (comparing ourselves to experiences with 20x+ bigger budgets). We raised $50k from Oculus (as a grant), and we ended up very close to our original budget.

We prepared a detailed budget with 10% flex built in, and tracked the costs vs estimates meticulously in a spreadsheet. Most creative projects go over budget, and having this kind of discipline allowed us to keep costs under control and scrutinize all expenses carefully.

Given that our thesis was to try to create high quality content for a fraction of the price that was typical in the industry, it was important for us to control for costs carefully.

Technology

AR Interaction
Photogrammetry – texture mapping to create a ‘cyber world’
Stereo rig for filming

We tested out a number of different technologies in the production:

  • Stereo 4k footage: Each eye has a different video (mimicking our normal eyes) at 4k resolution which makes it feel more real to the user. This is hard to achieve on low end devices.
  • Proprietary image + video cut out system: In order to get the quality of video that we wanted, we built a system to constrain the ‘action’ to a small part of the scene and overlaid video on top of a still image. This required a very precise shooting and post production process that we created ourselves.
  • Photogrammetry: We mapped the inside of one of our scenes (3d + textures) to allow transition between live action scenes and a computer generated world seamlessly.
  • Volumetric video: We used Depthkit to capture volumetric video (3d), for a scene with a ‘ghost’ in the virtual world.
  • AR Interaction: At the start of the first episode we created a HUD which the viewer could interact with – read emails, the news etc which added to the sci fi and first person interaction.
  • Branching Narrative: We created choice where users could send different texts (via their AR HUD) which changed the outcome of the story depending on the choices they made.

Festivals

We created something that was both innovative in the way it approached story telling, as well as the technologies that we implemented for VR. We decided to apply to a few film festivals (Sundance, Tribeca) in their VR experiences segments. We got fairly far with Sundance but were ultimately not selected.

I went to Sundance in 2018 anyway, and had a great time skiing and watching movies. Our Directors of Photography (Sensorium) had another VR experience which was featured, so it was fun to see them experience some of their other work.

Outcome

Here is a short video showcasing the first episode (although it’s much better experienced in VR).

We released the final product in the Oculus App Store. We were featured and had reasonable download and view rates (over 15k people). Users got the first episode for free, and had to pay to unlock the next three episodes. There were some bugs in the upgrade process that negatively impacted our ratings but for people who were able to upgrade, many seemed delighted.

We were still a step function off in terms of both addressable audience as well as conversion to payer rates to justify our original hypothesis and invest further. Neither of us are still working in VR.

Learnings

  1. Scope creep: This was a classic mistake that we should have realized earlier (as we’ve done this many times in games) but we got too excited about some of the tools and technologies and probably added too much (e.g. volumetric video capture, branching narrative) that was not necessary to test our original hypothesis.
  2. Stereoscopic vs. Monoscopic: I think we should have killed the stereoscopic requirement early in post production. Stitching of video so that it does not look warped or incorrect for both eyes is a real pain, and this would have saved us a few months as well as allowed us to ship a more polished experience overall.
  3. Developer experience: We built most of the application in Unity and used tools like Blendr, Adobe Premiere Pro as well. The workflow was pretty cumbersome and it was fairly manual to create scenes and test out in VR. We could have built a lot of automation ourselves but it was not worth it if we were just doing it for one series.
  4. VR User Experience: At the time, we were optimizing for users using Oculus Go devices (Android phones strapped to your face) and this entire UX was terrible (buggy, battery hog, performance and storage issues etc). Standalone devices have improved the experience substantially, but we’re still not at the stage where this will be mainstream.
  5. Running out of energy: Towards the end, it became a real grind to get it out the door. Everyone was tired, and the project took 30% longer than any of us expected.

Overall it was a great learning experience but it made me realize I don’t want to work in the ‘content creation’ business especially in entertainment. I much prefer working on tools for entrepreneurs and business, and hope to spend more of my career building technology for this audience.

Hiking the Annapurna Circuit

My wife and I hiked the Annapurna Circuit in April 2018, and it was one of the best trekking experiences of my life. Each day felt different, there was sheltered accommodation, and the hiking through the largest mountains in the world was truly epic. On the highest day we hit 5,416m (17,760 ft) via the Thorong La Pass. We preferred this trekking experience to Mount Kilimanjaro because it was less of an ‘up and back’ and felt more relaxed.

Here are a few recommendations:

  • Guide/Porter: You don’t really need a guide or a porter but it’s a nice way to support the locals, and is inexpensive. A porter is more useful than a guide. On some of the harder days, carrying a day pack makes the hike a lot easier than carrying all your gear. You can sort this out easily when you arrive into Kathmandu, through your hotel.
  • Gear: Don’t overpack. You can re-use good quality ski gear (here are my tips – make sure to layer) and the packing list is similar for clothing for Kilimanjaro. A good sleeping bag is essential to stay warm on the huts on the way. Pack some blister pads. Use hiking poles.
  • Add on the Ice Lake day hike in Manang: Stay two days in Manang. You can do laundry here and watch movies at the ‘cinema’. The Ice Lanke was a really awesome day, albeit a little challenging (4,600m and about 8 hours long). One of my favourite days of the trip. We were supposed to go to Tilicho lake but the path was closed because of avalanches.
  • Complete the “half circuit” and fly back: Our trek was 15 days, and we started in Besishar and ended in Jomsom, where we flew to Pokhara a little over half through the full circuit. A lot of the research shows the last half of the circuit as a bit of a ‘grind’ with cars on the road adding to a less pleasant, more dusty experience.
  • Hang out in Pokhara after to relax: This is a chill, hippy city in Nepal. We had massages, good food, did some yoga and a few relaxed hikes and boat rides. We also treated ourselves to a nice hotel. It was wonderful to spend 2-3 days here relaxing after the trek.
  • Lodging: Accommodations are all pretty similarly priced in the villages along the way. If you arrive earlier, you’ll get a better pick of the available rooms. We were also able to “shower” (mostly out of a tap or bucket) every few days.
  • Food: Pack some chocolates, or other tasty snacks but know that you’ll have hot meals for lunch and dinner along the way and plenty of places to stop for tea. I’d recommend eating mostly vegetarian, and mostly Dal Bhat.
  • Leisure: Get a local sim card so you can communicate on the trail (no-wifi) and bring your Kindle to read in the evenings. Bring a deck of cards too. The scenery is epic but modern phones can probably suffice, unless you’re an avid photographer.

I also like the Kathmanduo blog which is well written and has a lot of details, but is a little old (2011).

Here are some photos of the trip!

‘Fresh Faced’ early in the trip
K2 In the Background (I think)
Ice Lake Hike

Epic Views Every Day
Vistas
Tea House Life
Little Bridges on the pass
Throng La Base Camp – cold!
Throng La Pass – the highest day
Throng La Day
Last few days of the trek
Unhappy to be on the tiny plane

Productivity Tip – Text Shortcuts

My favourite recent productivity tip is using text replacement which is built into to MacOS natively (System Preferences -> Keyboard -> Text). These rules also sync with iOS so you can use it on the go with your iPhone. It’s native in the operating systems so you can use these shortcuts in whatever program you’re using (iMessage, WhatsApp, Email, Google Docs etc).

I’ve set up about 20 text shortcuts and use them many times a week. I set up the shortcuts to begin with “>” which makes them hard to trigger accidentally and also lets me to use words/phrases that are easy to remember. Some examples:

  • Phone number: >ph
  • Address: >add
  • Intro thank you and to bcc: >intro1
  • Intro nice to meet you: >intro2
  • Schedule a call with me: >schedule with Calendly link

I had Alfred (paid product) for this previously, but it’s a step function worse because it’s not deeply integrated with both MacOS and iOS. The only benefit from Alfred is that it supports Rich Text (formatting) and the ability to add text with links – I now just put links in brackets. I has previously also used canned responses in Gmail but this method is superior.

It does take a little bit of time to set up and get familiar to using the shortcuts but once you get going it’ll save you a ton of time.

Trends Accelerated by Covid-19

Covid-19 has led to significant changes in how we live, work, and interact with each other. In some cases, they have accelerated trends that were already in motion, and in other cases forced changes that we did not anticipate or expect.

In the next few years, I think we will go through a rapid pace of innovation and re-imagination powered by entrepreneurs, and here are a few trends I’m excited about and interested in exploring further.

Distributed Work

The best summary I’ve read on the acceleration of distributed (not in person) work is this one from the CEO of Automattic, Matt Mullenweg where he talks about change happening slowly, and then all at once driven by this catalyst. Automattic has been fully distributed since its inception, and Matt has been a champion of distributed work for years and the benefits of accessing a global talent pool, and working asynchronously.

The #WorkFromAnywhere Podcast series led by the folks at Greylock is also excellent and CEOs of companies like Box, Quora, Okta, Figma and Zapier speak about their transition to working from anywhere.

I’m particular excited about the tooling that will be developed, both in terms of specific software as a service products to drive much better distributed collaboration, but also the underlying plumbing that ties all these tools together.

ECommerce

This article from Ben Evans on the growth of eCommerce is a must read. The UK went from 20% to 30% eCommerce penetration and analysts say that Covid has accelerated the growth of eCommerce by 5 years. This is dramatic, and will change the way may of us purchase, discover new products, and how creators distribute their products.

This also changes the nature of distribution/logistics and the entire supply chain. Companies like Shopify and Amazon have doubled their market cap (adding over $60BN, and $850BN (!!) of value respectively to shareholders since mid March 2020).

Major Cities

The nature of major cities and concentrated urban areas is going to evolve. I wrote about my thoughts on megacities recently here, and I also liked this piece from Fred Wilson about how a reset was much needed in NYC and how the city could evolve into something better. Many of my friends have ‘accelerated’ moving to their ideal living areas and and left places like NYC and London. My wife and I, having just had our first baby, are asking ourselves the same question – is it worth staying in NYC if we don’t intend to stay longer term? The pandemic has forced a conversation we likely would not have had for a few years.

Flexible Work

I believe that the best creators and experts are no longer going to need a ‘normal job’ and will be able to work flexibly and monetize their unique skills talents much better than before, and this will be socially acceptable and maybe even celebrated. I like the writing from Li Jin (Atelier Ventures) about the Passion Economy and Unbundling of Work from Employment (which I also opined on here).

Startups will create innovative tools, and platforms to help craftspeople to discover projects, collaborators and showcase their work (e.g. Contra). Much of the benefit that we get from a ‘normal job’ (e.g. competitive healthcare insurance plans, retirement accounts, etc) will also be available to creators through saas products.

Building Relationships

As humans, we yearn to build new relationships and deepen relationships with folks that we already know. Traditionally we’ve built these relationships in person with repeated interactions and meaningful collaboration on projects. Being forced into lockdown has forced us to explore alternatives.

I’ve personally been experimenting with platforms like Enrich (curated network of similar executives), Fractal (1×1 matching with other product people), Village Global Events (with startup founders and investors), and am starting the On Deck Angel Fellowship soon. These are all digital communities with fairly niche audiences, which I think will become more common.

I’m hoping that these will lead to meaningful relationships and collaboration and also improve the chance for serendipity despite not being able to spend time with folks physically. I’m excited that these platforms could open up the possibility of meeting interesting people all over the world, and not just limited to my place of residence.

I’m not sure how this will play out with larger conferences, where most of the value is in relationship building and improving the probability of serendipitous connections often through extended hang out time (often over meals and drinks). I expect that recreating much of the value will be possible, but will require some first principles thinking.

In my own recent experience, I wrote about how the funeral for my grandfather was actually much more inclusive and rich because it was virtual and allowed for more people to attend that were close to him (like his sisters).

Personalized Services

Folks who provide coaching, classes or specialized services are all going through a similar, accelerated transition.

Companies like Peloton have successfully taken spin classes and made them virtual, allowing both synchronous and asynchronous (on demand) classes with world class instructors. Each class can now be attended by step function more people which greatly improves the ROI for each class.

Experts providing specialized, personalized services like physiotherapy, child psychology, lactation consulting can all increase their addressable customer base and people who are in need of very specific services can access a larger pool of specialists which is better for both groups. They both need tools to make it easier to discover each other, and improve the experience of booking and transacting (e.g. Ribbon Experiences).

Digitial Payments and Services

Digital payments and digital services (e.g. digital hr, or payroll) to help businesses transact with their customers and run their teams will also see more new users, and increased adoption. I think these products will be ultimately sticky even after Covid-19 because they function better both in person and remotely, and allow for more flexible customer and employee interactions. In my personal investments in these areas I’ve seen increased volumes and good retention through the pandemic.

These are just a few areas where I’ve personally observed changes in my own life or with folks close to me, and I’m excited to learn more and closely track how these trends evolve.

Hiring Product Managers at Scale

In this post, I summarize a process that I recommend for hiring product managers at a midsize or growth company, adapted for a distributed hiring environment (most applicable to a company that will hire multiple product managers).

I’ve hired and trained over 40 product managers over the course of my career, and this draws on my experience as a product hiring manager and team lead.

Internal buy in and scope of role

When hiring product managers (PMs) at a mid size company the most important thing is to have internal support from the executives and the design and engineering partners. There should be a strong desire to hire PMs to help build better products in a better way and to bring in more structure and systems to the product development process.

Once there is buy-in from these stakeholders, organize the teams into sensible working groups (e.g. by user journey such as onboarding/growth or by key metrics such as conversion/retention or by product line).

I prefer a matrix structure (although has tradeoffs) where PMs ‘own’ each of these areas in partnership with a design and engineering lead (with around 5-10 engineers per PM, depending on the project). I also suggest that engineers and designers report into their own functional leads and PMs direct the scope and priorities of the projects.

Hiring process

It’s essential to have a clear hiring process and system both for the sake of your internal team and for the candidates. Most companies are incredibly disorganized about hiring, but a little bit of work can save a lot of time in the future, especially when hiring many folks for the same role.

Internal Team 

  • Recruiter: There should be a consistent point of contact for the candidate during their application process – ideally a recruiter. The recruiter communicates with the candidate, lays out the hiring process clearly, and moves them through the process. They act as a liaison between the hiring manager(s) and the candidate. They often do the initial resume screens and have an essential input into hiring because they get to know the candidate so well. 
  • Hiring Manager: The hiring manager is the person that is hiring for the role. They are the person who ultimately makes the decision to recommend the candidate as a ‘hire’ or ’no-hire’. This is typically a senior product leader.
  • Interviewers: Each interviewer should have a clear set of criteria that they use to evaluate the candidate. The interviewers should be excellent at the functional areas that they are evaluating candidates and hold the quality standard for the organization. The best people should be involved in late-stage interviews and this should be a core part of their job description.

Hiring Process

  • Resume screen: Internal and external candidates should submit a Resume / LinkedIn profile which should be screened upfront (recruiter + hiring manager). Candidates who pass this phase should move to a conversation with the recruiter, followed by the hiring manager.
  • Interviews: Interviews should consist of a standard set of, very well calibrated questions that can be asked by a variety of interviewers representing the different development functions (e.g. design, engineering, product, marketing). A structured hiring guide improves consistency and calibration, and can reduce bias from the hiring process. 
  • Central Tool/ATS: Interview feedback should be stored in a central place/tool (e.g. Greenhouse or Lever) and each interviewer’s feedback captured clearly (with a hiring recommendation). This allows us to both evaluate interviewers and the candidates – e.g. some interviewers bias towards higher or lower scores.
  • Written Exercise: If you are hiring in a distributed environment, try to find candidates with strong communication skills (particularly written skills) and clarity of thought. All candidates should complete a written exercise as part of their recruitment process which could include:
    • Break down a product you love – what you like, what you don’t like, how you would make it better (1 page)?
    • What is your favorite technological shift and why?
    • Write a ‘product spec’ to address a specific problem that the company has (better if it is a real problem).
  • Trial: If possible, ask the candidate if they would be open to a two-way trial (which is compensated) where they try and solve a real problem and collaborate with an internal team. This is time consuming (20-40hrs for the candidate, 5-10 hours internally) so very few candidates should go through this process if you decide to incorporate trials. You may filter out some good candidates because of the time commitment, but candidates who join are more likely to be successful.
  • References: I think that final candidates should be referenced checked by the hiring manager, especially if there are open questions. Backchannel references are the best (but avoid people at their current company) otherwise, ask the candidate for references. Here are some questions that I like:
    • How do you know the person? (gauge depth of relationship)
    • What are their strengths?
    • What are their areas for development?
    • What percentile would you put them in relative to similar folks in their position?
    • Would you hire them again?
  • Decision: For borderline candidates, the panel of interviewers should have a sync or async discussion – e.g. a private recruiting slack channel for hiring. The hiring manager is ultimately the decision maker. From start to finish, try and keep this process fast (e.g. under one month, and track the throughput).

Candidate experience

Candidates should have a great experience, understand how they are being evaluated and have consistent clear communication through the process.

  • Hiring Criteria: Candidates should understand the criteria by which they are being evaluated and the steps in your hiring process – this should be a templated email or a public blog post that you can send to product candidates.
  • Point of Contact: Candidates should have a clear point of contact (ideally the recruiter), to ask any questions about timelines and next steps.
  • Acceleration: If a candidate performs very well in early interviews or comes in through a trusted referral, they should be bumped up to the top of the queue or potentially skip steps so you don’t lose great people because of slow process.

How to Assess

When hiring, it’s important to be explicit about the skills you are looking for, and get a sense for where candidates are truly exceptional.

Here are the dimensions that I think you should use to assess candidates in the interview process:

  • Analytical Ability: AB Testing, Interpreting metrics, Data-informed decision making.
  • Product Judgment: System design, UX design to solve user / business problems.
  • Leadership: Inspiration, Influence, Empathy, Communication.
  • Execution: Prioritization, Getting things done when you say you will.
  • Technical Ability: Earn trust and respect from engineers as partners. Some roles will have a higher technical bar than others.

Each person on the interview team (3-5 people) should be responsible for evaluating the candidate along a subset of the interview criteria to create a balanced view. Ideally interviewers would ask the same questions to each candidate to calibrate their answers. I suggest that each interviewer test at least 2 dimensions of the list.

I suggest looking for candidates with an exceptional ‘A’ level strength, particularly in harder to learn skills like analytical ability and product judgement. I much prefer ABC candidates over BBB candidates because it’s possible to design complementary teams with AAA skills in aggregate.

Candidates should also demonstrate strong domain knowledge, and passion for the product, company and the customer. If they have prepared, it goes a long way (and it’s surprising how many candidates are ill prepared). If a candidate teaches me something new, or helps me challenge my own assumptions, that is wonderful. 

Appendix: Other resources

Google Criteria

  • Product Design: User experience, Design driven problem solving.
  • Analytical ability: Fluency with numbers, Using data to drive product decisions, dashboard design.
  • Technical ability: Understand technology and fundamental computer science principles.
  • Strategy: Go to market, Competitive analysis.
  • Culture: Googliness, Kindness, Leadership, Empathy.

Facebook Criteria

  • Leadership and Drive: Influence, Self-starting, Motivation, Persistence.
  • Execution: Goals, Metrics, Prioritization. Understand, Identify, Execute.
  • Product Sense: A design exercise to solve a specific user or business problem.
  • Engineering fit: Do engineers want to work with you?

Product Manager Articles

Here are a few articles about product management as an appendix, in case they are useful.