Naming things clearly

Here’s a memo I sent out to the engineering team this week:

Time for some real talk people.  We have a problem that we’re either not aware of or we’re tolerating.   I had to stop a meeting last month to address it.

A variety of stakeholders were getting a status report from a team, and it went something like this* (* names & details changed to protect the innocent):  “The Montesquieu project is blocked but the Montserrat project is complete.”  And we moved on.

I stopped the meeting and did a quick little quiz to see if people knew what the Montesquieu and Montserrat projects were. Let’s say that there was about 50% correct internal assignment of project name to project understanding for the non-team stakeholders.

As George Bernard Shaw once said, “The single biggest problem in communication is the illusion that it has taken place.”

I’ve seen this before.  People mix up the two project names.  Or they have no idea and don’t ask. Or they are actually thinking of that other project: Montagne!

It’s super fun to name things super fun, but it impedes our mission to help people get jobs.

Collaboration is how we accomplish great things together.  Rapid and clear communication is essential to collaboration.

Here’s why you should name projects as plainly as possible:

  • Recognition – when people hear the name, they need to quickly associate it with the project.
  • Recall – people need to be able to recall the project name when thinking about the project.

You should apply the ‘unprimed association’ test when you name something XYZ.  The trick is to ask a few people what they think of when they hear of project/product XYZ.  Code names fail this pretty quick. It doesn’t have to be perfect, it just has to be helpful and close.

In the Incubator we established a rule to name projects as plain and clear as possible, because we needed our busy executives (SLT) to understand them and keep them clear in their heads.  That’s why we named them things like: ReferralsHiring Events, Chat, Guaranteed Response, Text2Apply, Interview Now, etc.

Imagine if they had accidentally mixed them up when conversing about funding (“I think we should fund Montesquieu and not fund Montserrat”).  Yikes.

Here I’m going to quote from the Harvard Business Review: “Use words that will resonate with those whose support and influence you must earn. If they can’t follow your ideas, they won’t adopt them” (hbr.og)

I’m going to give some examples – not to pick on them, but because they merely representative – and suggest alternatives:

  • Itasca =>  Resume Bank
  • Iris =>  Native App Vision
  • Captain Planet => Jobs-on-HP
  • Hodor => MatchFilterService
  • Beaker => AggFeedStore

The plain alternatives are longer, and in practise people might adopt acronyms or shortened versions.  That is ok. I will assert (without evidence) that acronyms are better at recognition and recall than code-names – when people use acronymed names (like Amazon SES), internally the acronym and the expanded-form (Simple Email Service) are connected in the brain.  But my brain simply doesn’t have any neurons connecting the name Itasca with the concept of a resume bank.

Harder to recall/disambiguate (all real project/product/tool names at Indeed): Wendy, Wenda, Waldo, Waldorf, Basestar, Deathstar, Stargate, Goldengate, Gate, Iris, Aurora, Orion, Janus, Philae, Curious George, Yellow Hat, Yamanote, Peregrine, Naboo, Nigma, Hodor, Horton, etc.

Easier to recall/disambiguate: Session Viewer, Query Manager, ITA Manager, Job Metadata Viewer, AdCentral, AggCentral, User Personalization Service, Notification Preferences Service, TitleNorm, Labeler, JobSearchService, TestStats, Salary Transparency, etc.

I feel like a grouchy old person as I write this.  I’m fun. Really, I am. But please stop having so much fun with your names.  This is not a mandate. This is just Brendan sharing his belief that we should use boring, simple, clear, recallable and recognizable names for our projects, services and tools.

Posted in Uncategorized | Leave a comment

Inconvenient Truths about Operating Mechanisms

Operating Mechanism is a fancy term for processes (and the corresponding tools) used to run an organization.  Examples of operating mechanisms are:

  • The quarterly talent review process
  • The developer interview process
  • Weekly product review
  • The growth-board / project-funding process

These are necessary to help the business do its work effectively.  Each operating mechanism has its own purpose (which should be clear to those involved – but sometimes isn’t).

After participating in running, changing and building a variety of operating mechanisms, I wanted to share my observations.  These are some inconvenient truths according to me:

1. Non-compliance is real

People ignore emails and instructions.  Things that are too complicated don’t get done.  People go on vacation. Higher priority items crowd out lower priority items.

I’ve seen people design operating mechanisms without understanding the need to be simple / fast / valuable.  People don’t comply well with mandates – especially if they don’t understand the why of it.

Also, people are forgetful.  If you put an operating mechanism in place that requires people to remember something (e.g. “please remember to tag these 3 random people in every ticket that affects some subgroup”) it will not happen.

And so you need to think about how the operating mechanism handles non-compliance.  Does the whole thing fall apart if someone doesn’t do their part?  Or is it more resilient – does it simply degrade? Interview accuracy degrades if you have to cancel an interview.  Approval chains grind to a halt if there isn’t a way to delegate or escalate.

2. Operating mechanisms operate within constraints

Almost every step in an operating mechanism operates under a time or people constraint.  We are always doing the best we can within the limits. It could always be better if we gave it more.  E.g. more peer-feedback during reviews, more time during calibrations, another phone screen before an onsite, more time to discuss in loupe, etc.  

Building an operating mechanism means using judgement about the diminishing returns from doing more.

3. You can’t A/B test an operating mechanism

When discussing an operating mechanism, someone at Indeed will inevitably propose that we A/B test it.  Usually accompanied by a wink 😉

You can’t A/B test an operating mechanism, because you can’t do random assignment and/or get enough power and/or align on metrics.  The closest you can do is have different groups pilot or try different approaches, then compare. Most often you evaluate and improve operating mechanisms by cycles of feedback, assessment/judgement, modification.  There are business studies that you can read about, but it can be hard to know if they are really valid, and if they apply to your particular circumstance. But studies can often inform various adjustments to try.

4. There will always be problems and complaints

It has been said that democracy is the worst form of Government except for all those other forms that have been tried from time to time” — Churchill

Operating mechanisms don’t get much love.  The best they can hope for is some respect.  But there is no such thing as perfection – where nobody complains.

To keep things efficient and fruitful, if you are discussing operating mechanisms, keep complaining to a minimum and stick to discussing concrete proposals for change.  Keep it real. Abstract discussions are rarely helpful.

5. Organizations outgrow their current operating mechanisms regularly

What got you here won’t get you there.  Things change. 

There will always be a sense that things are broken.  This is no excuse for not fixing things. It’s an explanation for those who are surprised that at fast growing companies like Indeed the operating mechanisms aren’t running perfectly smooth.  That’s because circumstances have and continue to change rapidly. It can be hard to get ahead of exponential growth.

So what should you do about these?

Here are tips for working with operating mechanisms:

  • Start with why.  Clearly and repeatedly articulate the purpose of the operating mechanism, whenever you first create it, and when people are proposing changes.

  • Remind everyone of the context.  Operating mechanisms are temporary, imperfect structures designed to accomplish goals in a changing environment.  Feedback, discussion and iteration/change are necessary part of the process. Remind everyone regularly.

  • Organize around written proposals.  Discussions around operating mechanisms can get abstract, unclear and wasteful.  Try as best as possible to stick to gathering feedback, then discussing written proposals.  To respect everyone’s time, focus on written proposals for change – there’s no point discussing a criticism if it isn’t accompanied with some suggested change.

 

 

 

Posted in Uncategorized | 1 Comment

Successful Second Products are Rare

The odds of developing a second successful product – that is commensurate with a first successful product – are low.

As someone who works on the product exploration side (a.k.a. launching new products) vs the exploitation side (a.k.a. optimizing and improving existing products) I am continually on the lookout for companies that are able to pull off this challenging feat.

It was heartening to hear other people acknowledge how rare this is.  On the Acquired.fm podcast covering PowerPoint they touched upon this:

Ben: I pitched you on two podcast ideas. One was, we should cover acquisitions that actually went well, because the trope is that none of them ever go well and they implode. The other one was, why don’t we do a podcast series where every episode is about the rare company that manages to have a billion-dollar-plus innovation twice.

David: That’s right. I remember that. And we decided not to do that because there just weren’t that many. It would be a very, very short podcast series.

Ben: Yeah. And I think our thesis was basically like, companies have the magic moment just once, because they’re so rare, that they find that product-market fit, the monetization, like just everything works. And then after that, companies keep finding ways to grow and expand that thing, but they generally fail to find a second complete one of those and particularly one that works under the same roof and doesn’t interfere with the other main business. Microsoft did it. They’re one of the few in the world.

Posted in Uncategorized | Leave a comment

How Can I Help?

One of the best business lessons I ever learned was at summer camp.  When I was sixteen I worked for a summer as a lifeguard and sailing instructor at a family camp in Ontario, Canada.  Families would attend the camp for a week – sleeping in basic wooden cabins, dining together in an old wooden dining hall, and doing campy activities like swimming, boating, and playing badminton.

The staff was comprised of sixteen and seventeen year olds – twenty four girls and twenty four boys – supervised by a dozen university students.  It was a fantastic summer.  Extremely fun, but also a lot of work.

A week before the camp officially opened to campers, the staff would arrive for training.  One of the key lessons was about customer service.  More specifically, we were taught to ask “How can I help you?” then try to help.

That lesson has stayed with me since.

Screen Shot 2018-05-19 at 2.36.13 PM

* names in these chat screenshots were changed to preserve anonymity

It’s been more than 20 years, and now my job is very different.  But I try to retain a default response whenever someone reaches out: “How can I help?”

Screen Shot 2018-05-19 at 2.45.56 PM

This can be hard when I feel like I’m swamped with work, and I’m not looking for an additional meeting, report, presentation or last-minute interview with a candidate.

Screen Shot 2018-05-19 at 2.39.56 PM

As I’ve become busier and responsible for a larger organization, this has meant that I’ve had to learn to delegate and say no a lot more.  But I still carry the lesson I learned at sixteen – to start the conversation with “How can I help?” and then to try.

Posted in Uncategorized | Leave a comment

Gmail wasn’t really a 20%-time project

I always thought gmail was an example of an employee coming up with an idea, and having automatic-permission to explore it due to Google’s 20% time.  It is constantly referenced in articles about the policy.

But there is some nuance here.  It turns out Larry Page asked Paul Buchheit to explore email:

Larry Page assigned the project to me. He said, “Build some kind of email something” and chose me because I had an interest in email. I tried to create a web-based email service in 1996, shortly before Hotmail, but didn’t complete it.

Interesting.  This takes away one powerful piece of evidence often used to justify policies that enable employees to pursue their own innovative ideas on company time.  In this case there was an executive sponsor right from the beginning.

[update 10/19/2018]  Paul Buccheit addressed this again:

As Google grew, did you keep the freedom to hack on things you found interesting? Was creating Gmail part of that?

Around 2001, Larry got frustrated that each group was setting their own priorities and not working on what he thought were the most strategically important things for the company. His fix was to eliminate management and organize engineering around specific projects. He and Wayne Rosing, who was the VP of Engineering at the time, would sit down with engineers and give them projects. When they sat down with me they said, “we want you to build an email something.” That was all the specification I got! So I went off to build something with email, which became Gmail.

Posted in Uncategorized | Leave a comment

Criteria for continued funding of new products

At Indeed we’ve been working on continually improving our processes for funding, staffing, executing and assessing new experimental products.

One of our challenges is assessing early products and figuring out whether to kill or continue them.  One approach used in the past has been ‘make-or-break’ goals.  This has typically been represented as a top line goal to reach at a point in time, such as: X hires, Y sales, etc.

Over time we have learned that these make-or-break goals do not provide enough flexibility for the product team and also do not provide enough flexibility for Indeed.  In certain circumstances the team might make the goal – but Indeed leadership is simply convinced that Indeed’s limited resources to test out new products should be spent elsewhere.  Conversely the team might not make the goal – but leadership is convinced that this was the wrong goal, and that further investment is warranted.  So we are no longer doing make-or-break goals.

What we are trying instead, is to use a metered-funding approach, where an investment committee provides funding in stages – and the project teams have autonomy to use their funding and pursue their vision, but when the money is running out they need to return to the investment committee for more funding.

So how does the investment committee decide if projects get further funding?  Here’s what I’ve been coaching Incubator teams with:

“For projects to continue they need to provide compelling evidence that is surprisingly positive that shows their product can produce sustainable value in a meaningful market.”

Let’s pick this apart:

  • Compelling Evidence – Bring verifiable data and evidence.  No evidence-based-learning means no more funding.
  • Surprisingly Positive – Sets the bar high, if there aren’t aspects of the experimental product that are working surprisingly well we should probably put our investments elsewhere.  Most projects will not find something surprisingly positive.
  • Sustainable Value – There is confidence that the product can achieve sustainability.  We don’t want to launch new products that are simply re-packaging our sponsored job advertising business into a different model and selling it for a loss.  Teams need to understand the cost of acquiring users, the cost of servicing them, and how these are trending.
  • Meaningful Market – We believe this market could be sizeable enough or strategic enough to be important to Indeed.

There is room for interpretation with these and over time we will provide more clarity and guidance, with examples.

By continually experimenting with a portfolio of new products, Indeed should be able to compare projects side-by-side, and invest more in the ones that better meet this criteria (and kill projects that fail to meet this criteria).

 

Posted in Uncategorized | Leave a comment

Seeds, Pods and Big Bets

Indeed is investing more and more in discovering and building new products.  As the product team discussed and evaluated our successes and failures – I found it difficult to communicate without some common terminology around the different types of innovation projects underway.

So here are a couple of terms I proposed that were intended to help the discussion.  In all cases I’m talking about trying to find new successful products – new sources of revenue that are pre product-market fit.  Initiatives to grow core business are outside of this discussion.

Seeds: Small projects; less than one quarter long; pursuing a solution to an opportunity; developing and piloting an MVP solution; gather some initial benchmark performance data.

Pods: Medium size projects; 3-4 quarters long; pursuing solutions to an opportunity; enough time to gather benchmark KPIs, to run some tests to see how the KPIs improve, and to try a few different approaches to the product.

Big Bets: Large projects; 6+ quarters long; strong commitment to a new business opportunity; try to launch and scale it quickly.

Projects of any size that have found product-market fit are no longer seeds, pods or big bets.  They’re new products and they need to be scaled as large and as efficiently as possible.  Also, these are not strictly stages.  These are different approaches to product discovery and development.

 

Seeds Pods Big Bets
Timeline 1 quarter 3-4 quarters 6-8 quarters
# of Engineers 1 engineer 2-3 engineers 3-5 engineers
Marketing Budget ~$15k $50k – $500k / qtr $250k – $2M / qtr
Purpose Pilot a specific solution to a specific problem.  Clarify an opportunity. Pursue a solution to a specific problem.  Some flexibility to pivot as the project proceeds. You believe you know the solution to a problem.  Go big and launch it.
Goals Go from zero to one. Quarterly goals.  A modest level of initial performance constrained by cost.  e.g. 10 hires at <$500/hire.  Improving significantly each quarter. Quarterly goals.  Showing growth, and performance improvement, constrained by cost.
Approvals Required None Senior Leadership approval Senior Leadership sponsor
Business Model Not established Establishing Should be clear

 

What are they good for?

Seeds

Seeds are useful because they bring an idea to life.  It’s fine to research an idea – like creating a site dedicated to jobseeker events – but the requirement to actually pilot something adds a level of fidelity to it.  It’s way easier to engage with something tangible – to both poke holes in the idea, and to see the full potential.  Show, don’t tell.  The nice thing about seeds is that they don’t need Senior Leadership support.  You can – and should – support seed projects that few people would bet on.

Seeds also support the creation of pods and/or big bets.  Both pods and big bets are a substantial investment of limited resources (product, engineering, sales).  Starting with a seed is a way to get some clarity and hopefully some real data before committing to a pod or big bet.

You should expect to stop after the seed phase and then make a decision about what the next step is.  That might mean putting the seed idea on the shelf.  It might mean gathering the resources to create a pod.  But it’s a mistake to continue at the seed level too long.  They need more resources (engineering, marketing) to have much of a chance of real success.

Pods

Pods are like a small funded startup with a year of runway.  They have a problem identified, they are building and launching a product.  Their goal is to find product-market fit, or to show a clear trajectory on the KPIs towards product market fit – before the $ runs out.  Lean startup approaches to product discovery and validation apply here.

For pods to be successful they require a certain amount of investment.  They require 2-3 engineers.  They require product marketing support – with a budget.  They might require some integration with other products.  And they require some Senior Leadership support to help secure resources.

Big Bets

If you think you’re onto something, if you’re committed to it – then go big.  Pros: bigger product faster.  Cons: what if you don’t have product-market fit?  Then you’re burning lots of resources as you try to find it.

Big bets require an engaged executive sponsor because of the significant investment involved.

Ok, I understand the proposed terminology, so what?

Companies should be pursuing a mix of these concurrently.  They all have a role to play.   You should make it easy for entrepreneurs in the org to pursue a seed project they’re excited about.  This could be an innovation rotation, a labs/skunkworks project, or the permission to launch a nights-and-weekends project.

I would propose that pods are one of the most efficient ways to find product-market fit; enough resources to actually try something out and learn, but with limited cost to the company.

Finally – You shouldn’t be afraid to place a limited number big bets on the most promising opportunities in your space.  Bigger risks but bigger rewards.

Posted in Uncategorized | Leave a comment