When we were a twenty-person company in 2011, General Assembly embraced Objectives and Key Results (OKRs) to align everyone’s goals and increase transparency. Formulated at Intel and Google, OKRs are a useful management framework popular at growing tech companies.
As some of you may know, I like finding ways to apply management techniques to my personal life. It’s a useful quirk. For instance, our quarterly planning sessions with my wife Amanda – think of it as board meetings for our relationship – have moved conversations about potentially stressful topics like personal finance from the dinner table to a structured, dedicated session.
But lately I’ve gone a step further, using OKRs to set and monitor my personal goals. And it’s awesome. The combination of high-level, qualitative Objectives with measurable, quantitative Key Results balances the inspirational with the tactical. Here’s an example of one of my personal OKRs for this quarter:
Objective: Be healthier than I’ve ever been
- Go to the gym 3x per week (30 total visits in the quarter)
- Fewer than three fast food meals in the quarter
- Resting heart rate under 75 bpm by EOQ
As you can see, each high-level objective is tied to multiple measurable key results. A good set of OKRs generally has 2-4 Objectives and 2-4 Key Results per Objective.
When writing good OKRs, nuance is key. Here are four guidelines I’ve found useful:
Key Results are Not a Task List. It’s natural for us to make lists of things we need to accomplish – mow the lawn, take out the trash, get groceries. Those aren’t Key Results. Key Results are how we measure the success or failure of what we set out to do. Avoid key results that seem like tasks rather than measurements of accomplishment.
Write inspiring objectives. Objectives don’t have to be measurable, but they should inspire you – or your team – to drive for a common goal. “Be healthier than I’ve ever been” is a strong, clear objective.
Write measurable and indisputable key results. Key results are how you measure progress against your Objective. They should be quantitative and not open to interpretation. In this case, if I achieve all three key results — 30 visits the the gym, fewer than three fast food meals, and a resting heart rate under 75bpm — I will consider my Objective achieved.
Set Stretch Goals. Many veterans of OKRs say that you should be pleased to accomplish 70% of your Key Results. If you do better than that, you’re either a master of management or you should set more aggressive goals. Hitting fewer than 50% of your Key Results should be a warning sign.
So why does this work? Humans long for the ability to claim success and celebrate accomplishment. Without clear goals, it’s never going to be clear whether you’ve achieved what you set out to accomplish. OKRs are a way to frame and define accomplishment.
I set personal Objectives and Key Results on a quarterly basis. The time period is really a matter of taste. I’ve found yearly objectives to be too long — too much changes in a year, and it’s disheartening to find that your goals are no longer relevant — and monthly objectives to be too short to make meaningful impacts on your life.
This is one system of many. Regardless of the framework you use, set goals. You can’t decide on a path until you know where you’re going.
In my brief time thus far as a VC, I’ve seen entrepreneurs repeatedly screw up one key metric again and again: customer acquisition cost (CAC).
Along with lifetime value (LTV), CAC is one of the most important metrics consumer-facing businesses need to watch. But so many of them get it wrong and do so in a very dangerous way.
The entrepreneurs who get it wrong calculate CAC as follows:
CAC = Advertising Spend / Total New Customers
CAC = Advertising Spend / Total New Customers Attributable to Paid Advertising
Counting all your customers in your CAC metric – rather than just those acquired through paid marketing – is playing with fire. As an entrepreneur, I learned that the worst possible result of a test is a false positive: a result that incorrectly makes a hypothesis look correct when it actually doesn’t work. False positives will waste time, money and precious focus.
Calculating your CAC as a “blend” of all customers is just asking to be on the receiving end of a false positive. Your paid campaigns could very well be unprofitable, but you are making them appear profitable by blending in customers who found your product through organic channels – referral, word-of-mouth, organic search, or social, to name a few.
This is a huge problem since traffic from those organic channels doesn’t scale linearly with traffic from paid channels as a paid advertising budget increases. If you are spending (say) $1,000 on paid marketing and getting 20 new customers per day, it does not follow that your CAC is $50. 5 of those customers may be from paid marketing campaigns while the other 15 are from organic sources, so your CAC is actually $200. When you scale your paid budget to $2,000, you won’t be getting 40 customers per day – only 25 at best. And that’s assuming that CAC stays constant as your budget increases, which is a very optimistic assumption.
Using “blended CAC” in a presentation is a very quick way to get me to pass on an investment. When I hear a pitch, I care less about whether or not a company’s initial paid marketing tests were profitable or not. There are numerous factors that can drive campaign profitability, and there are a lot of ways to acquire customers these days. Rather, I care about how an entrepreneur uses data and thinks about key metrics. Showing a blended CAC means an entrepreneur is loose with the numbers at best and deceptive at worst. And as a company grows and the numbers get bigger, either of those are very bad things.
Entrepreneurial narratives are everywhere. From executive education classes to TED to General Assembly’s own enterprise programs, it’s not difficult to find the story of a successful entrepreneur in almost any industry. These stories inspire not just aspiring founders, but also innovators within Fortune 500 companies and creative agencies.
But what are we to take from those stories? How does anyone – even someone within a larger organization – draw meaningful, applicable lessons from an entrepreneur’s first-person account of success? After all, there’s a huge element of randomness in any new venture. For example, Chad Hurley and the founders of YouTube made a lot of great decisions, but they weren’t the first team of smart people to build an online video site. In the words of investor and entrepreneur Chris Dixon:
“I’d been around the web long enough to remember the dozens of companies before YouTube that tried to create crowdsourced video sites and failed. […] YouTube built a great product, but, more importantly, got the market timing just right.”
But every stock trader knows that timing markets can be impossibly hard, so taking lessons from YouTube’s success is not only difficult, it’s dangerous. Drawing the obvious but – in many cases wrong – conclusions from a entrepreneurial success story can lead to cargo cult thinking, wasted money and lost time.
Applying lessons from entrepreneurship within larger companies can be even more challenging. Innovators within large institutions and brands know that bringing new ideas to the table isn’t enough. In addition, they must overcome organizational inertia and traditional mindsets to effect change.
So how do we learn from these stories? Here are five tips from my experience:
1. Failure is a data point too. Many people focus only on entrepreneurial successes, when in fact just as much can be learned from the much-more-numerous failures. Learning from others’ failures is an efficient way to ensure you don’t repeat them. And keep in mind that even within successful companies there are always numerous products, initiatives, and hires that failed. Studying how great teams fail and react to failure is one of my favorite topics of conversation with entrepreneurs.
2. Appreciate the details. Entrepreneurship is a game of finer points, not broad strokes. Success is often the product of many small decisions on topics that can range from landing page design to company culture to early brand decisions. A deep discussion of these topics – and how entrepreneurs came to key decisions – can be far more informative and interesting than broad questions like “How did you come up with the idea?”
3. Meet the lieutenants. It’s practically a cliche to say that an entrepreneur should surround him- or herself with great people. But it’s also fundamentally true. Great entrepreneurs recruit great managers, and many of the critical decisions in a successful company’s history are made by non-founder operators, not the founders themselves. Often these operators know how to wield “soft power” in a way that is much more applicable to innovators within large organizations who don’t have a founder’s moral authority.
4. Understand persuasion. At the earliest stages, entrepreneurs need to convince others to see things their way, to ignore the obvious pitfalls and believe in a vision. Without that power of persuasion, it is impossibly difficult to find investors, recruit early hires, and close clients. Whether starting a new company or innovating within an existing organization, those abilities are universally applicable and one of the best practices to learn from successful entrepreneurs. When watching an entrepreneur speak, it can be more informative to pay attention to how they are saying something than what they are saying.
5. Metrics, metrics, metrics. Great entrepreneurs set key metrics closely tied to their success and drive their teams toward those metrics. Understanding how entrepreneurs chose the right – or wrong – metrics and incentivized their teams to pursue those metrics can paint a detailed picture of how a company’s operations, finances, and culture are all integrated. And like the other tips here, how a startup sets metrics should be very similar to how a department within a large company sets them, making the lessons very applicable across organizations. While the methodologies used to succeed may shift as a company grows, the key metrics should not.
I believe that anyone can learn from an entrepreneur’s story regardless of their role. With these pointers in mind, entrepreneurial stories can be vastly more educational and applicable.
Building General Assembly has been the most remarkable and inspiring professional experience of my life. We have helped more than 12,000 people pursue careers they love across 12 campuses worldwide and have been a part of some powerful stories along the way. When we set out to teach technology, business and design, we didn’t realize that we were going to create the first premium education brand to emerge in more than a century. That is truly humbling.
The GA team now comprises 400 of some of the smartest, most genuine people I know. We have seen incredible and talented individuals rise through our organization as first-time managers, while others have joined the senior team bringing decades of experience in education and leadership. I have immense faith in my co-founder Jake and the senior management team to guide GA through the next part of this journey.
Four and a half years after making the decision to start General Assembly with my co-founders Jake, Matt and Adam, I am excited to say that I am joining our lead investor Maveron as a venture partner and returning to my passion for early stage companies. I am not leaving entirely — I will be staying on part-time to help build our Credentialing Network as we develop a team around that critical initiative.
Maveron has been the ideal partner since making a bet on us three years ago, and I’m thrilled to join their team. As I’ve gotten to know Jason, Dan and all of Maveron, I’ve been impressed with their vision of building a truly consumer-only venture firm with deep expertise in product development, brand, and customer acquisition. And they’ve been able to execute on that vision with early bets on Zulily, Julep, and General Assembly, among others. As the home of great brands and consumer-savvy entrepreneurs, New York is a wonderful place to expand the Maveron family.
For most of my life, I viewed the education system as an institution immune to change. After building General Assembly, I see the power of communities of individuals challenging established systems and building better outcomes. GA’s growth is not a black swan event but rather a story that we will see with increasing frequency as entrenched industries feel the impacts of technology and design. This shift will be an incredible force for good, as sectors that have been traditionally slow to embrace technology — such as education, healthcare, and finance — are the largest in the world and touch every individual.
I have had the privilege of spending my life so far enabling that story by being an entrepreneur, and I hope to continue to do so. My co-founders and I are doubly fortunate that General Assembly is not just another company, but an organization that has had a transformational positive impact on so many lives. For that, I am incredibly thankful but remain hungry for more.
Remember when gaming was one of the hottest verticals for venture investment? Four years ago, you couldn’t visit TechCrunch without reading about the latest social-mobile studio or game with dizzying stats.
No more. Zynga has been battered, traditional publishers never recovered, and new super-casual studios struggle to get traction in the public markets. Even companies riding the freemium wave like King are trading at 3-5X EBITDA multiples, numbers on par with industries like facilities management and for-profit education. Multiples that would’ve been unthinkable in 2010.
But there’s something going on in gaming right now, and it centers around four inflection points:
1. Casual game distribution is getting more expensive… way more expensive. CPIs (cost per installs) have risen by 56% on iOS in the year with no signs of slowing. The app marketplace is an oligopoly, dominated by a few big players that leverage owned channels — their successful games — to push new releases to the top.
Getting in the top 25 free apps — a critical channel to reaching a mainstream player base — has gone from expensive to borderline-impossible in the past two years. There are over 2,000 new apps released every day, the majority of them games. And most of them will generate little to no interest or revenue.
Given this, it’s not surprising that…
2. Casual game monetization is a race to the bottom. Mobile has always been a cesspool of questionable tactics and sleazy marketing. But the increased cost of distribution has meant that games need to resort to increasingly aggressive monetization strategies to keep LTV in line with CPI. Those who don’t are locked out of distribution channels. It’s a classic race to the bottom, with little incentive to favor user experience over monetization when targeting casual players.
It’s started to get regulatory attention and will only be so long before many of these tactics go the way of $9.99/month ringtone subscriptions.
3. AAA-quality game development is cheaper… way cheaper. Unity is absolutely dominating indie game development, and for good reason: it’s a flexible tool that allows small development teams to build high-fidelity games with orders of magnitude smaller budgets than were required just five years ago.
Other tools like Corona are playing a part too, but Unity is the clear leader here and is changing the way immersive gaming experiences are built.
4. Niche core and hardcore game distribution is easier than ever. Steam Greenlight has enabled games that never would have gotten mainstream coverage — let alone made it onto a Gamestop shelf — get in front of the right gamers at a fraction of the cost of past distribution channels.
Furthermore, core and hardcore games will get unique opportunities on new platforms (Oculus) and are finding new ways to reach savvier players who know are sick of the race to the bottom on mobile.
There may not be a venture-fundable, Zynga-scale model here — most of the success has been confined to hit-driven indies to date — but the landscape is changing.
Since the Renaissance, scientific experiments were the domain of Science. And Science had its own way of doing things. Science derived questions from first principles, proposed hypotheses that may answer those questions, and designed the methodologies that would prove or disprove those hypotheses.
That’s how Science worked, and nothing else worked quite like it. Business was art; Business was Don Draper doodling his visions on a cocktail napkin. Politics was even further afield, an art buried in the fog of war. Science and Science alone ran real experiments.
But then some Bad Things were done in the name of Science. Things that shook the foundations of Science and threatened to bring down the whole operation. To prevent those Bad Things from happening again, Science began regulating itself, introducing concepts like Informed Consent and requiring experiments to be approved by Institutional Review Boards. Most people agree that those changes were for the better.
But then other people started figuring out what Science had been doing all this time. Finance was probably the first to find it, bringing experimental methodologies to trading in the late ’80s and early ’90s. Advertising soon followed suit and has become overwhelmingly sophisticated in the past ten years. Obama’s 2008 campaign was a watershed moment for Politics, with rigorous message testing and voter data methodologies at work for the first time.
One by one, domains of Business and Politics adopted the Scientific Method. Their subjects, naturally, were human beings. But since Science only regulated itself – not everyone who used its methods – these new domains don’t have the same procedures and oversights. Any organization with an audience could run experiments on them to see what message, user flow, landing page, ad campaign or button color worked the best. Some of these experiments could have a meaningful and lasting impact on their subjects – when an OkCupid test works, more people hook up, resulting in marriage, kids, STDs, emotional trauma and all kinds of chronic effects that would put any IRB into a tizzy.
And now Facebook is getting FTC attention for doing something every savvy web-based company has done in the past fifteen years. And OkCupid and others are defending them, claiming (correctly) that everyone does it.
Now everyone doing it doesn’t make it okay, per se. But it does raise questions of what “informed consent” means. One could argue that signing up for Facebook and accepting friend requests is consenting to seeing whatever those friends may post, even if Facebook is presenting them in a way that is likely to evoke certain emotions. And the informed consent argument is even stronger with OkCupid; they’re not trying to hide the purpose of their site from prospective users.
Of course, the specific methodologies of informed consent aren’t anywhere in play. There’s no paperwork or review, just a ToS that no one ever reads.
Perhaps the problem here is not that these companies are running experiments without informed consent, but that the implementation of informed consent needs to be totally re-thought when applied to Business and Politics. Signed forms and IRBs aren’t going to work – we need a methodology that fits companies that run thousands of experiments every day and test hundreds of variables, the vast majority of which have little to no lasting impact on their users.
Perhaps rather than going after Facebook, informed consent advocates would be better off tackling the issues with ToS and getting companies (and users) to care about the documentation that already exists rather than adapting new business practices to an old scheme that was created to solve different problems.
Last night I had to drive from Albany to New York. My Virgin America flight from San Francisco got diverted, then had mechanical problems, and next thing I know I’m facing the prospect of spending Saturday night in the Schenectady Hampton Inn.
With the good fortune of sitting near the front of the plane, I was second in line at the Alamo counter at the Albany airport to rent a car to make the two-and-a-half-hour drive back to the city. I was feeling pretty good about my prospects. The man in front of me rented a mid-size for $80. When I got to the counter and handed over my license, a troubled look came over the representative’s face.
“Oh wow — everything just changed.”
“The prices. The computers saw what happened with your flight. It’s a lot different now.”
Alamo ended up charging me $600 for the pleasure of driving a rental car back to New York. I was fortunate to find some fellow passengers to split it with me — company on the road is always nice too — but I still have a bad taste in my mouth. It was an example of price gouging at its worst.
Uber has taken a lot of flak over practices that look similar at a high level. Uber’s algorithms see surges in demand and raise prices accordingly. Seems like old-school price gouging, the same kind that Alamo used against me in Albany.
But while I’m fuming at Alamo, Uber’s practices have never bothered me. It really comes down to the elasticity of supply: Regardless of how much Alamo charged, they weren’t getting more rental cars to Albany that night. The first dozen people at the desk got cars and the rest were left with their hotel vouchers and the hope of a flight the next day. The increase from $80 to $600 per car was simply a maximization of profit — an economic transfer from the customer to the company. No additional economic value was created by the price increase.
Uber, on the other hand, has supply elasticity. That is, supply changes quickly with the price offered. If a customer is willing to pay $200 to get from the Upper East Side to the Meatpacking on a rainy Friday night, there’s going to be a driver willing to do it. Increasing prices brings more drivers out, making it possible for more transactions to happen that “make sense” for both parties — even if those transactions are significantly more expensive than they would be on a sunny summer day.
Price gouging laws were built with the assumption of totally inelastic supply. Price gouging laws exist to prevent Alamo from doing the kind of thing it did in Albany on Saturday night, or to prevent a grocery store from increasing the price of bread during a blizzard when more bread can’t be brought in. When these laws were written, inelastic supply was the rule. Logistic and supply systems didn’t exist on top of a digital layer enabling real-time changes in supply in response to demand. But for some companies like Alamo, that digital layer still isn’t used to increase supply — just to raise prices.
I’m glad I got back to the city on Saturday night. But it left me wondering why all the flak has landed on Uber for ensuring that supply can meet demand while Alamo is left to gouge and extract at will.
It’s been exactly 60 years since Peter Drucker published The Practice of Management, bringing professional people management into the world of social science. And over the last twenty years, it’s gotten cheaper and easier to build a company, companies that are more often led by non-traditional managers with more leeway to try new things. Together, these forces are leading us into a golden age of people management.
In the history of organized work there has never been such a diversity of management tactics and structures. From holacracies to adhocracies to flat structures to four day weeks, entrepreneurs and managers are trying new ways to get things done and keep people happy at a faster rate than ever before. Just look at the headlines on Hacker News — you’re almost always able to find a couple front-page pieces on a new org chart or management hack.
I don’t agree with many of these tactics. Most won’t work. But some will, and aspects from others will get incorporated into more traditional management canon.
There’s been a lot of negativity floating around the tech world these days. This post isn’t about that. But it is about always seeking to do things better and iteratively improving the world around us, and in that spirit I’m optimistic.
In their December newsletter, University Ventures declared Bridgepoint’s licensing of Forbes’ brand to create the Forbes School of Business to be education’s “Mickey Mouse moment”. The letter compared the deal to Walt Disney’s creation of high-quality, branded theme park experiences that out-competed the questionable and unreliable carnies of decades past.
Brand crossovers in education have happened and will continue to happen. But this trend won’t result in high-quality experiences any more than blockbuster movie crossovers create high-quality video games.
From the letter:
For in a world where universities take centuries to establish themselves and their brands, there are myriad brands that currently have nothing to do with higher education but that could be very useful for higher education institutions interested in developing high quality programs that produce higher ROI for students and that clearly communicate those values to the marketplace. These brands could provide prospective students with shorthand information about the value proposition in a market where prospective students are crying out for better information about the likely return on their investment in a degree program.
Unfortunately, “quality” and “value proposition” are not unidimensional metrics that translate across industries and products. Brand crossovers in this industry have made a splash when education meets the self-help aisle but rarely beyond this. Think Tony Robbins and the Learning Annex; The Apprentice and Trump University. In those cases, students are looking not for a long-term skill set but for inspiration and a silver bullet, for which brands that communicate “immediate success” are highly valuable.
Unfortunately, licensed brands are often used as a stand-in for product quality rather than an indicator of it. The games industry offers a cautionary tale. For years, TV/movie/event IP was licensed to create new game titles, generating a bunch of famously terrible games.
Eventually, the tide turned. Games journalists looked for reasons to attack the next big-budget shovelware, framing the entire category as corporate garbage. Today, the entertainment-crossover-into-games genre is effectively dead, with interactive studios investing their biggest budgets in game-specific IP rather than licensed brands.
Education offers an even harsher environment for brand crossovers. Unlike games, education is typically viewed as an investment rather than consumption expense. Customers — especially those precious 10% whose tickets aren’t subsidized by the federal government — are suspicious and discerning. And unlike the late-90s gaming industry, the education space features hundreds of existing brands specialized in a variety of learning outcomes.
There’s value to the Forbes/Bridgepoint deal and think it will quite likely work out well for all parties involved. But brand crossovers are hardly a new thing, and it’s far too early to claim a watershed moment. The for-profit education theme park is still run by the carnies.
One distinguishing factor between inexperienced and senior software developers is their ability to answer the following question about an application they’ve built:
“As usage increases, what’s the first point of failure?”
Knowing which query or resource will need to be rewritten or replaced soonest is a meaningful — albeit simplistic — indicator that an engineer understands what he or she has built.
I like to apply the same heuristic to entrepreneurs. Experienced founders can be differentiated from novices by their ability to answer the following:
“As the business grows, what’s the first bottleneck you’ll encounter?”
Or stated another way, what problems will be need to be solved at each stage of the business’s growth? Let’s take a child’s lemonade stand. In this example, we have a particularly enterprising child who notices that the local supermarket can only supply 100 lemons per day, which equates to approximately 200 glasses of lemonade. At $1 per glass, that’s only $200 per day! The child thinks, “If I want to grow my business beyond that size, I need to buy wholesale” and begins spending time researching suppliers. But in this case, supply may not be the primary bottleneck. If only 100 cars come down the stand’s street every day — and only a percentage of those buy lemonade — s/he doesn’t need to find a wholesale supplier before finding a new location, or perhaps training an employee and opening a second stand.
It’s useful for an entrepreneur to think about their job less as growth and more as de-risking — identifying and working through the primary risks of the business. This is especially true if the entrepreneur wants to raise money. For the lemonade stand, the ability to buy wholesale isn’t a major risk — many small businesses do that with ease. Growing the customer base and scaling to multiple locations, on the other hand, is a problem worth tackling.
“Scaling prematurely”, “hiring ahead of the curve”, and “capital inefficient” are all phrases that VCs use to describe companies that tackle the wrong issues in their business’s growth. Entrepreneurs rarely err by implementing poor solutions to the right problems but rather by solving the wrong problems.
By this point we all know things are bad in PIGS. But how bad, and why? Matthew O’Brien at The Atlantic included a simple graph in his argument that Spain is Beyond Doomed, charting the growth the long-term unemployed in Spain:
Anyone who has tried to do business in continental Europe — particularly Spain — knows that firing employees is effectively impossible. This can make businesses much more skittish when it comes to hiring full-time employees in boom times, favoring instead unprotected part-time contract labor.
And when I saw that chart, I was reminded of another:
This one isn’t as dramatic, but the underlying problem is the same: a regime that protects a group of established individuals creates a “permanent underclass” of those on the outside looking in. In Spain, permanent workers are almost impossible to fire or downsize, just like tenured faculty in the US higher education system. The result in Spain is an underclass of indignados chronically under- and unemployed; the result in higher education is an underclass of poorly paid adjunct faculty with few prospects of advancement.
Neither system is sustainable. Both discourage effort and innovation, driving talent away. In academia, many of the best students flee research and end up in industry. Leaving a country isn’t as easy.
In the spirit of eating your own dog food, I took General Assembly’s Intro to Rails this past fall. The whole process of learning Rails by trying out pretty much every online tool available led me to a few thoughts:
0) Prior knowledge is rarely clear-cut. I’ve done some combination of front-end development, game development and transactional SQL in the past. Most of which didn’t help and possibly hurt my efforts to learn Rails. Many people I meet have similar quasi-technical backgrounds and are never sure where to start learning. It’s important not to skip things just because you think you’ve seen it before.
1) You have to be passionate about what you’re building. I don’t think anyone can learn to code in the abstract, bouncing from one generic example to another. But…
2) You can be too passionate. Your first, second, third and twentieth attempt to build something will all suck. It may be easier to get started if you’re not building the end-all-be-all personal project to end all projects. Instead, build something to solve a really simple personal problem without any (obvious) commercial potential.
3) Embrace de-leveraging. You can’t walk in with a mindset that you’re just going to learn enough to know how to hire someone to do the hard work. While you may end up doing that, for the moment, you’re de-leveraging and writing the code yourself.