Today, a simple lesson that so many of us miss at great peril. In fact in your role, at this very moment, your company is making a mistake in terms of how it values your impact on the business.
The lesson is about the limitation of optimizing for a local maxima, usually in a silo.
We are going to internalize this lesson by learning from Microsoft. It is a company I love (am typing this on my beloved ThinkPad X1 Carbon Gen 5, using Windows Live Writer blogging software!). I bumped into the lesson thanks to their NFL sponsorship.
If you were watching the Oakland Raiders beating the hapless New York Giants (so sad about Eli) this past Sunday, you surely saw a scene like this one:
Quarterback Geno Smith using his Microsoft Surface tablet to figure out how he added two more fumbles to this career total of 43. Or, maybe it was him replaying the 360 degrees view of the three times he was sacked during the game.
The Surface tablet is everywhere in an NFL game. Microsoft paid $400 million for four years for the rights, and just renewed the deal for another year (for an as yet undisclosed sum).
For all this expense, you'll see players and coaches using them during the game (as above). The Surface branding also gets prominent placement on the sidelines – on benches, on movable trollies and more. It is all quite prominent.
Here’s one more example: Beast mode!
I adore Mr. Lynch’s passion. Oh, and did you notice the Surface branding?
Now, let’s talk analytics and accountability.
NFL ratings are down, but an average game still gets between 15 m – 20 m viewers. That is a lot of pretty locked-in attention, very hard to get anywhere these days.
The question for us, Occam’s Razor readers, is… What does the Surface Marketing team get for all this money?
If the Surface Marketing team is like every other team at every other company engaged in sponsorships and television advertising, it’ll measure the same collection of smart metrics like everyone else.
First one will be Reach. The Surface team is likely measuring it with deep granularity (by individual games, geo, days, times of days, and a lot more). I’m confident that their analysis will show they are getting great Reach.
The team will rightly be congratulating itself on this success.
Next on the list, having spent enough of my life with TV buyers, I can comfortably say that the Surface team is also expending copious amounts of effort measuring one or more dimensions of Brand Lift metric. Ad Recall, Brand Interest, Favorability, Consideration etc.
Brand Lift is most frequently measured using surveys.
Given the number of times Microsoft Surface, or its branded presence, shows up in a game (52 times in my count in the OAK – NYC game), I believe the Surface team is getting very positive reads from its post NFL ad-exposure surveys.
After 52 times most people would recall the ad, surely answer the survey with some interest in the brand, and everyone (except Coach Belichick) seems to like using the tablets, a favorability that will surely transfer to a whole lot of viewers.
This would, indeed should, result in more congratulations in the Surface team.
The two-step approach above reflects the most common approach Marketers, and their Agencies, use to measure success. Did we reach a large audience? Do they remember anything?
The answers to these two questions power job promotions, bonuses and agency contract renewals with higher fees.
I believe this is necessary, but not sufficient.
I believe this approach optimizes for a local maxima (the media buying bubble) and does not create the necessary incentives to solve for the global maxima (short or long-term business success).
Let me illuminate this gap.
Here’s the global maxima question: How many Surface tablets have been sold due to this near-blanket coverage in NFL games via precious undivided attention?
That was the question that crossed my mind during Sunday’s game.
I had one data point handy.
According to TripIt I’ve visited 156 cities across 32 countries in the last few years. During these trips, meetings and meetups, I've never seen a Microsoft Surface tablet in the wild. Not one.
That’s not completely true. I have seen one frequently. The one I bought for my dad four years ago.
One data point does not a story make.
To assess a more complete answer, we turn to our trusty search engine Bing…
The picture above starts 12 months after Surface inked the $400 million NFL contract. The Surface's share of shipments is so small, it does not even show up in a graph.
Not being content with just one view of success, I tried other sources.
The data from IDC, shows no meaningful Surface anything. Statcounter provides an interesting view as it measures actual use the Surface when accessing the two million websites that use Statcounter. Surface is at 0.29% share.
This is a bit hyperbolic, but in the grand scheme of things… No one is buying a Surface.
Local maxima view of success: The Surface team’s NFL contract is a smashing success. The team is getting great Reach and great Brand Lift. Contract with NFL renewed for another 12 months.
Global maxima view of success: Microsoft is losing.
[Key caveat: The data Statista and IDC provide capture shipments. It is possible that the Surface is being sold directly in a way that neither of these two sources would capture those sales. Perhaps some kind of B2B sales. To overcome this possible issue I’ve used the Statcounter data to capture usage. Still, there is a possible scenario where none, or not enough, of the Surfaces sold visit those two million sites.]
Sadly, Microsoft is not alone in this local maxima focus. Most companies function in a similar manner. Yours. Mine. Other people’s. Our collective mistake is that we don’t think critically enough about what we really are solving for. Our company’s mistake is the incentive structure they put in place (which almost always rewards the local maxima).
Let me give you two examples of this sad local maxima obsession that crossed my desk just this morning. All in the space of one hour.
Local – Global Maxima Example 2: Gap Inc..
A report has been published on The Age of Social Influence. Its goal is to aggressively recommend the strategy of marketing via Social Influencers. Here’s the publishing company’s intro of themselves: “We are a powerful data intelligence tool that combines the knowledge and insights you need to deliver a successful celebrity and social influencer marketing strategy.”
Their claims of this wonderful Social Influencer strategy is based on a survey of 270 respondents. 270. It seems like an oddly tiny choice by a powerful data intelligence tool company (PDITC).
They have all kinds of numbers from the 270 survey sample showing glory.
The very first example in the report of a brand winning hugely with a Social Influencer strategy is Gap.
Here’s a screenshot from the report…
While we all love Cher, seriously she is special, this is a classic local maxima let’s only look at what will make us look good to pimp stuff we want to strategy.
What would be a global maxima if you are going to use a company as a poster child?
Here’s Gap’s financial performance over the last five years…
Gap Inc. has been struggling for years, flirting with financial disaster recently in every facet of its business.
I invite you to explore other financial data on the eMarketer Retail website. Look at Revenue, Earnings, Margins, Employment… Everything is super sad. For an additional valuable lesson, click on Digital as well. It shows the social performance of Gap (illustrating even the local maxima is quite suspect).
I dearly wish the Gap survives, they make good quality clothes.
I also wish that the powerful data intelligence tool company would have chosen to focus on looking at the global maxima success before using Gap, and the other examples in their 40 page report. That would have made their drum banging for Social Influencers more persuasive. It would also have resulted in fewer clients of powerful data intelligence tool company shuttled in the direction of spending money on something that mostly likely will not produce any business results.
Local – Global Maxima Example 3: Amazon
A celebration was shared with me for 31 custom gifs created by Giphy for the up and coming retailer Amazon.
Here’s a non constantly looped, to ensure you’re not annoyed, sample…
The celebration was based on the fact that the total view count for these 31 custom gifs was 31 million.
[Sidebar: Always, always, always be suspicious of numbers that are that clean. 31 gifs being viewed a clean 31 million times is cosmically impossible. Seek the faq page to understand how views are measured. Identify that there is no clarity. Now, be even more suspicious.]
I’m afraid in my book views don’t even count as a local maxima. Even if they are in yours, I hope you’ll agree they are a million miles away from a global maxima.
I wanted to share this example from Amazon because you can’t use the global maxima of overall business success I’ve used above. Even if Jeff Bezos goes around hitting people with feather dusters, Amazon will keep selling more and more products. They have already reached perpetual motion.
What do you do when it is difficult to identify the global maxima from a super-tactical animated 31 gifs with 31 million views effort?
Try to move four steps up from wherever you are. Global maxima lite.
In this case, here’s a great start: % of Users who shared the gif who are not current Amazon customers.
So much more insightful than Views, right?
We are shooting for a deeper brand connection, by an audience that holds business value for us. Sure these people are annoying their friends, but hey at least as Amazon we can remarket to them – and friends (!) – and convert them to Prime customers!
I’m sure you can think of others that are five, six and eight steps above Views. (Share them in comments, and earn admiration.)
It does not always have to be revenue or profit. But, please don’t pop the champagne on views, impressions and other such primitive signals of nothingness.
On the topic of measurement, let’s go back to Microsoft and brainstorm some strategies for their unique use case.
What should Microsoft have measured?
Purely as an academic exercise I’m leaving aside the possibility that the Surface is simply not a good tablet. That would certainly impact sales – marketing or no marketing. But, since Microsoft went back for year five, it is safe to assume at least they believe it is a good tablet.
Ok? It is a good tablet.
Again as an academic exercise I’m going to ignore the four year horizon. There is no question that at the end of year two Microsoft had overwhelming proof from a multitude of data points that the NFL contract was not selling any Surfaces. They did not need Big Data or Artificial Intelligence to come to that conclusion. If they could not get out of the contract, at the end of year two a better use of $100 mil spend per year would have been to change the covers on the Surfaces to Xbox green, and change the numerous printed brand opportunities on the sidelines to Xbox as well. A great selling product, with a much bigger overlap with the NFL audience than the Surface.
Ok? We are not looking after year two.
During the first and second year, what could we have measured as Microsoft if we wanted to do better than the local maxima? Better than Reach and Brand Lift metrics?
Let me plant three ideas (please add yours via comments).
An enhanced survey would be a good start. Along with measuring ad recall etc., they could also ask how likely are you to choose the Surface over the iPad as your next tablet?
It is a tougher question than do you remember the ad or what tablets can you name. It is going head to head with the thing people usually say when they mean tablet. And, you are looking for switching. A strong behavior shift, a harder yes to get when I’ve done surveys. All this brand exposure, if its working, should shift that key intent signal.
Really easy to do. And, you can easily get thousands upon thousands of responses – you don’t have to settle for 270. It would have given the Marketing team a leading indicator that no one is going to buy the Surface as a result of the NFL partnership. The signal could have been received even a couple months in, and certainly by the end of year one.
Time series correlations would have been a great start right after the first week of the contract. How many people are visiting the Surface website on Sundays? Is that materially significant compared to weeks prior or weeks where there were not as many games? Was there an improvement on Sundays in digital sales? How about retail sales on Mondays?
This is simple stuff. Even visits to the site would have been a nice low level signal.
As the season went on, we could look for test and control opportunities. The NFL always has blackouts in cities/states where the stadiums don’t have enough attendance. This past weekend it was in two states, complete blackout of free broadcast games. Is there a difference in site visits, online conversion rates, offline sales, between states that had one game broadcast on Sunday, two games broadcast on Sunday and no games broadcast on Sunday?
A little more complicated. The site stuff is easy to segment. For store sales Microsoft could easily get data from its stores in malls, and likely also from retailers like Best Buy with a little arm twisting. This data would have shows Microsoft, a few months in, that the global maxima might not be reached.
If you don’t have this type of ubiquity, Matched Market tests are also fabulous in these cases to discern if a specific marketing strategy is having a business impact.
Three ideas that I hope will spark many more in your mind when you shoot to measure the global maxima.
I want to briefly touch on one refrain I often hear about these long term efforts, or short term efforts that are not working but are looking at a longer horizon: So what if the results are not there. This is a long term brand building play, Apple did not become a beloved brand in one year.
There is a kernel of truth there, brand building take time. There is a kernel of BS there as well, Apple is Apple primary because of its innovative products.
Let’s not talk about Microsoft in context of the above statement as even if we assume there was some long term brand building happening, it did not translate into business success.
When you hear a statement like that, after you launch a new underwear, cooking range, VR headset or whatever… Obsessively measure more than the local maxima to discern signals in the short term that illustrate that the long term brand building play is not just an excuse to flush a lot of money. Both the Gap and Amazon examples have ideas to inspire you.
Or consider that even your long term brand building play, in the short term should cause you to take noticeable amounts of market share. It won’t be 80% in the short term, but neither is that statement a reason to spend more money if all you got is 5% in year one and 10% in year two.
Don’t settle for opinion.
Use data.
You have data.
Bonus: The real winner of the Microsoft NFL contract?
The NFL of course.
Microsoft makes great hardware. To make it work for the NFL, Microsoft surely wrote lots of custom software for the NFL’s specific use cases. Microsoft likely invested in tens of millions of dollars of camera equipment, wifi/networking upgrades in every stadium, deployed a small army of Microsoft employees to do on-site tech support before, during and after the games in every single stadium. And, more and more and more.
The NFL should be paying Microsoft $110 million a year to upgrade the ability of its coaches, players and teams to have access to this state of the art technology to compete more effectively every Thursday, Sunday and Monday!
The NFL is slated to make $14 billion in 2017, they can surely afford to give $110 mil a year to Microsoft.
Back to the real world… Even when you measure short term success, please do not be satisfied with a local maxima. Even in the short term you can measure something better. On the long term, you have all the elements you need… Definitely measure the global maxima!
Do this because it is the right and smart thing to do for your company. But, a tiny bit, do it because in my experience (across the world) global maxima solvers progress exponentially faster in their career. Turns out, delivering business results matters. :)
As always, it is your turn now.
Do you have a suggestion for what Microsoft or Gap or Amazon should measure as their global maxima? If you’ve been successful getting your CEO to focus on the global maxima, what approach really worked? If you were the role of the Chief Scientist of powerful data intelligence tool company, how would you measure the impact of Social Influencers in a more intelligent manner?
Please add your powerful ideas, brilliant critique and innovative strategies in comments below. I look forward to hearing from you.
Thank you.
Smarter Career Choices #3: Solve for the Global Maxima! is a post from: Occam's Razor by Avinash Kaushik
from Occam's Razor by Avinash Kaushik http://ift.tt/2A6kjCA
via IFTTT
