The collective conclusion from 2006’s Netflix Prize competition was this: You can’t beat massive model ensembles. Ten years on, I’m still convinced the larger lesson was completely overlooked: that sometimes, when modeling human behavior, there are better KPIs than RMSE.
In 2011, I began work on my own SaaS recommendation service (called Selloscope). As part of that work, I evaluated the performance of a Slope-One model (similar at the time to Netflix’s Cinematch algorithm) against a spiffied-up object-to-object collaborative filter. The latter doesn’t produce an error score, so I devised a different goalpost: If I remove random recommendations from each user’s profile, how well does the model fill in those gaps?
Continue reading “I still think we collectively learned the wrong lesson from the Netflix Prize contest”
Having spent the past dozen years in an Analytics role, both as an individual contributor and as a manager, I’ve had the opportunity to see the analytics function work within centralized, decentralized, and matrixed organizational structures. The relative merits of building a centralized versus decentralized Analytics organization depend largely on exactly what you expect to get out of the Analytics function in your organization, so it’s important to consider the pros and cons — and to structure the Analytics function correctly to meet your organizational goals. Continue reading “Centralized vs Decentralized Analytics: All You Need To Know”
There are two very commonly known things about search engine optimization (SEO).
1) Great, targeted content drives PageRank.
2) Google’s page rank algorithm is recursive.
Given how common this knowledge is, it’s surprising that so many Web sites haven’t put 1 + 1 together: Your pages earn a PageRank based on content and links, and then they lend that PageRank power to your other pages based on how you link to them.
In effect, your internal link structure may be causing your most powerful, highest-ranked pages to spread their PageRank weight across your entire site, instead of to the specific campaigns or initiatives you mean to support with those individual pages.
The takeaway: Instead of thinking of your SEO efforts as a collection of channels (FaceBook, Twitter, YouTube, and the obligatory blog) meant to drive content — think of SEO as a structured set of Content Tiers, each of which are built and interlinked to support your marketing and SEO objectives.
Enter: the SEO Matrix.
Continue reading “Enter the ( SEO ) Matrix”
I’m always surprised that you don’t see more direct marketers hired into Web analytics roles — they’re expert in a number of skills such as predictive modeling, experimental design, and statistical analysis that really turn the Web analytics role into a strategic source of business revenue. Along those lines, I wanted to give an example of how a tried-and-true direct marketing technique — using regression modeling to predict how likely each user is to convert — can allow media buyers and SEM managers to significantly reduce the amount of time it takes to determine which campaigns are winners and campaigns are losers.
Companies rely on pay-per-click and online ad buys to drive growth. While buying clicks is easy, ensuring that individual campaigns are profitable – before significant losses have been incurred – presents a host of challenges to media buyers. Continue reading “SEM for Longer Time-to-Convert Businesses: How to Stop the Bleeding”
In his recent post Are You Rational?, Seth Godin makes a broad argument that nobody is rational all the time (true enough): that some decisions fall squarely within the domain of rational methods (e.g., analyzing your Adwords click-thru rate) while other things are best approached irrationally: falling in love, appreciating music or wine, or generating ideas for new businesses and startups. He goes on to say that “irrational passion is the key change agent of our economy.” Simply put, he proposes that there are entire domains of human endeavor that are better managed with the “gut” — a common belief and arguably a staple of American culture itself.
A great professor once told me, “If a good essay is one that’s fun to argue with, yours is a great essay.” It’s in this spirit that I can’t resist offering a counterpoint to Godin. My argument: that passion and irrationality are two different things. Passion has place, but the time when it was OK to “go with your gut” is well behind us. Continue reading “A Rationalist Responds to Godin’s Blog Post “Are You Rational?””
There is a perennial question in Web analytics: “Are the numbers up?”
Certain web metrics can be highly variable on a day-to-day or week-to-week basis. Daily Unique Visitors (UVs) and Daily Visits are just two examples of metrics that can change dramatically from one day to the next. These big swings in day-to-day numbers can make it difficult for managers to tell whether their KPIs are really trending up or down.
Continue reading “How to identify real trends in user behavior”
Let’s start with a simple premise: The more often something happens, the more often people write about it.
Sounds reasonable, right?
Albert Saiz and Uri Simonsohn, in their article “Downloading Wisdom from Online Crowds,” demonstrate that the relative frequency of documents returned by a search engine can be a good measure of how frequently a phenomenon occurs. For example, if you want to know the relative cost of living in all U.S. cities (or the relative amount of corruption, or perhaps even how good the golfing is) then simply searching for “Dallas cost-of-living” and “San Francisco cost-of-living” may give you a great index. If it works as advertised, this is a fantastic general research tool for analysts and marketing researchers. Let’s take a look.
Continue reading “Using search engines to measure social behavior”
But these tools are not how you compete on analytics. Continue reading “What it takes to compete (and win!) on analytics”
In 2003, Frederick Reichheld published a Harvard Business Review article entitled The One Number You Need to Grow. Reichheld’s article described a method for computing a simple, easy-to-understand customer satisfaction metric called the Net Promoter Score — and ushered in a flavor-of-the-month management practice that has left a bad taste in the mouth of academics and serious marketing researchers Continue reading “Why “The One Number You Need To Grow” is one number you should probably avoid”
A few days ago, Frédéric Peters (Cupidon at Cupidon.be) posted an interesting question to the LinkedIn’s Internet Dating Executive Alliance group: Given the oft-cited success of the online dating industry, what percentage of Google’s ad revenue is driven by the online dating industry?
Based on Frédéric’s question, I felt compelled to run a quick back-of-the-envelope estimate of the share of Google’s ad revenue contributed by the U.S. online dating industry based on Continue reading “Is the online dating industry propping up Google’s ad revenue numbers?”