There is a perennial question in Web analytics: “Are the numbers up?”
Certain web metrics can be highly variable on a day-to-day or week-to-week basis. Daily Unique Visitors (UVs) and Daily Visits are just two examples of metrics that can change dramatically from one day to the next. These big swings in day-to-day numbers can make it difficult for managers to tell whether their KPIs are really trending up or down.
The following graph depicts a typical (as far as that goes) graph of something like unique visitor (UV) or page view counts, and illustrates how difficult it can be to make a judgment based on just a visual inspection of the data series:
One way to identify trends in a highly variable metric (such as Daily Unique Visitors) is to create a simple moving average of that metric. (This is the same moving average you can add to Excel graphs when you right-click your line and select “add trend line.”), In the following example, overlaying a 21-day simple moving average gives us a much better idea of how our UV metric is trending over the longer term:
Although this simple moving average does a good job drawing out longer-term changes in our metric, it may not provide the immediacy many business owners require. To meet that need, we’ll need a way to identify whether more recent observations (e.g., yesterday) were exceptional. Fortunately, while many of us web analysts are new to analyzing data with high volatility, it’s a challenge that stock market analysts have been working on for years.
While simple moving averages give the same weight to all observations, technical stock analysts have developed a kind of moving average (called an exponential moving average, or EMA) that gives more weight to the most recent observation (e.g., yesterday). This means that if the UV numbers from yesterday were exceptional, you’ll be much more likely to see the change on a graph – the number won’t be averaged down as much by all the other days. (Check out ClearStation.com for some additional technical detail as well.)
The real power of using an exponential moving average (EMA) graph comes when you compare EMAs from two different periods. For example, if you place a 13-day exponential moving average (we’ll call this the “fast line”) and a 50-day exponential moving average (we’ll call this the “slow line”) on the same graph, you’ll that the 13-day EMA may trend above the 50-day EMA for some periods. It may also trend below the 50-day EMA for other periods. When the “fast line” trends either above or below the “slow line” for any length of time, you have a real, bona-fide trend. In the following example, the downward trend between December 23 and January 3 is now visually apparent:
As you can see, by overlaying two exponential moving averages with different time periods (in this example a 13-day EMA and a 50-day EMA) we’re able to quickly and easily identify bona-fide trends upward or downward in our data that we might not see as easily any other way. Also, if either the “fast line” or the “slow line” doesn’t match up with your business trend — for example, the chart above, the 13-day EMA shows a big jump coming out of the weekend — you can choose a longer or shorter period to smooth out that activity.
Here’s a link to an Excel spreadsheet with working examples of 13- and 50-day EMAs to get you started. If you want to play around with changing the 13- or 50-days to something else, this is a good place to start.
In a future post, we’ll talk about adding empirical guardrails to give us another indicator of whether today’s numbers are really off the charts or not.
4 thoughts on “How to identify real trends in user behavior”
What is the advantage of doing this over standard deviations? Tia
Hey Steve, thanks for the great question! You’re a step ahead of me: I use the EMA technique to visually indicate whether overall performance is really trending up or down. To that — as you thoughtfully point out — it’s possible to add “guardrails” based on standard deviations. The guardrails then tell you whether any single day is inside or outside a typical range, while the EMA line identifies whether the movement is reliably a trend.
I’ll be writing about combining EMAs with standard deviations in a subsequent post, and post some SQL and SAS code for producing EMAs as well.
Thanks for reading!
Great article, I look forward to the day when more of these capabilities are standard in web analytics products. It really crossing the chasm between data/reporting to understanding/insight.
For sites with significant shifts in volume between weekdays and weekends, I either average the base data on 7 days intervals or use only the weekday data. The picture becomes much clearer and exceptions are evident.
The biggest variation in the day-to-day chart are the differences between the weekdays and the weekends.
So the first thing to do if you want to look at longer-term trends is making a week-to-week chart in order to gid rid of this oscillating pattern. If by then the trend still is not clear, only then you should move on to more sophisticated methods.
Try explaining your EMA to a HiPPO !