Analytics technology is rapidly changing. From the launch of Urchin in 1998 to the arrival of Universal Analytics this year, data collection and configuration has vastly improved giving us the power to drill down data and report on a huge range of metrics. However, with great power comes great responsibility, and through pioneers in data analysis such as Avinash Kaushik, Justin Cutroni and Eric Peterson we have been guided into a new approach to reporting.
The new way of reporting
This approach is about putting the focus on those critical metrics which will help you measure the success of your business objectives. Once the metrics are decided, the optimisation process can begin. Analytics is not just about understanding performance of your site, it is about improving the performance through testing and having confidence in the results.
How does the new way of reporting effect the way we look at Accuracy and Precision?
Accuracy and precision are very important factors when it comes to data measurement. Accuracy is how close the measure is to the true value. Whereas precision is the ability to reproduce the same result under the same conditions again and again. A common representation of the extremes of accuracy and precision are shown through the archery example:
An example of how the difference between accuracy and precision can cause the data to be interpreted in different ways can be shown by measuring the number of conversions each week:
We can see from the graph that the accurate results are close to true value, however week 6 shows a decrease in conversions from week 5 which could lead to false conclusions about the performance of the campaigns during the week. Conversely, the precise results are far below true value, but we can see that the performance increase from week 5 to 6 is the same as the true value increase. Therefore conclusions made on the impact the campaigns are likely to be true.
Which has more value, accuracy or precision?
Previously, the answer for most people was likely to be accuracy. In the time when the data was far less complex the focus was to try and maximise accuracy of the data. However now we want focus on improving performance, therefore precision means we can be confident that we are moving in the right direction.
In an ideal world we would love to have access to the true values. Unfortunately for now, analytics packages aren’t going to report true value. We have to understand and accept there isn’t much we can do to improve accuracy and concentrate on improving performance with precision.