Message to Consumer Insights Managers: “Social Media Listening is for you too!”
A few months ago, I was presenting at an MRS conference. At the end of my presentation, a consumer insights manager from a blue-chip multinational came up to me and said: “My CEO has asked me if we can use social media listening for insights; what is your opinion, could we?”
To which I responded with a question (not good manners, I know) :
“Does your company currently buy any social media monitoring service? “
“Yes, our PR department does but I do not have access to it”.
My passionate response that followed lasted about 5 full minutes and went more or less like this:
(start the video at 59 seconds)
“We researchers need to claim back what is rightfully ours: the social media listening and sentiment analysis discipline. The PR and customer service managers were quicker and saw the benefits of this new source of information first. As a result, a proliferation of social media monitoring, Software as a Service (SaaS) applications appeared on the market targeting the PR and customer service managesr. These applications were mainly created by tech companies which do not really care about high data quality, their focus was flashy dashboards. CEOs of companies - out of fear that they are losing control of their brands - were happy to invest in the flashy dashboards in order to protect their brands from negative consumer sentiment – see “war rooms” of Nestle and Gatorade. They also saw an opportunity to create another channel of communication and offer customer service through social media along the other platforms already used such as live chat, call centres etc. None of these social media monitoring tools were designed for the Consumer Insights Manager. The sentiment accuracy of most tools like that is less than 60%; When a client uses a search term on a DIY (SaaS) platform, a lot of noise is returned that skews the true picture if we are not very careful. The short answer to your question is yes, listening combined with asking questions can help companies synthesize unique insights”.
Sentiment Accuracy Breakthrough
The R&D team of DigitalMR has just achieved a breakthrough in terms of sentiment accuracy. Allow me to first share our definition of sentiment accuracy using an example: if we provide a brand with a report that says there were 100 positive comments about their brand during last week, then sentiment accuracy is the percentage of comments that were actually positive. If for example 80 were positive, then the sentiment accuracy was 80%. The rest of the comments will have either been negative, neutral or irrelevant. What needs to be clarified though is that the maximum achievable sentiment accuracy by an algorithm cannot be 100%. The reason being: the way we measure sentiment accuracy is by having humans randomly select posts and check if they agree with the sentiment annotated by the algorithm. Humans themselves will disagree at least 1 in 10 times; thus if this is true, the maximum achievable sentiment accuracy by an algorithm is 90%.
Having said all that... back to the breakthrough of the DigitalMR R&D team. A combination of machine learning algorithms with computational linguistic methods has delivered 99% sentiment accuracy; the caveat is that the algorithms predicted 99% of the time how 3 specific DigitalMR curators would have annotated sentiment. If we were to give the unstructured text in question to other people e.g. clients, they may find that the sentiment accuracy was not 99% but 87% for example; this happens because of the ambiguity of posts, especially when posters include positive and negative sentences in the same post about the product category in question.
The 5 Conditions that need to apply for the Consumer Insights Manager
For social media listening reporting to work for the Consumer Insights Manager, the following 5 conditions need to apply:
- There should be an automated way to eliminate the noise (and there is a lot of it for search based social media monitoring) and only focus on the relevant comments
- The automated sentiment accuracy of any report should be declared and ideally should be higher than 75%
- The gigabytes of unstructured text harvested from the internet has to be turned into high accuracy quantitative information, enhanced by qualitative analysis and meaningful drill-downs
- Conclusions and recommendations should be part of any social media listening report for the consumer insights manager
- The possibility to follow up the web listening report with questions should be available in order to probe on unresolved issues flagged through the listening report.
Finally, this great source of information called the web can be used by the market research professionals if they want to adapt to the needs dictated to us by the digital economy. This is about gathering, analysing and understanding unsolicited comments on brands and product categories in general. High data quality is of paramount importance, more so than flashy dashboards - which incidentally CEOs love. Unfortunately, the current users of social media monitoring reports forget to ask the question: ‘What is the sentiment accuracy of my data set?’
Share this article: