The 3 things you should get right if you use Social Media Listening

Are you Listening?

Social media listening has many names; the most accurate term to describe this new marketing discipline is probably Active Web Listening. “Web” is more appropriate than “social” because when people share their views about brands, organisations and people, they do so not only on the well known social media sites but also on blogs, forums, and commercial websites (such as Amazon). Sometimes we also want to listen to what is in the news – editorial content – as well. The word “active” emphasises that it is not enough to just "listen", you have to do something about it, which assumes that you understand what people are saying and what the issues are. Having said all that, the most popular term used in a Google search by people looking for solutions as such is: 'social media monitoring'.

Now that we have the nomenclature out of the way, let’s discuss how to do social media listening properly; we need to pay attention to 3 things really:

  1. Noise

  2. Sentiment Accuracy

  3. Drill-down capability

Let’s have a closer look at these 3 things one by one:

  1. Noise
    Any given query that will initially be used for the monitoring of a subject or product category will, almost for sure, produce posts that are not relevant to the subject . Sometimes the irrelevant posts are 80%-90% of the total posts harvested from the web. For example if we have a query with just one search term e.g. Apple (Computers), we will get lots of posts about apple – the fruit. The usual way to get rid of noise is to use a Boolean logic query, something along the lines of: Apple AND Computers OR phone OR Tablet NOT taste ….etc.

  2. Sentiment Accuracy
    This is probably the most difficult problem to solve when it comes to making sense out of social media. Most end-clients (brands) of social media monitoring and analytics have developed ways to extract value out of their existing social media monitoring dashboards, without making use of sentiment analytics. In other words, they know how many posts are talking about their brand and their competitors, but they do not know how many of these posts are negative and how many are positive. They also have no idea what their Net Sentiment Score benchmarked with their competitors is (NSS is a very useful metric and a DigitalMR trade mark). We believe the reason they chose to ignore sentiment is simply because no supplier of theirs is able to deliver a sentiment accuracy over 60%.

    negative, neutral, positive

    This ended on December 31st 2014 when DigitalMR completed the 2.5 year development of listening247. Through the use of a unique combination of machine learning algorithms and computational linguistic methods, the DigitalMR R&D team was able to achieve sentiment accuracy over 85% in multiple languages and product categories. A machine learning model usually delivers 70% - 75% sentiment accuracy initially, and then with continuous fine tuning (for about a month) it climbs slowly but surely to 85% and even higher. 

    The key to establishing the sentiment accuracy is for a number of humans to agree with the posts processed (and the sentiment detected) by the algorithms. We use random samples and ask the end user (client) or an independent third party to go through the posts and annotate sentiment manually. We then compare the results of listening247 and those of the human annotations, and establish the degree of agreement. The caveat here is that sentiment accuracy can never be 100% since even humans do not agree 20%-30% of the time due to sarcasm and general ambiguity.

  1. Drill-down capability
    The drill-down capability depends on two things: a drill-down dashboard and an appropriate taxonomy that describes the topics discussed around a subject or product category. It is fairly easy to drill down into posts about a single brand, and then within that brand to drill down into a key term used in the discussions, and then within that term, to look at only the negative posts. What is not easy to do is look at the posts around a topic or discussion driver, then drill down to see what the sub-topics around that main topic are, and then drill down further  to see what people are saying about one attribute (of the many) within the (chosen) sub-topic. After all that, we can still have a look at a specific brand, the sentiment, and the source of the posts at the attribute level; a total of 8 drill-down levels are possible with an approach like this.

A delegate at the MRS Healthcare research conference last week in London said that if anyone could take thousands of posts in any language, and was able to analyse for topics and sentiment, they would consider this a superpower equal to that of super heroes such as Superman and Spiderman. Well it is quite telling that a colleague in the business of market research did not even know that this is possible and that the only superpower we need to achieve it is machine learning capability.

Here is where the magic comes in (if you get the above 3 things right): Social media listening takes unstructured text (consisting of thousands of posts), provides structure to it which allows us to see a quantitative analysis and interpretation otherwise impossible, and furthermore allows you to get to a few homogeneous posts that you can read for a qualitative analysis take and further probing.

Can Market research get any better? What do you think?

Market Research as we know it is perfectly unsuited for the digital economy

example of displaying insights


In the January 2015 edition of Research World – the ESOMAR monthly magazine – there is an article that I co-authored with Dimitris A. Mavros, the Managing Director of MRB Hellas, the third largest Market Research agency in Greece. The article is about the impact of the digital era on traditional market research agencies; our premise is that if they remain traditional, the impact will not be good!



 Here are our 10 predictions about the future of the market research industry:

  1. The traditional market research agencies that refuse to change will go out of business
  2. DIY market research will catch on even more and will democratise our sector
  3. Social listening analytics will be a must-have for every marketing and market research manager
  4. Agile research will become mainstream and will be facilitated by online communities
  5. Micro surveys and intercepts will eventually replace long monthly customer tracking studies
  6. Processing behavioural data in motion and delivering real-time micro insights will be a core competence of any insights expert agency
  7. Adjacent marketing services such as customer engagement, enterprise feedback management, customer advocacy, will become solutions offered by the market research companies of the future
  8. Data scientists will be the new insight experts, utilising a lot more predictive analytics than rear-view mirror analytics
  9. The code of conduct of market research associations such as ESOMAR and MRS will be revised as it does not apply to the digital economy. If not, the new breed of MR agencies will refuse to be members of such archaic organisations, and the latter will die out
  10. Nielsen will no longer be the largest market research company in the world

Client vs Supplier next-gen market research interest

The above table from GRIT Winter 2014 more or less confirms some of our predictions; the source of this data is market research agencies and end clients of market research. The social media analytics is interesting because 47% of end clients claim to be using it whilst only 34% of the agencies claim the same. This could mean that other technology companies are being used by the end clients that are not market research suppliers.

I would be very interested to start a conversation with colleagues who have an opinion on the matter. As a company, DigitalMR holds the above positions since 2010, when it was established; in 4 years we did not have to change our minds on any of them, if anything, we see social traction confirming those positions. I am sure there will be more than two opposing views and maybe we can define different narratives and segments of us.

The bottom line is: change or perish. If you are a traditional agency it is not too late. A good first step will be to include in your solutions portfolio: social media listening and online communities. DigitalMR is looking for selected market research agencies to be its partners in certain countries and industry sectors. Please do get in touch, if nothing else, we can have a pleasant chat about the future of market research or have coffee if you are visiting London.

Do you know of any 2nd or 3rd generation social media listening platforms?

example of displaying insightsMarket research companies do not really have a reputation of being a very innovative bunch; for years there has been no revolutionary innovation in market research tools.

Social media listening is a discipline that, as we wrote many times before, was elusive for market research. Both the corporate and agency researchers did not see the value; they did not trust it, it was not “representative”…yes, they used the 'R' word…ooouuuuuuuuuhhhhhhhhh. The first generation of social media monitoring tools (which are still in full use today) are not deemed useful for market research by the corporate researchers; rightly so if you ask me. Their sentiment accuracy is less than 60% and usually only capable of dealing with a specific language. There is no capability of drilling down into topics and sub-topics, when they provide sentiment it is only at the brand or search term level, and the percentage of noise (irrelevance to the topic) that a user search query returns is 80-90%.

I had not realised that our own listening247 is a 2nd generation social media listening tool until Lenny Murphy (@lennyism), Editor-in-Chief of the GreenBook Research Industry Trends Report and blog,  called it that during a phone conversation a couple of weeks ago. I guess what 2nd generation means is a platform that was specifically developed for consumer insights- one that addresses all the shortcomings of 1st generation tools that are mentioned above. Social media research is now a very concrete discipline with new tools such as listening247 and online community tools (such as communities247) that complement it. We believe that the biggest shortcoming listening247 addresses is multilingual sentiment accuracy - the result of almost 3 years of hard work and relentless focus of the DigitalMR R&D team is being able to consistently reach over 85% sentiment accuracy in any language.

I was very pleased to hear Lenny go on describing what he thought 3rd generation tools will be like; he said they will link sentiment to customer behaviour and customer profiles, there will be more focus on analysing images, voice, and video for sentiment and more granular emotions. He also spoke about using text analytics to deal with sources of unstructured text other than social media e.g. email databases, instant messaging, call centre conversations etc. In case you are wondering why I was pleased, you can reach out and ask me on @DigitalMR_CEO.

Some of these new market research methods will disrupt traditional market research as we know it even further. The do-it-yourself aspect of these new platforms will help democratise the space and allow current non-users to convert to online market research, mainly because these platforms will be affordable, efficient and effective.

End clients of market research have already started realising that 1st generation social media monitoring tools have very low sentiment accuracy, and even if they are captive to these tools, they still ask DigitalMR to score their posts for sentiment. In most cases, the sentiment accuracy difference between listening247 and the other tool is over 30 percentage points. If you would like to know how we define and measure accuracy please ping me on Twitter and I will be more than happy to provide definitions and examples.

At DigitalMR we have a couple of other rabbits in the hat, not sure if they will help us qualify as a 4th generation social media listening tool, but we are certainly happy that we are currently considered 2nd generation going on 3rd :) since the majority of the current players are still gen 1.

Communities plenty for 7...or 8: The DigitalMR Christmas community

communityDigitalMR prides itself on its multilingual social media listening platform - listening247. Funnily enough, the multilingual aspect is also an integral part of the DigitalMR team. We are a multi-ethnic company, either from different countries or with different backgrounds, from machine learning scientists to psychologists, researchers and software engineers, you couldn't put together a more diverse bunch if you tried; well recently we discovered that our musical interests are very different as well – which comes as no surprise.

During our Christmas party in our office in London, our CEO Michalis asked whether someone would like to go to an opera concert with him (to hear the British Tenor Alfie Boe) as he had a spare ticket, to which everyone responded with something along the lines of “ehh, hmm, umm, no thank you”.

Taken by surprise, he responded “what, no one wants to go and listen to Alfie sing?!' and then attempted to woo us by interrupting the Christmas Carols playlist to play a song by the tenor he was going to see (at the O2 arena). Although we all admitted that the music sounded fine (..ish), classical songs were just something that none of us would like to endure. This discussion prompted the idea for everyone to play their favourite song/music, so that we could all get a better understanding of what the others listen to; little did we know, we were about to start our own market research community, about music.

We transitioned from Death Metal, to Reggae, to House/Electronic, Opera, Funk Rock, Croatian Pop, Chinese Folklore, Greek Rock; let's just hope the office next door wasn't listening in.

Although our music was very diverse, we all listened to a significant amount of each other's music and shared our thoughts; despite the fact that we didn't want to admit it to each other, we had a suspicion most were dreading having to listen to one another's music.

teamwork makes the dream workSurprisingly, the whole “activity” was not unpleasant, each person explained what their song of choice was about and what it meant to them and in a way, it brought us closer together as a team.

After that, we all worked together as a (co-creation) community and decorated a fantastic looking tree, if we may say so ourselves. Just imagine all these people in the picture above, working around one tree at once, you would think that chaos would ensue, but it didn’t, each one of us worked with the other to build this tree. (Finishing touches like a red Ethernet cable garland by Chris the Digital Systems Director are what make our tree unique.)

This dynamic is not dissimilar to what takes place in online communities; not only does it not matter how diverse an online community is, as long as they are gathered for one common interest it makes it even better - in this case, it was a Christmas party that turned into music exploration. Members of online communities are more engaged and motivated when they are united by a common purpose and when their size is less than 300 members; connecting as humans is much more powerful than connecting for online market research alone.

Happy holidays from the most diverse market research company we know!

Beautiful tree with ethernet cable garland

PS. “I find the opera quite boring to watch” – Alfie Boe : )

The other 5 reasons why every organisation should build a community online

An online communityIn our previous blog post we mentioned the top 5 reasons why every organisation should build an online community. There are of course more than 5 reasons, so in this blog post we will describe the other 5 , and who knows, maybe we can come up with another 5 in the future :).

Here are the other 5 reasons why every organisation should build and operate private online communities with its stakeholders:

1. Open a window into your customers’ lives

An online community of customers can make the necessary marketing-related information available on tap. It is now possible for all levels of your organisation, not just the marketers, to connect with customers and get first-hand information from them – see them on video and hear them express their thoughts.

2. Increase the number of marketing resources/employees without having them on the payroll
Imagine having an online community of 200-300 customers who can participate in any marketing decision you will ever make. Instead of sitting in a meeting room brainstorming with a handful of colleagues in the marketing department, you can now involve another 300 people in your decision-making process without adding headcount on your payroll.

3. Impress your Board/CEO
Board Directors and CEOs, especially in B2C, love to see and hear a consumer talk about their needs and their products and brands. They are many degrees away from talking to consumers on the street, so bringing the consumers into their boardroom elevates their understanding of the ultimate client and helps them make more informed decisions.

4. Reduce your market research cost
Especially when the target customer is difficult to recruit due to low incidence; owning online communities of customers and being able to invite them to participate in research projects can provide a dramatic saving. For any incidence rate of customers below 10%, a substantial saving can be achieved in the case of needing sample for online surveys and online focus groups.

5. Increase the speed of accessing insights (reporting)
Having a few hundreds of customers on private online communities first of all eliminates the need to find them; that alone takes days if not weeks out of a traditional research project. On co-creation communities, where customers can participate in research as many times as we want them to, not only are we not worried about expert respondents but on the contrary, we want them to become really good marketing employees honing in on their creativity and other skills. Frequent participation and especially designed data collection tools (e.g. short polls) can get surveying time down to 24 hours with a real time observation of the progress. Also, employing the advanced text analytics methods used in listening247 such as machine learning, unstructured discussions on the community wall or a bulletin board can be analysed in real time.

I hope the 5 other reasons make as much sense to you as the 5 main reasons described in the previous blog post. We are very keen to hear your views on other good reasons why an organisation should make it a priority to engage with customers on bespoke online communities.