Social Listening and Online Communities: 1+1=3?


Two out of the top 3 trends in market research repeatedly reported by Greenbook’s GRIT report are social listening and online communities. The third is mobile research which, being a method of collecting data for surveys, can be part of online communities anyway.

We have written about private online communities and social media listening separately many times before, but this blog post is dedicated to the power of integrating the two disciplines.

Back in February, the CEO of Kantar Research Eric Salama spoke at the Insight Innovation Exchange conference in Amsterdam, about his view of the future of market research. One of the concepts that stuck with me was that in the future, market research will be divided in “learning applications” and “action applications”. My interpretation of these two types of apps is that the former is pure market research as we know it, and the latter are adjacent marketing activities that today are not governed by the ESOMAR or the MRS code of conduct. Examples of action applications are programmatic advertising, customer advocacy, and agile customer engagement.

Two of the following three ways to integrate social listening and online community platforms are action applications, and one is a learning application. Let’s see if you agree that 1+1 will equal more than 2 in these three cases:

  1.  Member recruitment for online communities
    For the first time in the history of marketing and market research, we can now find respondents for ad-hoc research or members of communities based on their perceptions, without having to use a screener questionnaire. We can use social listening to gather all the posts from the web that: are aligned with an idea, agree with a concept or express love for a brand. Because the expressed opinions on social media posts are unsolicited, they are of better quality than those expressed in a screener questionnaire used with people from a consumer panel. The panelists have an interest to figure out how to answer “right” so that they will be invited to participate in a survey (expert respondents).
  2. Listen-probe-listen-probe
    A virtuous circle can be created by integrating listening and communities. A brand or organisation can first “listen” to what people say on the web about the subjects of interest, and then engage with the members of their private online communities to ask questions (probe) about what they learnt from the harvesting and analysis of online posts. Through the probing they are bound to discover information that will improve the way they do their social media monitoring. And so on and so forth… Every time they complete a listen-probe-listen cycle, new valuable insights can be extracted that were never attainable before.
  3. Amplified customer advocacy
    Product category influencers can be identified through the content of their online posts and the size of their networks. They can then be invited to join an exclusive private online community for co-creation of digital content and customer advocacy amplification i.e. the sharing of the digital content with their friends and network.

Connecting the dots is a very powerful notion in market research. As shared on this blog several times, we firmly believe that a true business insight is more likely to be the result of synthesizing data from multiple sources as opposed to analysing a (small) data-set to death. The insights expert is a necessary part of this equation (1+1=3). There is also a new breed of a human skill-set that is becoming more and more an integral part of those market research agencies that “get it”; it is the data scientist who is among other things a machine learning specialist not daunted by tera-, peta-, hexa or zeta-bytes. Thoughts?

The 3 things you should get right if you use Social Media Listening

Are you Listening?

Social media listening has many names; the most accurate term to describe this new marketing discipline is probably Active Web Listening. “Web” is more appropriate than “social” because when people share their views about brands, organisations and people, they do so not only on the well known social media sites but also on blogs, forums, and commercial websites (such as Amazon). Sometimes we also want to listen to what is in the news – editorial content – as well. The word “active” emphasises that it is not enough to just "listen", you have to do something about it, which assumes that you understand what people are saying and what the issues are. Having said all that, the most popular term used in a Google search by people looking for solutions as such is: 'social media monitoring'.

Now that we have the nomenclature out of the way, let’s discuss how to do social media listening properly; we need to pay attention to 3 things really:

  1. Noise

  2. Sentiment Accuracy

  3. Drill-down capability

Let’s have a closer look at these 3 things one by one:

  1. Noise
    Any given query that will initially be used for the monitoring of a subject or product category will, almost for sure, produce posts that are not relevant to the subject . Sometimes the irrelevant posts are 80%-90% of the total posts harvested from the web. For example if we have a query with just one search term e.g. Apple (Computers), we will get lots of posts about apple – the fruit. The usual way to get rid of noise is to use a Boolean logic query, something along the lines of: Apple AND Computers OR phone OR Tablet NOT taste ….etc.

  2. Sentiment Accuracy
    This is probably the most difficult problem to solve when it comes to making sense out of social media. Most end-clients (brands) of social media monitoring and analytics have developed ways to extract value out of their existing social media monitoring dashboards, without making use of sentiment analytics. In other words, they know how many posts are talking about their brand and their competitors, but they do not know how many of these posts are negative and how many are positive. They also have no idea what their Net Sentiment Score benchmarked with their competitors is (NSS is a very useful metric and a DigitalMR trade mark). We believe the reason they chose to ignore sentiment is simply because no supplier of theirs is able to deliver a sentiment accuracy over 60%.

    negative, neutral, positive

    This ended on December 31st 2014 when DigitalMR completed the 2.5 year development of listening247. Through the use of a unique combination of machine learning algorithms and computational linguistic methods, the DigitalMR R&D team was able to achieve sentiment accuracy over 85% in multiple languages and product categories. A machine learning model usually delivers 70% - 75% sentiment accuracy initially, and then with continuous fine tuning (for about a month) it climbs slowly but surely to 85% and even higher. 

    The key to establishing the sentiment accuracy is for a number of humans to agree with the posts processed (and the sentiment detected) by the algorithms. We use random samples and ask the end user (client) or an independent third party to go through the posts and annotate sentiment manually. We then compare the results of listening247 and those of the human annotations, and establish the degree of agreement. The caveat here is that sentiment accuracy can never be 100% since even humans do not agree 20%-30% of the time due to sarcasm and general ambiguity.

  1. Drill-down capability
    The drill-down capability depends on two things: a drill-down dashboard and an appropriate taxonomy that describes the topics discussed around a subject or product category. It is fairly easy to drill down into posts about a single brand, and then within that brand to drill down into a key term used in the discussions, and then within that term, to look at only the negative posts. What is not easy to do is look at the posts around a topic or discussion driver, then drill down to see what the sub-topics around that main topic are, and then drill down further  to see what people are saying about one attribute (of the many) within the (chosen) sub-topic. After all that, we can still have a look at a specific brand, the sentiment, and the source of the posts at the attribute level; a total of 8 drill-down levels are possible with an approach like this.

A delegate at the MRS Healthcare research conference last week in London said that if anyone could take thousands of posts in any language, and was able to analyse for topics and sentiment, they would consider this a superpower equal to that of super heroes such as Superman and Spiderman. Well it is quite telling that a colleague in the business of market research did not even know that this is possible and that the only superpower we need to achieve it is machine learning capability.

Here is where the magic comes in (if you get the above 3 things right): Social media listening takes unstructured text (consisting of thousands of posts), provides structure to it which allows us to see a quantitative analysis and interpretation otherwise impossible, and furthermore allows you to get to a few homogeneous posts that you can read for a qualitative analysis take and further probing.

Can Market research get any better? What do you think?

Market Research as we know it is perfectly unsuited for the digital economy

example of displaying insights


In the January 2015 edition of Research World – the ESOMAR monthly magazine – there is an article that I co-authored with Dimitris A. Mavros, the Managing Director of MRB Hellas, the third largest Market Research agency in Greece. The article is about the impact of the digital era on traditional market research agencies; our premise is that if they remain traditional, the impact will not be good!



 Here are our 10 predictions about the future of the market research industry:

  1. The traditional market research agencies that refuse to change will go out of business
  2. DIY market research will catch on even more and will democratise our sector
  3. Social listening analytics will be a must-have for every marketing and market research manager
  4. Agile research will become mainstream and will be facilitated by online communities
  5. Micro surveys and intercepts will eventually replace long monthly customer tracking studies
  6. Processing behavioural data in motion and delivering real-time micro insights will be a core competence of any insights expert agency
  7. Adjacent marketing services such as customer engagement, enterprise feedback management, customer advocacy, will become solutions offered by the market research companies of the future
  8. Data scientists will be the new insight experts, utilising a lot more predictive analytics than rear-view mirror analytics
  9. The code of conduct of market research associations such as ESOMAR and MRS will be revised as it does not apply to the digital economy. If not, the new breed of MR agencies will refuse to be members of such archaic organisations, and the latter will die out
  10. Nielsen will no longer be the largest market research company in the world

Client vs Supplier next-gen market research interest

The above table from GRIT Winter 2014 more or less confirms some of our predictions; the source of this data is market research agencies and end clients of market research. The social media analytics is interesting because 47% of end clients claim to be using it whilst only 34% of the agencies claim the same. This could mean that other technology companies are being used by the end clients that are not market research suppliers.

I would be very interested to start a conversation with colleagues who have an opinion on the matter. As a company, DigitalMR holds the above positions since 2010, when it was established; in 4 years we did not have to change our minds on any of them, if anything, we see social traction confirming those positions. I am sure there will be more than two opposing views and maybe we can define different narratives and segments of us.

The bottom line is: change or perish. If you are a traditional agency it is not too late. A good first step will be to include in your solutions portfolio: social media listening and online communities. DigitalMR is looking for selected market research agencies to be its partners in certain countries and industry sectors. Please do get in touch, if nothing else, we can have a pleasant chat about the future of market research or have coffee if you are visiting London.

Do you know of any 2nd or 3rd generation social media listening platforms?

example of displaying insightsMarket research companies do not really have a reputation of being a very innovative bunch; for years there has been no revolutionary innovation in market research tools.

Social media listening is a discipline that, as we wrote many times before, was elusive for market research. Both the corporate and agency researchers did not see the value; they did not trust it, it was not “representative”…yes, they used the 'R' word…ooouuuuuuuuuhhhhhhhhh. The first generation of social media monitoring tools (which are still in full use today) are not deemed useful for market research by the corporate researchers; rightly so if you ask me. Their sentiment accuracy is less than 60% and usually only capable of dealing with a specific language. There is no capability of drilling down into topics and sub-topics, when they provide sentiment it is only at the brand or search term level, and the percentage of noise (irrelevance to the topic) that a user search query returns is 80-90%.

I had not realised that our own listening247 is a 2nd generation social media listening tool until Lenny Murphy (@lennyism), Editor-in-Chief of the GreenBook Research Industry Trends Report and blog,  called it that during a phone conversation a couple of weeks ago. I guess what 2nd generation means is a platform that was specifically developed for consumer insights- one that addresses all the shortcomings of 1st generation tools that are mentioned above. Social media research is now a very concrete discipline with new tools such as listening247 and online community tools (such as communities247) that complement it. We believe that the biggest shortcoming listening247 addresses is multilingual sentiment accuracy - the result of almost 3 years of hard work and relentless focus of the DigitalMR R&D team is being able to consistently reach over 85% sentiment accuracy in any language.

I was very pleased to hear Lenny go on describing what he thought 3rd generation tools will be like; he said they will link sentiment to customer behaviour and customer profiles, there will be more focus on analysing images, voice, and video for sentiment and more granular emotions. He also spoke about using text analytics to deal with sources of unstructured text other than social media e.g. email databases, instant messaging, call centre conversations etc. In case you are wondering why I was pleased, you can reach out and ask me on @DigitalMR_CEO.

Some of these new market research methods will disrupt traditional market research as we know it even further. The do-it-yourself aspect of these new platforms will help democratise the space and allow current non-users to convert to online market research, mainly because these platforms will be affordable, efficient and effective.

End clients of market research have already started realising that 1st generation social media monitoring tools have very low sentiment accuracy, and even if they are captive to these tools, they still ask DigitalMR to score their posts for sentiment. In most cases, the sentiment accuracy difference between listening247 and the other tool is over 30 percentage points. If you would like to know how we define and measure accuracy please ping me on Twitter and I will be more than happy to provide definitions and examples.

At DigitalMR we have a couple of other rabbits in the hat, not sure if they will help us qualify as a 4th generation social media listening tool, but we are certainly happy that we are currently considered 2nd generation going on 3rd :) since the majority of the current players are still gen 1.

Communities plenty for 7...or 8: The DigitalMR Christmas community

communityDigitalMR prides itself on its multilingual social media listening platform - listening247. Funnily enough, the multilingual aspect is also an integral part of the DigitalMR team. We are a multi-ethnic company, either from different countries or with different backgrounds, from machine learning scientists to psychologists, researchers and software engineers, you couldn't put together a more diverse bunch if you tried; well recently we discovered that our musical interests are very different as well – which comes as no surprise.

During our Christmas party in our office in London, our CEO Michalis asked whether someone would like to go to an opera concert with him (to hear the British Tenor Alfie Boe) as he had a spare ticket, to which everyone responded with something along the lines of “ehh, hmm, umm, no thank you”.

Taken by surprise, he responded “what, no one wants to go and listen to Alfie sing?!' and then attempted to woo us by interrupting the Christmas Carols playlist to play a song by the tenor he was going to see (at the O2 arena). Although we all admitted that the music sounded fine (..ish), classical songs were just something that none of us would like to endure. This discussion prompted the idea for everyone to play their favourite song/music, so that we could all get a better understanding of what the others listen to; little did we know, we were about to start our own market research community, about music.

We transitioned from Death Metal, to Reggae, to House/Electronic, Opera, Funk Rock, Croatian Pop, Chinese Folklore, Greek Rock; let's just hope the office next door wasn't listening in.

Although our music was very diverse, we all listened to a significant amount of each other's music and shared our thoughts; despite the fact that we didn't want to admit it to each other, we had a suspicion most were dreading having to listen to one another's music.

teamwork makes the dream workSurprisingly, the whole “activity” was not unpleasant, each person explained what their song of choice was about and what it meant to them and in a way, it brought us closer together as a team.

After that, we all worked together as a (co-creation) community and decorated a fantastic looking tree, if we may say so ourselves. Just imagine all these people in the picture above, working around one tree at once, you would think that chaos would ensue, but it didn’t, each one of us worked with the other to build this tree. (Finishing touches like a red Ethernet cable garland by Chris the Digital Systems Director are what make our tree unique.)

This dynamic is not dissimilar to what takes place in online communities; not only does it not matter how diverse an online community is, as long as they are gathered for one common interest it makes it even better - in this case, it was a Christmas party that turned into music exploration. Members of online communities are more engaged and motivated when they are united by a common purpose and when their size is less than 300 members; connecting as humans is much more powerful than connecting for online market research alone.

Happy holidays from the most diverse market research company we know!

Beautiful tree with ethernet cable garland

PS. “I find the opera quite boring to watch” – Alfie Boe : )