Sentiment Analysis – voodoo or invaluable?
Let’s look at some Analytics and Insights into why Sentiment Analysis can be a valuable tool in relation to the business model.
The breadth and depth of possible applications of Sentiment Analysis is driven by the fact that the analytics is done on free text, or unstructured data, which represents anywhere from 80 – 90% of the data that an entity generates.
How Sentiment Analysis Works
What is Sentiment Analysis?
Sentiment Analysis is a subfield of NLP (Natural Language Processing). The goal of Sentiment Analysis is to determine the attitude of those who wrote (or spoke) a given piece of text.
One of the most well-known examples of Sentiment Analysis is Hedge Funds using it to buy and sell stocks. But the overall ‘science’ behind Sentiment Analysis has a huge application for brands and small businesses, such as:
- analyzing online reviews and gaining insights that relate to Social Proof
- understanding major roadblocks in areas such as Customer Support
- parsing through Social Media to gain predictive insights into what’s hot and what’s not
The basics of Sentiment Analysis is that you break text down into one of three possible outcomes:
- positive
- neutral
- negative
Conceptually, it seems simple enough, except when you get into the complexities of nuance in speech and the various contingencies that can be hard to analyze at scale (ie. words like ‘depends’ or ‘if’). There are multiple ‘Sentiment Analysis Challenges‘ that pertain to linguistics and slang, this is just one example.
In the example below, as a human (hypothetically the owner of the restaurant), there are multiple possible takeaways from this review, a review that would not be considered atypical in the restaurant industry.
When looking at one review, as an individual, it is hard to remain unbiased and judge the outcome objectively in this case. Furthermore, one individual may look at that review and say one thing, while another individual would come away with a completely different takeaway.
It’s estimated that people only agree around 60-65% of the time when determining the sentiment of a particular text.
MonkeyLearn
Sentiment Analysis on Unstructured Data
Through a combination of machine learning and NLP, software programs are capable of breaking down a bulk amount of free text – whether online reviews, social media posts, or internal messages such as on a customer support forum – and analyzing each one for its sentiment.
There are various types of Sentiment Analysis software on the market, each of which may have a slightly different use case than the other, including (but not limited to):
- MonkeyLearn
- Brand24
- Lexalytics
- Brandwatch
- Meaning Cloud
search 'sentiment analysis'
The various softwares pertain to one of many possible (but not limited to) use cases for any given brand or entity:
- Market Research (ie. Strategy Signals)
- Brand Monitoring
- Social Media Monitoring
- Customer Service
search 'sentiment analysis use cases'
The breadth and depth of possible applications of Sentiment Analysis is driven by the fact that the analytics is done on free text, or unstructured data, which represents anywhere from 80 – 90% of the data that an entity generates.
From 80% to 90% of data generated and collected by organizations is unstructured, and its volumes are growing rapidly — many times faster than the rate of growth for structured databases.
MongoDB
This means that any organization not only has unstructured data in the form of emails, messages, DMs, and SMS internally, but it also has to deal with that information externally on social media platforms and community forums.
Comparatively speaking, structured data is that type of text-based data that can be stored in a database, where it is much easier to query, analyze, or pull into reports.
Let’s take structured data first: it’s usually stored in a relational database or RDBMS, and is sometimes referred to as relational data. It can be easily mapped into designated fields — for example, fields for zip codes, phone numbers, and credit cards
MongoDB
Email is considered ‘semi-structured data’ because of the associated meta-data on emails, but the bulk of the message – and its corresponding meaning – is unstructured data.
Analytically speaking, we know that if there are multiple pools of data that pertain to:
- customer feedback
- purchasing intent
- brand equity
Then there is value in diving deeper into that data if at all possible. Parsing through unstructured data has always been a challenge in the Internet Era. Yet, if a decision is made that the data is worth it to dive into – strategically – then there is a high probability that something of great insight will be revealed.
Online Reviews and Sentiment Analysis
Nowadays, almost all consumer-facing brands have some level of reviews on them, from platforms that include, but are not limited to:
- Trust Pilot
- Google Reviews
- Facebook Reviews
- eCommerce platforms like Shopify
As we discussed in the post of Social Proof, these reviews have a large impact on consumer psychology and purchasing decisions.
Nowadays, people inherently trust these reviews almost as much as they trust personal recommendations.
Unfortunately, these types of reviews are not a bulletproof methodology, with frequent examples throughout history of them being gamed. Nevertheless, as a brand, you want:
- good public-facing reviews
- accurate and insightful data from those reviews
Let’s say a given brand had the following:
- 85% ‘Excellent Reviews’ on a site like Trust Pilot
- 4.5 Star Review on Google Reviews
A Sentiment Analysis exercise (using software) could break that down into the following categories, with the goal of gaining more perspicacity.
Overall Sentiment – Positive, Neutral, Negative
A readout would be generated showing what % of those reviews were positive, neutral, or negative based on a linguistic analysis of the text by any given software.
The positive % would not be 85% or higher. It would likely be in the 40 – 50% range, and then another 20 – 30% would likely be neutral. The negative would likely be higher than 15%.
A lot of this has to do with the human psychology of a 5 Star Review system, whereby giving a 4 Star rating could come with a contingent warning (ie. think about a service like Airbnb).
Sentiment By Rating – Analyzing 1 Star vs. 5 Star
Naturally, if we were to analyze each level of sentiment for 1 Star, 2 Star, 3 Star, 4 Star, and 5 Star, we would see that the negative reviews skew heavily towards the 1-2 Star range, while the positive reviews would skew towards the 4-5 Star range.
But a deeper, NLP-driven analysis would show a lot of neutral type bias at both ends, which would come laden with ‘suggestions’ or ‘hints’ as to how to make the product/service better.
Imagine sentiment like:
- ‘I would have given a better review if …’ or
- ‘I wanted to give them 5 stars but …’
Understanding what drives this type of sentiment could be a goldmine for many businesses.
Sentiment By Category
Let’s go back to our original ‘review’ that we created at the beginning of the post to show roughly how Sentiment Analysis works. Imagine that this text accompanied a 4-Star Rating.
As we can see above, there are two distinct Categories to analyze:
- Food Quality
- Service Experience
The ‘positive’ rating applies to the food quality, while the ‘neutral’ rating applies to the service quality. We can’t say it is obvious from just one review that service quality is poor at this particular restaurant, as every restaurant will have nit-picky customers. Nor can we say definitively that the food at this restaurant is 5-star, as not every customer is discerning.
As a result, we would need to look at multitudes of reviews and compare them across Categories. From there, a clear trend may emerge that:
- the Food Quality is 5 Star
- the Service Experience is 4 Star
Strategically, it is obvious at this point that improving the service-delivery experience becomes paramount and could bump the aggregate review rating of the business from (hypothetically) 4.5 Stars to 4.7 Stars, thus aiding greatly in Customer Acquisition.
Sentiment Analysis and the Business Model
This blog is principally based on business models and topics surrounding that, so if you have made it this far in the post and are waiting for some kind of advertisement for Sentiment Analysis Services, you will be disappointed.
There are several major ‘trends’ affecting the broad market right now, which are making it difficult for brands, both small, medium, and large:
- Recessionary Signals and Tapped-Out (high debt levels) Consumers
- Increasing Competition and CAC Costs Trending Higher
- A New Emphasis on Retention, Referrals, and Loyalty
There are many dimensions to the business model canvas, 9 to be precise:
A lot of the analysis relative to business models focuses on the top and bottom-line analysis, but there is a great deal of nuance in-between when we think about measuring customer value.
In that light, Customer Relationships are absolutely critical in any business model.
How does a company plan to build and maintain relationships with the customers it is serving?
Key Components of the Canvas
Sentiment Analysis can be used as a tool to strengthen Customer Relationships, among other elements. It can cut below a lot of the vanity metrics and visible data on a brand, and unearth unique insights that can be used to go deeper with certain Customer Segments or Sub Segments.
This can help a business with both:
- existing revenue streams
- or creation of new revenue streams
These insights that emerge can be used as a Strategy Signal, as mentioned above, but they can also be used as data to help with Customer Validation for new ‘ideas’ or proof-of-concepts.
Overall, Sentiment Analysis is not typically an ‘out-of-the-box’ solution that be transcribed similarly from one brand to the other. There are different use cases, and different softwares for each use case that touch on different dimensions of NLP and Text Analytics technology. But there is no doubt that Sentiment Analysis can be a VERY valuable tool for brands, big and small.
More on Sentiment Analysis
New Tech – NLP (Natural Language Processing)
Business Modelling – Customer Validation