Analyzing Chat Logs for Topics and Sentiment

I've recently been working with a client who uses on-line chat as a sales facilitation tool.  Sales chats as a use case for text analysis fascinated me from the beginning because the sheer number of people who shop via the internet is so vast and the number of companies using sales chat is growing so rapidly. If Etuma360 could do a good job of analyzing sales chats, potentially a lot of companies might be interested in Etuma's solution.

The client knew that they were having thousands of chat dialogs every month, knew that some sales agents were better at selling via chat than others and that some agents had lots of chat dialogs as compared to other agents.  But, without extensively studying all the chats, there was no way to know what the topics were that were being discussed, whether more successful agents chatted about different things than less successful agents, how customers felt about different issues being discusssed, whether repeat customers dialoged differently than new customers, etc.

After running a set of the client's data (several thousand chat dialogues) it was clear that Etuma was going to be able to help the client with both topic and sentiment readings on their chat data.  And, because Etuma can take background variables like agent number, region, website, or other data, its analysis was going to be able to tell the client what topics and sentiments expressed were for successful vs less successful agents, as there were clear differences.  Some other valuable insights came out of the analysis, including certain topics that started trending positively or negatively at at different points in time.  As chats are going on costantly, picking up newly trending topics, whether positive or negative gives the client real-time insights into issues that might be driving business. 

Lastly the topics Etuma identified were interesting in and of themselves.  Being able to see sentiment rated chat topics by itself showed the client what its customers were interested in, what they felt positive, or negative, about and how those topics related to sales.

All in all a very interesting exercise, with the take away being that analyzing sales chats is a valuable thing to do.  And, one that is important to do well when doing it.. 

Stewart Nash
www.linkedin.com/in/stewartnash

More on Feedback for Improving Sales Process Effectiveness

In my last blog post I talked about different ways for employing lead, prospect and customer feedback to improve understanding of customer needs (defined here as leads and prospects too).  One of the points I made in the post was that studies show that business buyers often are 60% of the way through the buying process by the time they contact you.  SaleForce recently blogged about this phenomenon in a post titled: "Why is Your Sales Message Irrelevant? (And 5 Other Questions to Close Deals)".   Article link here:  http://blogs.salesforce.com/company/2013/07/why-is-your-sales-message-irrelevant-5-big-questions.html.

I wanted to share their perspective, as I think it butresses my thesis on why sales organizations should be acquiring structured feedback on leads, prospects and customers.  An excerpt follows:

"Your sales message is irrelevant. Today's customers are better informed and more connected than ever before and to sell effectively, your message needs to be tailored. And your sales team--as well as your entire organization--needs to be plugged into what matters most to your customer (emphasis mine).  According to CEB, “Today’s business buyers do not contact suppliers directly until 57 percent of the purchase process is complete.That means for nearly two thirds of the buying process, your customers are out in the ether: Forming opinions, learning technical specifications, building requirements lists.”

Today’s informed customer presents greater challenges, as well as exciting opportunities.

Traditionally, salespeople have relied on call scripts and data sheets. In today’s world, the empowered customer demands a tailored message as relevant to their needs as the results of a Google search  But to improve sales performance, your salespeople must understand their prospects and customers including what they know (emphasis mine), and provide them with the best customer experience during the buying process--it’s a tall order!

The demands of the customer experience require tools that allow your reps to be in tune with the market, your customer’s buying profile, and the tactics to win."

--------------------------------------------------------------------------------------------------------------

My primary point in my earlier post was that sales organizations need feedback processes (illustrated above) in place to quickly assess and distribute "customer understanding" to sales reps.  And those processes should tie directly to the work individual sales people are tasked with.  For instance, individual results of a lead qualification survey should be delivered directly to sales reps making phone calls - prior to those calls being made.  The lead qualification data the survey provides would allow sales reps to better position themselves as consultative sellers, have more individualized customer knowledge at hand (making calls more efficient and effective) and to be more effective users of customer time in the selling process.

----------------------------------------------------------------------------------------------------------------

Stewart Nash
www.linkedin.com/in/stewartnash/

 

7 Ways Sales Process Feedback Can Help Sellers Sell

Customer loyalty research (NPS in particular) has definitively shown that by soliciting and acting on customer feedback, that loyalty and retention rates can be increased.  Results include more sales and more profitable sales.

I contend that by applying the same type of feedback management principles (at QuestBack we call it Ask / Act) that selling processes can be improved such that selling in all phases of the sales cycle is more effective.  This would mean higher rates of leads generated per suspect, higher rates of qualified leads per lead, higher rates of prospects converted from qualified leads and higher rates of customer acquisitions per prospect.  I've listed seven different areas where this kind of sales process feedback may offer benefits, though there are potentially more.

------------------------------------------------------------------------------------------

In my experience, the biggest flaw in sales processes is the necessity for sales people to record their activities in a CRM system.  Since they largely do only the required minimum data entry, at every sales process stage some customer insight is missing, inaccurate, out of date or a combination thereof.  This causes tremendous waste of energy across the organization, missed sales opportunities, poorly chosen sales pursuits and forgone opportunities to increase sales generally.

The problem is compounded by the progressively more difficult selling environment engendered by ever better educated prospects and customers.  Losing deals because of missing information drives sales managers crazy, and always has.  But, today where a new contact might be as much as 60% of the way through their buying process, having an inexperienced lead gen person contacting them might keep you out of a winnable sales cycle altogether.  Asking new leads to estimate their buying process stage by itself might help sales organizations perform better.

In my opinion, Feedback management can improve sales processes in the following ways:

• Lead Generation - Validating Lead Quality and Buying process
• Lead Nurturing - Is the lead still a lead?
• Prospect Mining - Have we missed potential prospects?
• Prospect Validation - Is the prospect a good fit for our solution(s)?
• Customer Entry Profiling - Validate solution "fit", document business objectives
• Customer Satisfaction / Loyalty - How are we doing vis-a-vis business objectives?
• Exit  / Win Back - Can we get a lost customer back?

Lead Generation: 
Many companies take lists of "suspects" and have their inside sales team make phone calls.  "Leads" generated this way are often not good fits and cost money to validate.  An e-mail based lead validation survey sent to each "Lead" within a feedback process would provide an additional automated step that ensures a quality lead is being generated and would improve data quality in the CRM.  Surveying inbound leads in a similar fashion would do the same for them.  Fakes, frauds and indifferents would rapidly drop off (many by simply ignoring your request for feedback).  Those leads who provide feedback will be "qualified" as being interested.  More importantly they have engaged with you and are, at least, likely to be a good fit.  Using Follow-up based on answers to key questions would allow for Sales Executives to immediately get involved with high priority leads. 

Lead Nurturing:
Any lead that meets the criteria for requiring nurturing should be periodically tested for continued interest, relevant business need, timeline, or other characteristic that might indicate a need to engage, or disengage with them.  Periodically surveying "nurture" status leads is a great way to determine if those leads are valid and if they need actions beyond additional nurturing.

Prospect Validation:
After a lead has been designated as a prospect, either because of a sales meeting, webinar, product demonstration, etc., feedback should be sought from them to validate their "fit" with your solution.  Any discrepancies between "fit" articulated by the customer and perceptions logged by sales personnel should be documented and researched.  If sales meetings are used in the process, those meetings can also be evaluated using feedback surveys.  Sales meetings should offer value for customers if they don't, it is important to know why.

Prospect Mining:
In any sales process that deals with lots of leads, some that should become prospects do not.  Either via oversight, misunderstanding or other reason, some leads are overlooked or mischaracterized as not being nurturable.  Whenever a lead is moved out of the sales process an Exit Survey should be offered to them.  Within the Exit survey, there should be an opportunity for the lead to requalify themselves for re-entry into the sales process.

Customer Entry Profiling:
After a deal is done with a customer one would think that the sales organization knows and has documented the most information it ever will have on the customer.  Amazingly, that turns out to not be the case in many instances.  Again, for a variety of reasons sales people neglect to do data entry on customers who are actually buying and why they bought.  Surveying customers upon entry allows your sales process to correct for human fault and acquire key information that will be needed later to retain the customer.

Customer Satisfaction / Loyalty:
Satisfied and more importantly Loyal customers stay longer, buy more and are more profitable than other customers.  Surveying for CSAT / Loyalty today is a "no-brainer".  If you aren't doing it you should be.

Exit / Win Back:
In similar vein to Exit surveys for prospect mining, Exit / Win Back surveys are a great way to ensure that customers who should be retained are retained.  Or, if not retained, then enough is learned from them about reasons for defecting so that other customers later on can be retained based on process or product changes.

------------------------------------------------------------------------------------------

By incorporating more Feedback / Actioning mechanisms into sales processes its possible to capture and institutionalize knowledge about why some leads transition to prospects and others don't.  Why some prospects transition to customers and others do not.  By actioning feedback you'll start increasing those ratios immediately.  Over time, better information will help increase the share of leads that transition to prospects and the share of prospects that transition to customers.  As we have seen through use of NPS increasing the share of customers who stay customers has a profound effect on company profits.  The same effect can be had by applying a feedback / actioning process to leads, prospects and customers. 

Is It Time For an EFM 2.0 ?

 

Enterprise Feedback Management (EFM) was a term originally coined to describe a lot of products designed for doing mainly Web Surveys.  Many of the products were little more than automated data collection forms.  Some however were more sophisticated, offering built-in follow up processes, respondent analysis and reporting.  As time has passed, more and more large businesses have adopted EFM tools that support follow-up, analytics, dashboards and other advanced capabilities.  They have recognized that data collection by itself isn't enough any more.  In fact, a business looking at EFM tools today wouldn't recognize a lot of the products from the 2005-2006 as real EFM tools at all. 

Is it time for EFM 2.0?

So what's changed?  I believe there are five major changes in EFM worth discussing:  Process support, Business Analytics / Visualization, Integration, Text Analysis and Social media / Mobile computing. 

• Process Support.  A key difference between research and operational intelligence (feedback) is that insights are operationally relevant immediately, often in real-time.  Business processes operate on an accelerated pace today.  Timely, relevant and actionable feedback lets a business process operate more effectively and more efficiently.  By taking feedback from customers, partners, employees or suppliers and feeding it in real-time to key "actors" like account managers, support staff, or others is now a critical capability for EFM vendors because it supports improved operational efficiency and effectiveness at the business process level.
• Business Analytics.  Because corporate organizational structures have become so much "flatter", data that was once developed and acted upon by local mid-level managers today must bubble up to higher level managers.  Removing the mid-level manager's data organization,  interpretation and action definition role and replacing it with Analytics, Dashboards and automated follow up processes allows flatter organizations to function better.  Many EFM tools originally did not do any feedback reporting, just "dumping" out MS Excel files for users to analyze themselves.  EFM 2.0 products not only possess their own analytical capabilities, but often they can generate dashboards or feed data (filtered or raw) to integrated BI / Dashboard solutions for further analysis / presentation.  Since this also often a real-time or near real-time capability.  Senior managers can become aware of issues and trends nearly at the same time as the operational "actors" who deal with feedback data at an operational level.
• Integration.  EFM 2.0 products use web services to integrate with a lot of different products (CRM in particular).  But, they also integrate with e-mail, business analytics, data warehouses, etc.  The point of integration, especially for CRM, is that two processes can be automated quickly through rules applied at the CRM level.  1. Deciding who gets solicited for what  feedback, and when.  And 2. Routing completed survey and alert data to internal actors for action.  Integrations speed up processes by automating data transfer to operational systems.
• Text Analysis.  With so much text based data being presented to businesses through survey open end questions, Web site feedback forms, Facebook pages, Chat processes, etc.  Being able to analyze this source of feedback is an increasingly key capability.  It is still somewhat of a challenge to act on in real-time for many businesses because methodologies (like Net Promoter) don't really exist for purely text based feedback yet.  However, advanced text analysis tools allow some key-word based actioning and certainly new topics that require actioning tend to bubble up through text analysis fairly quickly.
• Social Media / Mobile Computing.  When EFM was first defined, there was no real social media or mobile computing being done.  Today it is ubiquitous.  EFM solutions have to be able to integrate with Facebook, websites or other social media sites.  And, they must be able to utilize mobile devices for their feedback gathering and follow up functions. 

Traditional EFM vendors are all busy incorporating mobile, social media and text analysis capabilities into their solution sets.  I see this continuing.  EFM vendors are also trying to create integration capabilities that allow their platforms to connect with CRM, ERP and Data warehouse solutions.  I see that continuing too. 

At QuestBack, our surveys  distribute equally well via e-mail, web and facebook today.  They display and operate equally well on either desktop or mobile devices.  We're integrated with the Etuma360 text analysis engine, several CRM systems and a number of  BI / Dashboard solutions (while also improving our internal analytics and reporting capabilities dramatically).  Other EFM vendors are trying to do the same. 

What's clear to me though is that closed loop feedback processes are more important than ever and need to be process integrated more than ever before.  Simple survey / data collection tools really no longer fit the bill for these kinds of needs. 

Maybe it is time for an EFM 2.0.

-------------------------------------------------------------------------------------------------------------------
About the author - Stewart Nash is a USA based reseller for QuestBack AS and Etuma Ltd.  For over six years he has supported clients large and small with feedback manangement projects and EFM implementations. 

LinkedIn: www.linkedin.com/in/stewartnash
E-mail: stew.nash2010@gmail.com



 

Three Problems with NPS

Before anyone gets the wrong idea.......

I'm a big fan of the Net Promoter Score (NPS) and of process improvement systems that use NPS as a primary indicator of customer relationship quality.  And, I'm a regular user of NPS feedback for various projects I undertake (both my own and customer's). 

For as long as NPS has been around, there have been criticisms of the metric and methodology.  In my opinion though, none of the issues critics present outweighs NPS' value to a business.  Having said that, NPS implementations seem to consistently experience certain problems.  In research I've seen and in forums I follow, people report problems with:  Consistently "closing-the-loop", understanding their internal distribution of respondents (promoters / detractors / passives) and acquiring organizational commitment to "acting" on feedback.  So, I thought I'd talk about those three issues and suggest ways to address them.

#1.  Consistently "Closing-the Loop".

Almost every article ever written about NPS makes the point that closing-the-loop is critical to the success of the process. Yet, it still doesn't happen for much of the NPS feedback businesses collect.  Businesses not "closing the loop" will always experience problems with NPS.  Solving the "loop-closing" problem doesn't have to be terribly difficult, especially for detractor feedback.  But, in an era of web surveys and e-mail responses, filtering responses (in a variety of ways) and responding to each piece of feedback is easy (and surprisingly affordable) if you have the right tools, processes and messages. 

One reason people find "closing-the-loop" difficult is their feedback tool itself.  Typically "survey" tools are not feedback management systems.  If a NPS process is going to rely on a "survey tool" versus an feedback management tool (a good way to tell is that the product's name starts with the word "survey" or "question") it's likely that your "loop closing" process is going to be ad-hoc at best. 

A second tool related issue with "closing the loop" is CRM integration.  Lots EFM vendors base their loop closing processes on CRM integration.  And, for the most part it's an effective approach.  However, because it's based on integration, the loop closing process also has to be integration based, meaning that should scenarios change (i.e. different conditions require different loop closing responses) the CRM programming has to change and the feedback tool integration programming needs to change with it.  Both of these sets of changes cost money and take time.  So, are often not done and "loop closing" suffers for it.

In my view, basic "loop closing" is largely a technology issue for NPS processes.  If you have the right tools you can pretty much always "close-the-loop"

#2.  Respondent distribution amongst NPS categories.

Businesses like to adopt standards.  With NPS its been no different.  When using NPS its a mistake to dogmatically assume that 7's & 8's are always passives.  In your company "Passives" make actually be "5's", "6's" and "7's" or other combinations of scores.  There's always been some level of difference between detractor behavior (defection) amongst various industries using NPS (Think utilities, banks, cable companies, etc.).  In my opinion, most businesses are too dogmatic about implementing NPS and don't adjust their internal scoring to reflect customer behavior over time. 

To illustrate, here's a quote from Fred Reicheld in a recent NPS oriented LinkedIn forum: "if your prior estimate of the right category for a customer is passive---because they scored seven on LTR (likely to recommend) question--but then you observe that subsequent to their survey response, they referred three good customers, doubled their own purchases and complimented your service rep, then you really should recategorize them as a promoter".

The lesson: Raw NPS scores are just indicators of end state status.  They need to be matched up with other data, especially behavior data, in order to know if a given customer (or group of customers) classifies as a promoter, passive or detractor. 

About ten years ago, I participated in a project to define customer loyalty for an industry vertical of our customers.  NPS was a new concept (so we didn't employ it), with Reicheld's first book only recently published.  We categorized customer loyalty as being in two dimensions:  Willingness to refer and Willingness to "buy more". Loyal customers ("Promoters") scored high on the scale for both dimensions.  The Reicheld comment above brought back some memories.  But, its always been clear to me that loyalty is ultimately a behavior. 

Solving the categorization issue is simply one of analysis. If your "8's" exhibit similar enough referral and purchase behavior to your "9's" and "10's" categorize them as "promoters" and treat them that way (I have one customer who has always done just that).  If your detractors are just "0's" - "3's", act on them that way.  One of the advantages of NPS is that it uses an 11 point scale.  This makes it easier to adjust the "buckets" based on customer data analysis, or for industry or cultural differences, than it would be with a smaller scale.

#3.  Getting organizational commitment to "acting" on all feedback

This is the most challenging problem NPS users tend to have.  Typically, NPS processes are "owned" by a single business area.  Often a customer support function.  Other times its marketing or even sales.  Any time NPS (an enterprise level process) is owned by a single business area, acting on feedback that requires someone outside of that area to engage a customer is going to be a challenge and a place where the process can break down.  When the process breaks down, opportunities to "build" promoters get passed up or ignored and issues can fester.  A good example of this type of situation is when a business changes its billing practices.  Finance organizations aren't often closely connected to sales and support organizations.  So, changes in billing or collections policies aren't often vetted by sales.  If a change in these policies is driving down NPS, that information has to get back to the finance department if its going to be changed.  If finance doesn't see the effect of a policy on customer relationships they aren't likely to change. 

In my last blog post I talked about how some customer feedback can be categorized as "Not Obviously Actionable".  I should have stated it as "Not obviously actionable to the business unit sending the survey".  In NPS surveys, there's always a bunch of feedback that isn't obviously actionable, either from the perspective of what to do or who should do it.  Sometimes this kind of feedback is present in "open answer" questions.  Sometimes its because one or more loyalty drivers (product capability, billing practices, etc.) correlate highly with low NPS scores.

Solving the problem requires a management level commitment to high quality customer relationships and a mechanism, call it a "Larger Loop" that integrates NPS feedback data with other kinds of data (behavior data in particular).  Analyses (and their Visualizations) need to occur in near real time, to the appropriate company departments, so that they can see how their actions impact NPS feedback data. 

Clearly, there are more than three challenges that NPS practitioners face.  These are just three I have observed on more than one occasion.  Yet, NPS remains a great tool for understanding customers and what makes them tick.