• 01:19:00 pm on October 14, 2008 | 5
    Tags: , , , , , , ,

    The Uselessness of Granularity in Feedback Management

    Warning, we are embarking on another of my pet peeves: granularity.  What?  You don’t know what I am talking about about? Let me explain myself.  Granularity is how detailed of a scale you use when ranking feedback.  For example, you could use a simple Boolean scale, like Yes or No.  You could also use numeric scales, say between one and five.  Finally, you could use word scales, like Horrible, Bad, Neutral, Good, Excellent.  Either way, the concept is that you give the respondent a scale of choices to base their response to a specific feedback question.

    What could possibly be wrong with that?  Well, it is not the concept, but the implementation that gets me.  Same as with customer satisfaction and loyalty (you could click on these links for my previous rants on loyalty and satisfaction). The concept of giving people choices is good, as it creates way for them to express opinions on any topic.  It allows organizations to prioritize their efforts and focus on those that need more attention — a Horrible rating needs more attention than a Bad one.  Same thing for Good and Excellent.  See, the logic is good

    The most common used scale until recently was one through five.  That is not that bad, really, except for the odd number of options (more on this later).  However, some “genius” somewhere decided that five options is not enough.  We need to give the customers a scale from one to ten for them to grade us.  They call that granularity and the flawed assumption is that more options will give the organization a deeper understanding of the true feelings of their customers.  There are two problems with this logic:

    First, there are too many options. Customers are already hard to reach for qualified feedback, do you really want to add time to the survey – and so many choices for each question?  Consider a typical, not a good, customer satisfaction survey.  It has 12-15 questions.  Let’s assume that only nine of those questions have granular scales for answer.  In the time it takes a customer to decide whether questions number four and six deserve either a six or a seven the most likely outcome is that they will abandon the survey, or choose an answer without cause.

    The worse part of this, when you get the responses you cluster the answers because there is no real difference between six, seven, and potentially eight – they all become one!  So the time you asked your customer to take to choose the best granular-scale response was wasted, and the decrease in response rates is actually justified.  Customers realized long before you did the uselessness of granularity – and they stopped responding to granular-scale questions.  Not really worth the unlikely benefit of getting better definition of their needs and demands.

    I know what you are asking yourself: what scale should I use then?  A four-option, word scale: Poor, Bad, Good, Excellent.  Why four?  Remember when I said you did not want to have an odd number of options?  If you give customers five options, they will pick number three more often than not.  This is called, informally, fence-sitting and fulfills the purpose for customers to give you feedback but not take a position.  Was my service good? It was Neutral…what does Neutral mean?

    Anyway, I am now getting off my soap box.  Let me part with some statistics.  Among the people I recommended using this scale, and adopted it, customer satisfaction was up an average of 8 points, and response rate scored consistently higher – not with just one survey, but over time with returning customers.

    What do you think?  Are you going to try this?  Let me know your thoughts…

    About these ads
     

Comments

  • Glenn 12:53 pm on October 15, 2008 | # | Reply

    Well, it’s 7:52AM here and I just learned something. It’s going to be a good day. Thanks for the post.

  • Esteban Kolsky 1:30 pm on October 15, 2008 | # | Reply

    Glenn,

    Thanks for the comment. Brightens my day to know someone reads this stuff and finds some morsels of use out of it.

    Esteban

  • Haim Toeg 5:02 am on October 16, 2008 | # | Reply

    Esteban,

    Good post and thought provoking as usual. I agree with the bundling point you make and seen it happen a few times. On the other hand, your statement that when organizations implemented your suggestions customer satisfaction increased by 8 percent made me think of the importance of getting the survey right prior to deployment and the impact any change can have on the results and the consequent loss of benchmark data.

  • Esteban Kolsky 5:42 am on October 16, 2008 | # | Reply

    Haim,

    Thanks for a good comment. I was not convinced that putting that stat at the end made it work – you know how i feel about customer satisfaction – but i have not kept detailed track of any other stat across. that was the only one. the stats i have for some of them in response rates make more sense to me, but i don’t have sufficient to make it valid – just anecdotal valid, and i prefer not to go into that (which is why it was mentioned in passing).

    thanks for reading
    esteban

  • ryan 9:06 pm on June 28, 2010 | # | Reply

    Esteban … I stumbled into your postings on cust sat survey’s and have been very intrigued. Related to scale and how that translates into a “score”… I primarily use 1-5 (0=N/A) which seems to work pretty well for our needs, but we report on the average response…so something might come back 3.84 or 4.30, etc. Some of our capability areas now are strating to look at % satisfied – so the number of ppl that rated a question 4 or higher/number of total responses. Is there a method that is better? Or are they just different methods for different uses (maybe even just preference?)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: