• 02:04:52 am on January 25, 2009 | 6
    Tags: , , , , , , , , , , ,

    Surveys Done Right – Part 2 – Customer Satisfaction

    I have been dreading writing this entry since I came up with the idea for the series (have you read part 1 yet?).  It is not that I don’t know what to say, or that I don’t want to do it.  It is simply that my fear of providing “sample” survey that will later become “real surveys” for all people without customization or personalization really, really takes hold in this arena.  I mean, who has not had to write a customer satisfaction survey in the past?  It is probably the most used, misunderstood, and poorly implemented of all surveys out there.

    If we go by the surveys that I have seen, customer satisfaction surveys should only have one question since that is what most people care about anyways: “overall, how satisfied are you with us?”.  It seems that if the customer says they are overall satisfied then there are no problems.  Further, as long as we get a sufficient number of customers to say they are satisfied (somewhere in the mid-70s to mid-80s), and the number remains consistent or (gasp) even grows a little  we are perfectly set from the customer experience, loyalty, and satisfaction point of view.  After all, if 80% of people are satisfied – isn’t that good?  Well, yes and no.  I wrote about how you can ensure you will get a 90% of more in customer satisfaction surveys in the past (second most popular post in this site).  It is possible to score high – even increase your scores, and not be doing a good job.

    So, before we jump into the description of the survey and the sample questions (and before I beg you again to personalize and customize them), let’s make sure we understand one thing about this survey.  This “sample” survey is not intended to tell you how many people like you.  It is intended to, historically, provide a trending line of your customer satisfaction overall – but more important it is intended to give you real answers to partial satisfaction questions (e.g. were satisfied with the speed of the answer?) in accordance to your strategy (you do have one of those right?).  Finally, consider that I used these questions for customers that have customer satisfaction as a key metric and part of the insight.  I still believe that customer satisfaction is too flawed as a metric to be used reliably (also read this one), but a trending report cannot hurt – as it wouldn’t with any other metric.

    Too many words to get here, so here are the the essentials of a customer satisfaction survey (part 3 of this series, best practices for writing surveys – stay tuned!):

    • First, the most important part – KISS (keep it short and simple).  I use the rule of 5+/-2 (five, plus or minus two), with five being the magic number, and three and seven the limits.
    • Second, you MUST personalize the questions to the specific items you want to measure satisfaction on – which is why there are some questions below to use, but they may not fit what you are looking for.  Tie your questions to your strategy
    • Third, send them out every business cycle (varies by industry and function), and make sure you use panel management tools so you won’t keep sending them to the same person over and over (we know what they think – no one should get more than one a year), and that you don’t send them to people who won’t respond.
    • Fourth, statistical significance is great for market research or election day polls, not so much on customer satisfaction.  Make sure you get a large enough number to make it varied (I suggest at least between 3-4% of the population) and I am talking responses – not invitations.
    • Fifth, make sure you use the same questions the next time you send them out (yes, exactly the same wording) so you can keep historical trends.  Remember, customer satisfaction is an OK metric to keep as long as you use it as a historical trending metric, not a KPI

    Those are the basic rules for these surveys, but more will arise as you begin to implement them.  Of course, best practices for EFM and Surveys apply, so those are to be considered as well.  Now, without further ado, the list of questions I’d like to propose for your consideration:

    1. Overall, what is your satisfaction with Company XYZ? (hate this question, but it is an easy one to answer)

    Now, this question is to be answered in a numerical scale, I favor a scale of 1-4, you can use whatever works for your objectives.  A scale of 1-10 is popular with people who endorse NPS (Net Promoter Score), but I don’t buy into that.  A different way to ask this questions, which I prefer, is:

    1. Overall, would you say you are satisfied with Company XYZ?

    This question is answered with a simple YES-NO.  Why do I like this question? Two things, first the wording predisposes the respondent to say yes (which is a positive score).  Second, you can setup your scripts to react in the case of a negative answer – and since the question predisposes the respondent to say yes, when someone says NO they really mean it.

    From the following questions, pick some of the ones that reflect your objectives, vision, strategy, and purpose for this survey.  Remember, personalization and customization are highly encouraged – so if these don’t fit your needs go ahead and change them accordingly.

    2.1.  What is your satisfaction with Company XYZ web site / self-service solution?

    2.2. What is your satisfaction with Company XYZ representatives on the phone?

    2.3. What is your satisfaction with Product ABC from Company XYZ?

    2.4. What is your satisfaction with Company XYZ return policies?

    2.5. What is your satisfaction with Company XYZ use of email for customer service?

    Well, you get the idea.  Keep in mind that how you ask the question is going to depend on the answer, and see Q1 above to see some examples of different ways to write the questions.  Needless to say, again, the questions you chose for the second part are related to your specific, strategic goals and are bound to change for your survey.

    This is the longest post I wrote since we started this blog, but I wanted to make sure it was clear and explicit.  There is, of course, lots more to read and write about customer feedback, customer service, customer satisfaction… well, anything dealing with customers.  You will read more and we move along – but how do you like this post?  Interesting? Are you using it?  Please let me know your thoughts and experiences… and thanks for reading!



  • ZoomerangBlog 2:23 am on January 26, 2009 | # | Reply

    You made some compelling points in both your posts, thanks for sharing them. Customer satisfaction and loyalty should always be a major focus, even more so during economically challenging times. Online surveys are one of the best ways to gain insight into your customer’s mindset in order for you to market your product/brand as effectively as you can – especially when you can do the research for very little $$ or even for free.

    Studies have shown that loyal customers:
    Purchase your products and services again and again over time
    Increase the volume of their purchases
    Buy beyond traditional purchases, across product lines
    Refer your company’s products and services to others
    Become immune to the pull of the competition
    Give your company the benefit of the doubt when something goes wrong
    Keep in mind:
    It costs 7-10 times more to recruit a new customer than to keep an existing one
    A gain in customer loyalty of only 5% can lift lifetime profits per customer by as much as 95%
    An increase in loyalty of just 2% is, in some sectors, equivalent to a 10% cost reduction

  • M Sarasti 4:36 pm on January 26, 2009 | # | Reply

    Hello Esteban – Can you provide a little more on how/why the importance of statistical significance varies between market research and customer satisfaction surveys?

  • Haim Toeg 5:04 pm on January 26, 2009 | # | Reply

    Esteban – excellent post and thought provoking as usual. In my experience, having a professional statistician analyze the survey returns in correlation with other business data provides an excellent insight into the customer base’s state of mind and allows action items to be generated for future improvement. This is something I preach to my clients regularly and was wondering if you were considering mentioning it in your series (maybe as ‘part 4 – operational best practices’)?

  • Glenn 3:01 pm on January 27, 2009 | # | Reply

    Esteban, great posts. I linked to them on my blog and posed a question for you here: http://www.allbusiness.com/company-activities-management/operations-customer/11759904-1.html



    • Esteban Kolsky 5:50 pm on January 27, 2009 | # | Reply

      @ M,

      Well, here is the short answer to that question. Statistical significance means that, given a certain error rate, you can repeat the test and get similar results – plus/minus the error rate. To achieve that, error rates of 4-5% are acceptable, and that reduces the needed number of respondents. The problem applying that concept to a business setting is that a 10-point potential for error is too big – can you imagine if you get 80% this quarter and 70% next quarter? You would not have an idea of where to start looking for failure points. There are plenty of web sites and tools to help you calculate the statistical relevant needed number of responses given a certain error rate, but they tend to highlight very small sample sizes (e.g. during the elections, most of the polls you see ask just around 1,000-1,200 people – to represent 60+ million). It is better in business to deal with larger sample sizes so you can both be assured that the results will continue to trend equally as well as allowing you to capture more direct feedback from customers. Focusing on statistical significance will just reduce your sample and respondent size.


      Well, I guess that I don’t need to write it now – do I? Yes, part 3 is the best practice for surveys, and part four is the best practices for feedback management. Thanks for your comments… will definitely include in the post your suggestion of going pro. BTW, Part five is probably going to take longer since I am trying to collect a couple of good case studies of people who did what they were supposed to do and succeeded… but it takes time to do that. Actually, I should have never said it – now I am committed 🙂


      Thanks for the link! I will go read your question and answer that… today! Thanks for reading.

      Thanks all for the comments, keep them coming…

  • Esteban Kolsky 10:10 am on January 29, 2019 | # | Reply

    Reblogged this on thinkJar, the blog! and commented:
    part 2, saving content before it goes away forever.

    surveys, four parts blog experience from 10 years ago

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: