Things to keep in mind when running a customer feedback survey

I received a request from American Express to take a ‘short’ survey about their rewards program. I respect American Express as a company and use their products happily. The survey invitation offered 10$ to complete the ‘short’ survey so I chose to take it.

American Express would like your feedback. We would like you to participate in a survey about the rewards program from American Express. Your participation will provide us with valuable feedback and help us develop and provide rewards options to better meet your needs.

As a token of our appreciation, you will receive $10 from American Express*

The survey was written for their rewards program. I am a fan of American Express and also of 10$. While taking the survey a few things became readily apparent.

1. American Express thinks a whole lot about their rewards program
2. American Express did not market test this survey
3. The author of the survey writes brochures for a living
4. American Express is disconnected from their customers

Here is a sample of questions from the survey:

How important is it that the Membership Rewards program from American Express…

• Makes it easy to redeem points once you have identified your rewards?
• Offers the types of rewards you want?
• Allows you to use your points where you want when you want?
• Offers great product rewards?
• Is a rewards program that meets your needs?

These are loaded questions, not survey questions. If a segment of the population wants a rewards program that:

• Makes it hard to redeem points
• Offers the types of rewards no one wants
• Doesn’t allow you to use your points where you want
• Offers crappy products for rewards
• Doesn’t meet any of your needs

then perhaps American Express can design a card brand called The Masochist.

I found these gems 6 pages into the survey. The status meter indicates 30% completion. That means there are over 20 pages of survey questions, some 20-30 questions per page with a 10 point response scale. Can I, a lowly customer, be expected to rank such minutae on a 10 point scale? Am I in a position to choose whether “Has rewards that are up to date with current trends in the market” is a 7 or an 8 to me?

I originally agreed to take the survey to get the 10$. I honestly don’t care a lick about the rewards program. I think it would have been more beneficial for American Express to send a survey invitation such as: “Don’t like our rewards program? Don’t use it? Tell us why!” Sure the negative results won’t look good when explained to the boss, but it would illustrate areas for improvement. Wasn’t that the point of having the survey?

American Express can keep the 10$. I chose not to complete the survey and rather spend my time educating the general public on effective survey design.

Here are things to consider when designing a survey:

• Limit your answer system to five choices. 3 is better.
• Word your questions neutrally.
• Do not over rely on multiple choice answers.
• Ask a minimal set of questions.
• Provide a free text area. Heck, provide several! If you truly care what your customers think, then take the time to read this section most carefully.
• Remember your audience. Are they small business owners, or rewards program analyzers?
• Avoid financial incentives for survey takers.
• Elicit negative feedback.

Have you taken terrible surveys? What was your experience? Have I missed any key points for effective survey design?