Chapter 2 - Customer Feedback

Customers - Who are they?
How do you get the most out of them?
Customer surveys
Customer councils
Customer advisory board
User testing
Getting customer input back into the development cycle
Further reading

Who Are They?

As a Product Marketing person, your customers are the most important people you work with. They are the ones who ultimately fund your company and determine the success of your product. It sounds silly to even have to say it, but it is one of those things that are often forgotten in the rush of trying to complete day-to-day tasks.

Your customers will often go several levels deep. The sales channels (whether they be direct or indirect) are your customers.  Additionally, within the purchasing company, the evaluator of your product is your customer, the person who approves the PO for your product is your customer, and the end user is your customer (sometimes three different people, sometimes all the same person). The methods used for gathering input below should be used for all levels of customers.You should do surveys, for example, of both your sales force and your end users (with different questions depending upon the audience).

All your customers should be treated as the critical resource that they are. They are all directly responsible for the success of your product and company.

How do you get the most out of them?

Getting the most out of your customers? Isn't that a little backward? It is true that you want to give your customers your very best, but how can you do that if you don't understand what they need?

Something so obvious and so simple - getting real customer feedback into the development process - is very often ignored in fast-paced engineering driven companies. It is ignored for many reasons, but most often it is because it is not always easy to get customer feedback in a form that is truly useful and timely.

Customer data needs to be collected regularly and proactively. If you only take information from the customers who are complaining loudly, you will have an unbalanced view of your product line. Many customers either suffer in silence or just stop using your product

A second benefit of gathering regular customer feedback, is that customers will also tell you what they like about your product. This information can be used for better positioning, entrances into new vertical markets, competitive analysis, and customer testimonials.

There are several methods for getting regular customer feedback. The following are a few that I recommend. These are all useful if done once, but become much more useful if done regularly. When you have regular customer feedback, it is easy to give yourself a report card and have a better understanding as to how you are progressing in terms of customer satisfaction.

Customer Surveys

An extremely useful tool and well perfected by the automotive industry is the customer surveys. Often times when you buy a car, or get it serviced, you will get a follow-up call from a customer service representative asking you a series of questions about the experience with the car dealer and how satisfied you are as a customer.

Ideally, you want to employ this model. Surveying new customers shortly after they have purchased your product and surveying your existing customers regularly -- once a quarter seems to be most effective.

Volumes have been written on customer surveys and the usefulness of certain questions, so I won't go into that detail here, but here are some basic guidelines:

  • Keep the survey short, 10 - 15 questions maximum.

  • For large-scale surveys use multiple choice questions for easy compilation of the data. Leave a section at the end for additional comments.

  • For large-scale surveys run a pilot test to make sure the questions flow and are understood by the target audience.

  • Do a phone survey because the return is better.

  • If you decide to do an email survey, provide some incentive for the customer to spend the time to fill it out. (Even if you do a phone survey, a small token of the company's appreciation for participating in the survey is a good idea.)

  • Use a basic vocabulary with words and phases people use in casual speech.

  • Use simple sentences that are specific and brief.

Setting up a system within your company for routine surveying your customers is the ideal scenario. However, there are at least two hurdles you must overcome to do this. One is that many companies don't sell direct and don't have their customers' names. One way to work around this is to use your web site to gather end users' names. You can do this by setting up a user group, providing tips and new product information to people who sign up. (For more information, see the Product Sustaining section of this book.) The second hurdle is that for a broad-based survey encompassing a broad group of customers you generally need to have corporate commitment in the form of both software and resources. Quarterly reviews of all customers is the most effective scenario because it will help you recognize trends in your customer base. However, a quarterly customer review process takes a commitment from the top of the organization. If you don't have that commitment, then start on a on a smaller scale.

Small Scale Surveys

For these sample surveys it is best to select a group of customers that are representative of your target market. Create a questionnaire with open-ended questions. You will find that you will get more information from your small sampling with open-ended questions. Make the phone calls yourself. You want to establish a dialog with these customers so if they think of additional information or opinions they feel free to contact you. Another alternative is to take a team approach. If you have multiple people in your group, each of you take ten customers to survey. Then you can get together to compare results.

The following are the advantages of small-scale survey

  • You can gather more detailed information

  • You don't need corporate commitment

  • The survey can be done quickly and easily

The following are the disadvantages of a small-scale survey:

  • Smaller sampling means less accuracy of representing the installed base

  • There is no mechanism in place for routine surveying of the customer base

Sales Channel Surveys

Your sales channels can tell you volumes about how easy or difficult your products are to sell. One of the most difficult aspects of sales channel surveys is getting the sales people to respond. They often have other priorities that they are working on - like making money.

I have found it is useful to survey the sales channel before a product planning meeting. When you tell them that you need their input to include in product planning they are generally receptive to helping you get the information you need to make their job easier. Another good time to survey sales people is in advance of a sales meeting to find out what type of support that they need (this is covered more in the chapter on Product Sustaining). In any case, it usually takes an e-mail and several phone calls to get feedback back from sales people.

Sample Small Scale Customer Survey Plan

Before writing the customer survey, write a short plan with that includes the objective of the survey, target audience, and the method.

- What specifically are you trying to understand from the survey? In the sample survey below, it the objectives are as follows:

  • Discover how many hours a week our customers use our product

  • Discover who are our main competitors and what our advantages we have over them

  • Obtain customer's ideas on what would make them use our product more often

  • Understand our customer satisfaction level

Target audience - Who is going to participate in the survey? What representation is this of the customer base?

Method - How is the survey to be performed? Who from your company will do the survey? How will the target audience be recruited? Will it take place over the phone or via e-mail/web? What will you provide for an incentive or thank you? How will the results be compiled, analyzed, and reported?

Sample Small Scale Customer Survey

  1. How many hours a week do you use the product?

  2. Describe how you use the product?

  3. Did you evaluate other similar products before selecting this one?

  4. Which products did you evaluate?

  5. What made you select this product over the other products?

  6. What are the three things that would make you use this product for more hours a week?

Sample Sales Channel Survey Plan

As with the customer survey, it is best to write a plan first to call out the objectives and methods of the survey.

Objective - What are you trying to understand through the use of this survey? In the sample survey below, we are trying to find out which products are the easiest to sell and why, who is the competition and how do the sales people sell against them, and the sales perspective on how to make the products easier to sell.

Target Audience - Who are your surveying? Is it your entire sales force, your distribution channel, or select second-tier resellers?

Method - How is the survey to be performed? Who from your company will do the survey? How will the target audience be recruited? Will it take place over the phone or via mail/e-mail? What will you provide for an incentive or thank you? How will the results be compiled and reported?

Sample Sales Channel Survey

  1. Which products are the easiest to sell? Why?

  2. Which products are the most difficult to sell? Why?

  3. Who do you see as the major competitor for a specific product?

  4. How do you sell against this competitor?

  5. What are the three things that would make it easier for you sell this specific product?

  6. What are the most important features that this specific product is lacking? (You may want to supply a list and let them place them in the order of importance.)

For more detailed information on how to run customer surveys and compile the data, see "The Survey Research Handbook" by Pamela L. Areck and Robert B. Settle.

Small scale surveys are very effective at getting answers to your specific questions, but the ideal scenario is a quarterly customer review process, supported at all levels in the company, that allows you to participate as part of the review team.  If your company does not have a quarterly customer review process, you should do everything in your power to initiate one.

Customer Councils

Customers are surprisingly eager to attend customer councils. If they use your product often, they want to be taken seriously and they want their product suggestions to be heard. Having regular customer council meetings is a great way to discover the following information:

  • Validation of ideas for new products and features

  • Finding the major flaws in your current product

  • Discovering how customers are using your product

  • Validating which are the killer features of your current product

  • Discovering additional competitive information (your customers have probably evaluated your product next to some competitor)

  • Learning about future technologies your customers are evaluating and how this will impact your product's future

How often do you run customer councils?

Customer councils should be held twice a year or quarterly. You may or may not want the same group of customers each customer council. You may want to have two groups and invite them to alternating councils if you are doing them quarterly.

Who do you invite to the customer council?

Large customers are always important to your company and should be invited purely for improving customer relations with them. Additionally you want to invite customers who fall squarely into your target market for the product. You may also want to invite customers who are particularly vocal. One of two of these very vocal customers will make sure that meeting turns into a good two-way dialog and does not become a presentation by the company to the customers.

Logistics of the Customer Council

Ideally the customer council should be held at your company so that as many employees can attend as is appropriate. If this is not possible, pick a nearby hotel. Alternatively you can run customer councils out in the field for maximum regional penetration.

Invitations should go out well in advance, a couple months if possible, to allow customers to schedule the meeting appropriately. You should plan on supplying meals for your customers during their stay and have one event or dinner with the group. You should also give your customers a token gift to let them know you appreciate their time. A company or product logoed bag or briefcase is one example of a give-away that is nice for people traveling to your destination.

Sample Customer Council Agenda

  1. Introductions of company personnel and customers - It is important that everyone is introduced. Ask that the customers not only introduce themselves, but tell a little bit about how they are using your product.

  2. Overview of product line - You will sometimes be surprised that many of your customers don't know your current product line. Give a brief overview of the features and benefits of your current products

  3. Overview of product plans - This is the non-disclosure part of your presentation and it is worthwhile to mention that to your customers when you start the presentation. The objective is to get your customers' feedback on the product plans. Ideally you want give as much information as possible and include a demonstration of a prototype if feasible, to maximize the feedback you will get. You may not have another opportunity to have all your key customers in one room again before the product is introduced.

  4. Q&A - This should be a very open forum. Ideally you should concentrate initially on feedback on the product plans. It may help to facilitate the discussions if you have a list of other items for discussion. These items could include features the customers would like to see in your current product line, new products suggestions, new technologies that your customers are using in conjunction with your products, asking your customers how they get information on new products including (for example trade journals or tradeshows).

  5. Breakout sessions - It is sometimes easier to get feedback in smaller groups. Occasionally your customer's competitors are in the same room with them and they don't feel comfortable speaking out about how they are using your product. There may also be certain product areas that are more interesting to some then others. By using more focused sessions you should get better feedback.

  6. Survey - It is important that your customers feel that attending the customer council was useful for them. Be sure to get them to fill out a survey stating what they liked and didn't like about the customer council and any suggestions they have for the next one.

    Customer Visits

    Regular customer visits are the best way to understand the acceptance of your product. If you know your product well, the sales people will probably be delighted to take you on customer calls. If your company has a formal quarterly customer review process, then it is a good idea to attend some of these reviews in person.

    Another way to get in front of key customers is to make an appointment to present the product roadmap or a new version of the product under non-disclosure.  Be sure to make these sessions a two-way dialog. Ask the same type of questions you would ask in the customer council including the following:

    • What is your company direction as it relates to the use of our product?

    • What are your needs and requirements for this type of product?

    • Do the products we are developing fit your requirements? Where are they deficient? What do you like best about the products we have just talked about?

    • What competitor's products are you evaluating? How are those products better/worse than ours?

    • What other similar/compatible technologies are you evaluating?

    Always remember that this is a sales call and you are there to sell your products in addition to gathering feedback. Have answers prepared for the deficiencies of your product, but let the customer know that their input is not only appreciated, but will be taken into account in the product design process.

    Customer Advisory Board

    In addition to the customer council, you may feel like you need to gather more timely information on specific issues during the development cycle. Questions such as how something should be implemented or what platforms should be supported always come up during the development cycle. An excellent way to get quick answers is to build a customer advisory board that can be communicated with over e-mail. Like the customer council, customers are generally eager to have their voices heard. The following are some suggestions for running a productive advisory board:

    1. Try to limit your communications so that you don't overwhelm customers with e-mail.

    2. Make the e-mail concise focused on a specific question.

    3. Make sure the e-mail asks a specific question, you may even want to give some example answers as well to further define the type of information that you are looking for.

    4. A once a month e-mail is generally acceptable to most customers, any more often might be overwhelming.

    5. Don't expect everyone to respond to every e-mail, you want to get a large enough group that having a 30% response provides you with enough information.

    6. Use your customer council as a starting list for the advisory board. Also advertise on your web site for volunteers.

    User Testing

    User testing is an excellent way to get instant feedback on new products or product features. It is also a great way to involve engineering so that they get a good glimpse of how potential customer reacts to a product. There is a broad range of user testing from simple in-house testing to having a research firm run a full-scale test of your product. The following example focuses more on the quick and dirty, do-it-yourself user testing. As always, it is beneficial to write a short plan before starting any activity like this to make sure that everyone is clear on the objectives and exactly what is to be tested. You should also have a kick-off meeting to go over the plan with engineering and get their input on what should be tested.

    Sample User Testing Plan

    1. Objective - What are you trying to achieve with the user testing? List specific objectives, then review your plan to make sure it will meet the objectives. For a product user testing the objectives are generally one or more of the following: validation of a design, assessing ease of use, or validation of demand for the product within the target market.

    2. Specifics to be tested - It is important that everyone agrees on exactly what is being tested. List specific product features and capabilities to be tested.

    3. Method - How are you planning on executing the user testing? For product testing it is most useful to have one-on-one sessions with a select group from your target market. You should discuss the following information in this section:

      • Where are you going to hold the user testing?

      • What equipment are you going to use? (Video camera, tape recorder, and/or one person taking notes.) The combination of a video camera and person taking notes is generally the most effective.

      • How much information are you going to give the participants before they see or start using the product?

      • What questions are you going to ask at the conclusion of the session?

      • How is the feedback going to be reported to the rest of the company? I recommend that one person summarize the feedback in report form. The videos should be edited to show highlights listed on the report. A meeting should be held with interested parties in engineering and marketing to go through the report and the video clips.

    4. Selecting the target audience - How are you going to select user testing participants? One method for finding an target group of users is to advertise on your web site. Offer some token for participation and have a form that people can fill out that will allow you to qualify the prospects for the user testing.

    5. Schedule and frequency - When are you going to run user testing in the development cycle and how many are you going to have for a given product? Include a schedule for the user testing with how many days it will take to run the user testing and compile the information for the report.

    Getting Customer Input Back into the Development Cycle

    No matter how much customer exposure you have had, engineering groups may be reluctant to take your word for the fact that a product or feature needs to be added or changed. Clear, concisely presented evidence always makes your arguments much more compelling.

    If you are doing customer surveys be sure to include engineering in the summary meeting. You should have a meeting that gives a quick overview of your survey and a summary of your findings. Don't expect anyone to read through reams of survey material. Present your information in a concise easy-to-read form. Do, however, make the full survey material available for any enterprising individual that wants to delve into it. Remember that engineering really needs to understand exactly what the customer is trying to do, so that they can come up with creative ways to solve customer's problem.  

    If you are doing user testing make sure that engineering sits in on those summary meetings, watches the video clips and has full access to all notes and videotapes.  

    Always include engineering in any customer council meeting. It is great for them to meet and listen to real customers who use the product every day. The customers will also appreciate having engineering there. It will make them feel that the company is taking their input seriously.

    If you are making regular customer visits, write up short reports when you come back and e-mail them to the appropriate engineering group. You may find that your e-mail generates some interesting ideas. In all cases, customer input needs to be summarized at your regular product planning meetings.

    Supplying Useful Feedback

    Remember when giving feedback that your input should be in the form of market requirements, not necessarily product features. It is most important to inform engineering of what the customer is trying to accomplish.

    To illustrate this, I will use the following simple example. You sell a product that serves your customer two meals a day.

    What the customer says: I want the product to serve me three meals a day.

    What the customer's real problem is: Don't take the customer's statement at face value. Ask why they want a product that serves three meals a day. The customer says that they are often more hungry than twice a day. You may want to ask if they are consistently hungry three times a day. The customer may say, "I don't know, but I know that I am hungry more than twice a day." If you just added one meal a day to your product, you might not really be serving the customer's need.

    What you should report to engineering: The customer is hungry more than twice a day and would like a product that serves them when they are hungry.

    What engineering may come back with: Engineering may comeback with an incredible design that triggers meals based on integrating a new technology that picks up on the food thought patterns of the customer. In some cases this may be overkill, and you will need to negotiate schedule vs. technology. But the point is not to tell engineering specifically what to build. Tell them the problem you are trying to solve, it is their job to find the creative solution.

    All your customers should be treated as the critical resource that they are. They are all directly responsible for the success of your product and company.

    Further Reading

    A great book on understanding your customers and incorporating that input into the development cycle is Richard Whiteley and Diane Hessan's book Customer Centered Growth.

    Other Relationships

    This chapter has touched on the importance of customer relationships and ways to keep your products selling through a strong communication with your customers. But as a product manager, there are other relationships that you need to worry about as well. You will need to have relationships with both analysts and the press. Analyst relationships can and will be a two way street. Press relationships tend to be more of a one way communication - you communicating information to them and them communicating it to the world for you. One of the most important means of doing this is through the product review cycle. The next chapter will cover press relationships and writing a product evaluation guide for the press (which can also be turned into a customer evaluation guide).

    Author's note:  Ok, so I haven't written the next chapter yet, but take a look at the write-up that I have on a product evaluation guide to get started.  I promise to get back to writing soon!  Barbara Tallent