Why do a survey?
The basic purpose for conducting a survey is to gain information to help with planning and marketing decisions. Whether your business involves a product or a service, having reliable information about what your customers want is essential. A survey is an objective tool for obtaining this information through asking questions to a sample of your current or potential customers.
What type of survey is best?
There are two basic types of surveys: self-selected and statistically valid random sample.
- A self-selected survey is a written questionnaire distributed through the mail, a newspaper or magazine, or simply left at a counter for someone to complete. In each case, the respondent chooses to participate (i.e. selects himself or herself). This type of survey can be politically wise if a company wants to make a statement that anyone who wants to participate in the survey can choose to do so. There are also times when it is valid to seek the opinions of people who care enough about your business that they are willing to share their ideas. However, self-selected surveys are not appropriate when you are looking for a reliable reading on what your total customer base or target audience is thinking.
- A statistically valid, random sample survey is based on scientific principals that make it possible to predict the responses of a total population, within a given error range, based upon a random sample of that population. The key here is that proper random sampling techniques must be used to assure that everyone in the population has an equal chance of being selected for the survey. Most valid random sample survey processes do data collection through phone calls or by in-person intercepts. The size of the sample determines the level of accuracy for a survey. The industry standard is a random sample of 200 (for any population over 2,000), which provides an error range of +/- 7% with a 95% confidence. In marketing research terms, this means that in 95 out of 100 cases where 50% of the population says “Yes” and 50% says “No” the results will be accurate for the total population from which the sample was drawn with an error range of +/- 7% (as the percentages become further apart – i.e. 40% and 60% the error range decreases).
A survey will be as valuable as the care taken to develop its focus and content. This is why a survey process is so important.
The planning workshop
The first step, and to my mind the most important, is to have a planning workshop to discuss, and agree upon, the goals and specific information needs for the survey. This is the time for each person who will be using the survey results to provide a list of information needs.
The project manager, or project team, determines who should participate in this workshop. It may also be valuable, and sometimes politically wise, to do one-on-one interviews with people deemed to have a special interest in the outcome of the survey who will not be attending the workshop.
Choose a setting that will be relaxed and comfortable, and allow two to three hours for the workshop This is not intended to be a formal, kick-off meeting where tasks are assigned, but rather a place for laundry-listing and discussing what is desired from the survey.
Information needs tend to exceed what can be effectively accomplished in a single survey, so the planning workshop is the place to reach agreement on priorities, as well as content.
Subjective input for an objective research tool
A representative sample of those being surveyed should be consulted prior to writing the survey questionnaire. Even though goals and specific information needs have been established, there may be issues, concerns or semantics that need to be understood and addressed.
A focus group discussion is an effective method for pre-testing subject matter and being sure that the survey addresses issues that are relevant to those being surveyed. The focus group discussion also checks out the language of the questionnaire to be sure those who will be surveyed will easily understand it.
A few years ago, I was doing a survey for an update of a comprehensive park plan for a Washington city. In a focus group, we learned that, while the public appreciated the many new neighborhood parks, they were disturbed that no restroom facilities were provided in most of these new, small parks. This was not intended to be an issue to be surveyed. When the subject was included in the survey questionnaire, it turned out to be the top priority spending choice among those surveyed.
Designing the survey questionnaire
The survey questionnaire is developed based upon the information gathered in the individual interviews, planning workshop and the focus group discussion.
Once the draft questionnaire is written, the project manager decides who should review it. It is essential that the project manager (or someone designated by the project manager) be the final arbiter of the precise content and language of the survey instrument. There are word connotations and personal agendas behind opinions, and a consultant’s time should not be used to settle internal agency disagreements. My preference is that any number of people can review the draft, but one person makes the final decisions on any edits.
Data Collection
There are several excellent local data collection companies. What is important is that a professional data collection service is used for fielding the survey. A professional interviewer (as opposed to volunteer or a student) is paid to get through each interview as efficiently and courteously as possible.
Before the data collection begins, I always meet with the interviewers and train them by reviewing each question and explaining the types of responses that are likely. The first few calls for each interviewer are monitored to be sure that words (especially names of places and people) are pronounced correctly, the probing is done appropriately when the respondent is hesitating, and that responses are not being biased in any way.
Analyzing the data and preparing the report
A well-written survey report should include a clearly stated purpose and methodology. Someone not well versed in statistics should understand language, and the conclusions and recommendations should be documented in easy-to-read tables and charts.
An Executive Summary precedes the detailed report and provides a stand-alone piece that can be widely distributed.
Using the information obtained from a survey
As a consultant, nothing is more frustrating than to do a good job for a client only to learn the information was left on the shelf because people weren’t sure how to translate the information into action.
I encourage my clients to have a follow-up workshop where the results of the survey are explained and discussed, and a strategic plan of action is developed.
About the Author: Carolyn Browne Associates (CBA) has been a successful consulting firm in the Seattle area for over 25 years and specializes in community involvement programs, marketing research, facilitation, promotion and community education projects for a broad range of public agencies and private clients. Carolyn Browne Tamler, principal of CBA, has managed comprehensive programs with special focus on city planning, public transit, environmental issues and public works projects. She is also a fine researcher and freelance writer. You can also learn more at www.envisionyourfuture.biz.
