Market research surveys are the stock in trade for gathering any sort
of market information. The aim is to gather information from a small
sample of a market in order to be able to predict what the whole market
wants.
In principle anyone can conduct their own market research surveys given
the time, an openness to listen and a little bit of learning.
Types of market research
Market research surveys can be divided in two - quantitative research
and qualitative research (a third category is observation studies, but
these are less common).
Quantitative research (quant) is about getting the hard
measures of a market - market share, how many people think... , how many
people saw our advertising, how many people would buy... . Qualitative
research (qual) is about the softer issues exploring why people
do things or think the way they do. These types of survey are usually
complementary - you might explore the reasons why people buy in qual, then
measure how important these reasons are in quant.
Quantitative surveys mean getting people to answer fixed questions in
questionnaires. These questionnaires can be completed over the telephone,
face-to-face, through the post or via the Internet. Because the objective
is measurement it is important that all people answer the same question.
Changes to wording can dramatically change your measurements. Consequently
quantitative interviewers are trained not to stray from the script and
good questionnaire design is extremely important.
Qualitative surveys are about exploring an issue with people. There are
no fixed questionnaires and interviewer use their interviewing skill to
draw views and opinions from people using a discussion guide (there are
some differences in the US and UK views of how qualitative research is
conducted). There are two main approaches to qualitative research - depth
interviews (one-on-one interviews) and the focus group (aka
discussion group) where the group dynamics means that individuals spark
ideas and discussion off each other to explore a topic. It is important
that the qualitative interviewer doesn't bias the discussion and lets the
interviewee(s) describe things in their own terms - how something is said
can be as important as what is said. Qualitative work is typically carried
out in person, but it can be conducted over the telephone or over the
Internet.
Both quantitative and qualitative research are based on the notion of
sampling to identify who to talk to and the idea of the interviewer as an
unbiased observer or collector of information.
Samples
At the core of market research is the idea of a sample. A sample is a
group of people drawn at random from all those people who are in your
chosen market. By "drawn at random" we mean that each person in
the market has the same probability of being questioned as any other. For
instance if you had a database of customers and wanted to select customers
"at random", you might put this list in alphabetical order and
take every nth person.
Some care is needed in choosing how to select a sample. If you do not
make the sample random, you risk introducing bias and so get misleading
results. For instance if you choose 1 in n people from a telephone
book, you miss out all the ex-directory phone users, who are more likely
to be the wealthier individuals. Typically a researcher will use their
judgement and experience to trade off the quality of the sample against
the cost or difficulty of contacting the right people.
Sample size
If you took a random sample of UK voters and 25% said they would vote
for the Blue party you would want to know how accurate this was. Random
samples are subject to known statistical laws which describe how accurate
each sample is likely to be.
If your sample was 10,000 people you would be 95% confident that this
reflected the views of all voters +/- 0.85 percentage points. In other
words there is a 19/20 chance that the "real" share of the vote
for the Blues would be 24-26%. (95% is the usual level of confidence
quoted, but you can estimate accuracy to any confidence level - 99 in 100,
999 in 1000 etc).
Because contacting people and making interviews costs money, there is a
trade-off between sample size and required accuracy. If in the example
above the sample size was different you would have different levels of
confidence.
|
Sample size |
Error (95% level) |
Accuracy
range of "25%" result |
|
10,000 |
+/-0.85 |
24-26% |
|
5,000 |
+/-1.20 |
24-26% |
|
1,000 |
+/-2.68 |
22-28% |
|
500 |
+/-3.80 |
21-29% |
|
100 |
+/- 8.49 |
17-33% |
|
50 |
+/-12.00 |
13-35% |
|
10 |
+/-26.8 |
0-52% |
If you look at the table above, there is practically no benefit in
going from a sample of 5,000 to a sample of 10,000 yet your costs would
have doubled. In fact, accuracy doubles every time you quadruple your
sample so accuracy can be very expensive.
Nonetheless, the sample size you use depends on the type of survey you
are trying to carry out. If you are trying to assess the impact of small
price rises or changes to attitudes after an advertising campaign, or
changes to satisfaction then you may want real fine accuracy (to +/-2%).
But if you just want a "yes/no" decision you can take a broader
view of results and accuracy of +/-10% may be entirely suitable. For most
quantitative market research studies a common rule of thumb is that you
aim to talk to 100 people for each significant subgroup in your market.
Note that the sample size is pretty much independent of the size of the
market you are looking to research (unless there are very very few
customers - under 500). So the error in researching a market of 100m
customers is
almost exactly the same as in researching a market of 100,000. You can't
cut the sample for a smaller market.
For qualitative research, the aim is not measurement but exploration.
Consequently sample size is less of an issue. Of more importance is to get
a complete picture of how a market works. However, clearly if you only
talk to 5-6 people the usefulness of the research will be less than if you
talked to 12-16 people.
Quantitative questionnaire design
Good questionnaire design makes it easy for the person answering to
answer and is clear, unambiguous and doesn't lead the person answering in
one direction or another.
Common problems are to ask questions that are too difficult to answer(eg
"Would you buy X in five years time?"), or unclear in their
meaning ("Do you think relationships will change the world?"),
or have double meanings ("How hot was your spicy pizza when you ate
it?"), or contain two questions ("Do you think corrupt
politicians and charity workers should be eradicated?"), or lead in
some way ("Do you use your air-polluting gas-guzzling car less often
than you use to?"), or fail to address the problem from the
interviewees point of view (ignoring key service elements is
common).
The order in which questions is asked can also be important. The
adverts that can be remembered without prompting (ie spontaneously - Which
ads can you remember seeing) are more likely to be top-of-mind than those
remembered with prompting (Which of these ads can you remember seeing).
Questionnaires, therefore typically run from the very general to the very
specific as this also helps the flow which reduces the time taken to
complete the interview. Consequently, interviewers are trained not to lead
the interviewees and to probe to get fuller answers (Why is that? Could
you clarify that for me? Could you explain), without prompting (Did you
say Mustang or Mazda?).
Ideally all questionnaires should be piloted. To find that questions
don't work after speaking to 1000 people is not very helpful. However, a
good initial test even before a pilot is to ask a member of your family to
complete the questionnaire as this will typically highlight the most
glaring of problems.
Qualitative research designs
The design of qualitative research depends on the subject and the
audience and the skills of the interviewer/moderator. If the subject is
personal or private then depth interviews will be better than group
discussions, if the subject is more widely considered and discussed then
focus groups may be best as the stimulation for discussion comes from the
group members. For certain types of respondent (eg business people), a
face-to-face group discussion may be not be feasible although telephone
conferences or email groups may be alternatives.
Interviewers themselves are also important in the qualitative process
as the level and quality of probing and questions will depend on the
interviewers understanding of the subject at hand, in order to identify
areas that need clarification or further exploration and to provide a
point of reassurance to draw more out of the interviewee. It is difficult
to discuss the issue of hedging with a financial director unless you
recognise that this might have something to do with exchange rate risks
and to have some understanding of what the implications of hedging could
be.
Often in qualitative research, it can be difficult for the interviewee
to express themselves adequately in words particularly where you are
looking at the emotional appeal of something as opposed to its practical
features and benefits. Researchers use a lot of games and prompts to draw
out different views. For instance in projective techniques researchers
might ask someone to say "If X were an animal, what type of animal
would it be?"and then explore the reasons behind these views. Most
researchers have a kitbag of tools and techniques that they find useful to
help capture views and responses. Typically the closer you are to the
creative design arena, the more exotic these techniques become. But for
most work, plain vanilla qual with an interviewer who knows the subject
area (but doesn't lead the respondent) are to be advised.
Group discussions add another twist as the interviewer in the guise of
a moderator has to ensure that the group captures and includes the views
of all the participants. There are dangers that one or two voices
pre-dominate negating the benefits of the group environment.
A key element of qual research is that you should record the discussion
and then transcribe notes later. Relying on memory is often not good
enough and listening to the discussion guide a second time can change your
opinion of what the interviewee really meant. Typically qualitative
research is reported as finding supported by quotes.
Doing it yourself
Typically the main problem with DIY is not so much not being able to do
it, but not having the experience of what works and what doesn't. For more
advanced subject areas, design expertise can be crucial. Having said that
if the first run is not crucial, then the experience will be really
helpful subsequently whether you do-it-yourself later or buy in
expertise.
The key steps are:
- Select the type of research you need (Qual/Quant)
- Choose how you want to get in contact/recruit the relevant people
- Design a questionnaire/discussion guide
- Recruit the relevant people/interview them/send out the
questionnaire
- Amalgamate and analyse the results.
The recruiting/interviewing task is the biggest. There are many
companies that will just do fieldwork for you or arrange interviews if you
provide them with lists. There are also many DP/transcription companies
that will process your completed questionnaires or transcribe your
interviews for analysis. All these steps can be carried out by hand, but
the biggest issue is normally time which is where sharing the workload
will help.
In terms of cost, most market
research is charged on a time basis plus a management/design
fee. If you took a general face-to-face population study of 1000
people. You might allow 15 minutes for the interview, 20 minutes
finding/contact time, 10 minutes for processing each
questionnaire - so 45 minutes per interview or 750 hours of time
(100 days), on top of which would be added time for
questionnaire design, production and dispatch, interviewer
briefing and management, creation of tables, analysis and
presentation. Typically on a straightforward survey these
elements should add about 15-20 days, although at a higher daily
rate.
For more complex studies, contact
time can have a far larger impact on the cost than the actual
questionnaire length. However, questionnaires that are too long
demotivate interviewers and put respondents off research and
typically need high levels of incentives to encourage
completion. For on-line surveys, sample cost can be a extremely
large element of the final bill.
Qualitative research is charged
on a similar basis to quantitative research, but as sample sizes
are smaller it is typically less expensive. However although
recruitment is far less expensive, the interviewer is more
expert and interview and analysis time and costs are typically
higher reducing the difference in cost between small quant and
larger qualitative work.
