Ensuring your survey sample reflects your target audience

In this post we’ll be exploring some of the ways of ensuring[nbsp_tc]those taking your online survey are representative of your target audience. In other words that your survey sample reflects the population you are studying.

Why do you even need to worry about who is taking your survey? Simple. If you’re making business decisions or even just communicating your findings you need to have confidence that the reported views and behaviour of your respondent sample reflects those of the wider group.

The group or audience you’re studying could be your user base, a target segment or even the entire population.

a reminder that you are running an online survey

The first consideration is a reminder that you are (probably*) running an online survey. By it’s nature your survey sample will very likely exclude anyone not online.

This means that your findings will naturally represent a slightly more technically-savvy audience. Any questions you ask on device use and ownership, for example, will give results that likely differ from an otherwise-identical telephone or face to face survey.

It is possible to correct this bias by supplementing your online survey sample with an offline study of the 10-30% (depending on your market) of adults not online. Alternatively you can correct your data by down-weighting the data. But simply being aware of the nature of any bias in the data is generally sufficient for most market researchers.

If your survey is a quantitative study of your website or app visitors, an online survey is simply perfect. However, there are still considerations to ensure that the sample reflects the user profile.

If you choose to embed a survey invitation on your website you may be more likely to reach those who are keen to air their thoughts, usually because their experience has been at one of the 2 extremes: very poor or excellent.

You may miss out on the (likely) large section of your users who have had an ‘ok’ experience.¬†Perhaps it met expectations, was reasonably efficient but instantly forgettable. Something that is rarely a bad thing!

it is generally better to run a pop-up invitation

To ensure a good cross section it is generally better to run a pop-up invitation (N.B. there are a few settings options to consider here) to appear to all visitors, or a certain percentage of them.

If you are running a pop-up invitation to your survey it is advisable to keep the survey live over a minimum of a full day, even if you have very high traffic levels. This ensures that you are inviting responses from different user profiles across the day.

Let’s say you run a parenting information service and are doing an evaluation of your website. You would need to collect responses from stay-at-home parents who may respond during the day and also working parents who may be more likely to visit your site in the evening.

Depending on your site it may also be preferable to run your survey during both the week and weekend. Again you may find you reach a different user profile.

By running a pop-up invitation over a number of days you can ensure you are reaching a good cross-section of users. If you already hold good demographic or other data on your users you can check the respondent profile coming through to ensure it is consistent.

if you are using social media, blogs and other external sites you have much less control over those opting to take your survey

If, instead, you are running a market study you have to be more careful with your respondent recruitment. This is because you are likely to be reaching well beyond the confines of your user or customer base.

With a reasonable budget available you can ‘buy’ a survey sample from an audience panel provider such as Google or Survey Monkey. This gives you confidence that you are reaching a target demographic.

But if you are using social media, blogs and other external sites you have much less control over those opting to take your survey.

Many surveys use incentives to attract responses, but it can be a mistake if you end up reaching those who are only interested in the potential reward.

For a truly representative market study, especially one that will inform business and investment decisions, we’d recommend buying a quality sample. Or at the very least not offering a prize incentive.

So we’ve discussed some of the considerations for attracting a survey sample that is as representative as possible of the audience you are studying.

To get some free advice on your project please do contact us!

* you can use online survey software to collect data offline too