Location: Home » News & Views » Consultation » The Toolkit » Guidance 5.3

GUIDANCE 5.3 – SAMPLING
CALCULATING SAMPLING SIZE
Framing questions
Designing a postal or self completion questionnaire
Designing face-to-face or telephone interviews
Types of Questions
Piloting the questionnaire/interview
Raising response rates for postal questionnaires
Raising response rates for face to face interviews
Raising response rates for telephone interviews

GUIDANCE 5.3 – SAMPLING

The guidance below sets out the various methods of sampling. For guidance on Sample Size click here

Random Sampling

Random sampling is a sampling technique where you select a group of subjects (a sample) for study from a larger group (a population). Each individual is chosen entirely by chance and each member of the population has a known, but possibly non-equal, chance of being included in the sample.

By using random sampling, the likelihood of bias is reduced.

Simple Random Sampling

Simple random sampling is the basic sampling technique where we select a group of subjects (a sample) for study from a larger group (a population). Each individual is chosen entirely by chance and each member of the population has an equal chance of being included in the sample. Every possible sample of a given size has the same chance of selection; that is, each member of the population is equally likely to be chosen at any stage in the sampling process.

Stratified Sampling

There may often be factors which divide up the population into sub-populations (groups / strata) and we may expect the measurement of interest to vary among the different sub-populations. This has to be accounted for when we select a sample from the population in order that we obtain a sample that is representative of the population. This is achieved by stratified sampling.

A stratified sample is obtained by taking samples from each stratum or sub-group of a population.

Cluster Sampling

Cluster sampling is a sampling technique where the entire population is divided into groups, or clusters, and a random sample of these clusters are selected. All observations in the selected clusters are included in the sample.

Quota Sampling

Quota sampling is a method of sampling widely used in opinion polling and market research. Interviewers are each given a quota of subjects of specified type to attempt to recruit for example, an interviewer might be told to go out and select 20 adult men and 20 adult women, 10 teenage girls and 10 teenage boys so that they could interview them about their television viewing.

For further information contact the Research and Intelligence Unit, County Hall, Spetchley Road, Worcester, WR5 2NP – Tel 01905 766175

(With Thanks to Lancashire University for the information)

CALCULATING SAMPLING SIZE

Sample sizes are not dependent upon the size of the population. As long as the sample represents a small proportion of the population and is randomly selected, then it will give reasonably accurately results. For general surveys, most research companies use 1,000 responses.

If you want to analyse sub-groups, you will need at least 100 responses from each group to ensure a degree of accuracy – so you could begin by deciding how you want to analyse the results and build up a sample size from there. For example if you wanted to look at the differences between men and women – you will need at least 100 men and 100 women in your sample. If you have a range of different age groups or ethnic backgrounds, then again you will need to ensure you have at least 100 people in each category.

For postal surveys, you will need to send out to a wider sample to make up for low returns.

SAMPLE ERROR

Surveys represent a snapshot in time and can only be approximate: if another sample was selected, the results may differ. In a random sample, it is possible to calculate this ‘sampling error’ and the results are often called ‘confidence limits’. Generally, the larger the sample size, the lower the sampling error will be.

For example, the table below shows the approximate error for a question where 50% of respondents gave a particular answer:

Sample size Error(+/-%)

100 10

200 7
500 4
1000 3
(IdeA Knowledge)

For more information on Sampling, Calculating Sample Size and Confidence Intervals view the Brent Council Consultation Toolkit.

http://www.brent.gov.uk/consultation.nsf/24878f4b00d4f0f68025663c006c7944/1ba5f40a89659aaa80256b28005e24ff/$FILE/Consultation%20Toolkit%20Apr%202002.doc

SAMPLE SIZE CALCULATOR

Use this handy sample size calculator:

http://www.surveysystem.com/sscalc.htm

Framing questions

• Questions should be ‘neutral’ and not lead people to answer in a particular way.
• Do not ask two questions in one, or make the question too long and overcomplicated.
• Do not use difficult or technical language or ask hypothetical questions such as asking the respondent to imagine their reaction in a different situation.
• Where possible use pre-coded, tick box response questions where the answers are provided for the respondent to simply tick their choice or in the case of face-to-face surveys read out a number on a showcard.
• Avoid open-ended questions, people are less likely to answer these as they require work and thought on their behalf. Open-ended questions involve a heavy workload in terms of data processing and analysis.

Designing a postal or self completion questionnaire

• Split the questionnaire into sections that flow logically and begin with a brief paragraph introducing each section.
• Keep the wording and design consistent throughout the survey.
• Choose an appropriate font and size (11-13pt) and highlight questions and important text in bold. However do not go over top with formatting!
• Design the questionnaire so that is appealing to the eye. Do not overload pages with questions or information, white space is a good thing. Maximise your margins to make full use of the space on the page.
• Number questions and parts of questions logically and clearly.
• Provide ‘GO TO’ rules instructing respondents where to go and what questions to answer next. Think about how changing the order of your questions can help this.
• Help the respondent by stating ‘Tick one only’ or ‘Tick all that apply’.
• Be consistent in your design and question framing i.e. always fro from positive to negative or 1 to 5.
• Ask straightforward questions first, leaving any sensitive, more difficult or contentious questions until last. You can ask ‘About You’ questions either at the beginning or the end of the survey.
• Do ask ‘About You’ questions which will add value to your analysis. Analyse results by groups such as age group, sex, district etc and compare results to see how opinions vary. What you ask here depends on your survey topic.
• If you ask open ended questions ensure you leave enough space for comments. Provide blank boxes rather than lines which limit what people can fit in, everyone’s handwriting is different. Also consider how you are going to analyse this information and take it into account.

Designing face-to-face or telephone interviews

• Give the interviewer(s) a detailed briefing about the purpose of the exercise and how you are going to use the results. Provide a laminated letter of authenticity for the interviewer to show the respondent.
• Everything the interviewer needs to say should be provided on the questionnaire for them to simply read out to the respondent.
• Split the questionnaire into sections that flow logically. Write brief introductions that the interviewer reads out before each section.
• Keep the wording consistent throughout the survey.
• Number questions and parts of questions logically and clearly for the interviewer to follow.
• Provide detailed instructions, verbatim and ‘GO TO’ rules for the interviewer to follow throughout the whole survey. Think about how changing the order of your questions can help this.
• The interviewer should indicate to the respondent questions where they can choose one option or more than one.
• For face-to-face surveys use laminated showcards that list and number the responses to each question. The respondent then simply reads out the number that matches their answer.
• Be consistent in your design and question framing i.e. always from positive to negative or 1 to 5.
• Ask straightforward questions first, leaving any sensitive, more difficult or contentious questions until last. You can ask ‘About You’ questions either at the beginning or the end of the survey.
• Do ask ‘About You’ questions which will add value to your analysis. Analyse results by groups such as age group, sex, district etc and compare results to see how opinions vary. What you ask here depends on your survey topic.
• If you ask open-ended questions ensure you leave enough space for comments that the interviewer writes down. Provide blank boxes rather than lines which limit what people can fit in, everyone’s handwriting is different. Also consider how you are going to analyse this information and take it into account.

Types of Questions

Pre-coded or closed questions – a list of pre-coded responses are provided for the respondent to simply ring or tick. Each option is assigned a numeric code which is used for data input and analysis. The option of ‘Other’ may also be offered with space for respondents to write in their answer. Be sure to include a full range of options so that you don’t get lots of people ticking ‘Other’.

Rating or attitude scales – can be used to measure agreement or satisfaction. Ensure that you provide a fair range of responses that are balanced e.g.

How satisfied are you with the bus service overall?

Very satisfied, Fairly satisfied, Neither, Dissatisfied, Very satisfied

Or

To what extent do you agree or disagree with the following statement “The quality of Council services is good overall”

Strongly agree, Agree, Neither, Disagree, Strongly agree.

Open ended questions – for these question types no categories or answers are suggested, the respondent is free to express themselves in their own words.

E.g. Why are you dissatisfied or very dissatisfied with the service provided?

(Please write in the box below)

Classification questions – ‘About You’ questions such as sex, age group, employment status, tenure, ethnicity. These are very useful in breaking down the results into different groups for analysis.

Piloting the questionnaire/interview

Pilot the questionnaire before doing it for real. Test either with colleagues or a sample of people. Check the questionnaire and showcards thoroughly for spelling errors, go to rules, instructions, responses, question numbering and coding. Do take people’s feedback into account, if they say they don’t understand something or find it difficult to complete or answer then you should do something about it! Time how long it takes someone to complete the questionnaire or interview it may take longer than you think.

Raising response rates for postal questionnaires

• Include the Worcestershire County Council logo on the mailing envelope will aid recognition and makes your letter stand out from the rest. Reprographics can arrange to have our logo printed on your envelopes, note this can take up to 4 weeks.
• Include a clear, concise covering letter signed by an appropriate senior officer signifying its importance. Underline what will happen with their answers, explain why it is important that they respond.
• Don’t make your questionnaire too long people will be put off straight away. Consider stating how long it should take them to complete or using another method.
• Include a pre-paid envelope so that people can easily return their completed survey.
• Send a reminder or reminders two to three weeks after you sent the survey. You should only send reminders to those people who have failed to respond. Sending reminders to everyone is wasteful and will be perceived as inefficient. To do this you must assign ID numbers to the people you send surveys to. This ID number is then written onto the survey and recorded as completed surveys are returned.
• Offer an incentive. Selecting one respondent at random who wins an appropriate prize. Ensure you do not over or under spend and cash may not always be appropriate (it can affect people’s benefits) consider vouchers. LINK See Social Services guide to incentives LINK

Raising response rates for face-to-face interviews

• Send a letter in advance of the interviewer calling at the respondent’s house, explain what the survey is about and why it is important. Include your contact details so that residents can contact you should they not wish to take part.
• Interviewers should carry and ID badge and a signed letter of authentication.
• Brief the interviewer fully so they can answer any questions the respondent might have about the survey.
• Give interviewers a clear opening statement to read out when they call at a person’s house. Don’t be too forceful but be persuasive and truthful.
• Give an indication of how long the interview might take.
• If residents appear reluctant offer to call back at a more convenient time.
• Interviews are best conducted in the evening or weekends. If residents are out interviewers should call back at least 4 times. Leave a calling card/letter asking them to contact you to arrange an interview.
• Offer an incentive such as entry into a prize draw.

Raising response rates for telephone interviews

• Give interviewers a clear opening statement to read out when they call at a person’s house. Don’t be too forceful but be persuasive and truthful.
• Give an indication of how long the interview might take.
• If residents appear reluctant offer to call back at a more convenient time.
• Interviews are best conducted in the evening or weekends. If residents are out interviewers should call back at least 4 times.
• Offer an incentive such as entry into a prize draw.