Questionnaire construction

Questionnaires – series of questions asked to individuals to obtain statistically useful information about a given topic[1] – when properly constructed and responsibly administered, become a vital instrument by which statements can be made about specific groups, or people, or entire populations.

Questionnaires

Questionnaires are frequently used in quantitative marketing research and social research. They are a valuable method of collecting a wide range of information from a large number of individuals, often referred to as respondents, it can be students, workers or any person whom you require information from.

Adequate questionnaire construction is critical to the success of a survey. Inappropriate questions, incorrect ordering of questions, incorrect scaling, or bad questionnaire format can make the survey valueless, as it may not accurately reflect the views and opinions of the participants.

Different methods can be useful for checking a questionnaire and making sure it is accurately capturing the intended information.

Initial advice may include

Empirical tests also provide insight into the quality of the questionnaire. This can be done by:

Questionnaire construction issues

Methods of collection

There are a number of channels, or modes, that can be used to administer a questionnaire. Each has strengths and weaknesses, and therefore a researcher will generally need to tailor their questionnaire to the modes they will be using. For example, a questionnaire designed to be filled out on paper may not operate in the same way when administered over the phone. These mode effects may be substantial enough that they threaten the validity of the research.

Using multiple modes can also improve access to the population of interest when some members have different access, or have particular preferences.

Method Benefits and cautions
Postal
  • Usually a simple questionnaire, printed on paper to be filled out in a pen or pencil.
  • Low cost-per-response for small samples. Large samples can often be administered more efficiently by using Optical Character Recognition, or by using other modes.
  • Mail is subject to postal delays and errors, which can be substantial when posting to remote areas, or given unpredictable events such as natural disasters.
  • Surveys are limited to populations that are contactable by a mail service.
  • Reliant on high literacy levels
  • Easy to allow survey participants to remain anonymous (e.g. using identical paper forms)
  • Limited ability to build rapport with the respondent, or to answer questions about the purpose of the research.
Telephone
  • Questionnaires can be conducted swiftly, particularly if computer-assisted
  • Opportunity to build rapport with respondents may improve response rates,
  • Researchers may be mistaken for being 'tele-marketers'
  • Surveys are limited to populations with a telephone.
  • Are more prone to social desirability biases than other modes, so telephone interviews are generally not suitable for sensitive topics[8][9]
Electronic
  • Usually administered via a HTML-based webpage, or other electronic channel such as smartphone app.
  • This method has a low ongoing cost, and on most surveys costs nothing for the participants and little for the surveyors. However, Initial set-up costs can be high for a customised design due to the effort required in developing the back-end system or programming the questionnaire itself.
  • Questionnaires can be conducted swiftly, without postal delays.
  • Survey participants can choose to remain anonymous, though risk being tracked through cookies, unique links and other technology.
  • It is not labour-intensive.
  • Questions can be more detailed, as opposed to the limits of paper or telephones. {Respicius, Rwehumbiza (2010)}
  • This method works well if the survey contains several branching questions. Help or instructions can be dynamically displayed with the question as needed, and automatic sequencing means the computer can determine the next question, rather than relying on respondents to correctly follow skip instructions.
  • Not all of the sample may be able to access the electronic form, and therefore results may not be representative of the target population.
Personally administered
  • Questions can be more detailed and obtains a lot of comprehensive information, as opposed to the limits of paper or telephones. However, respondents are often limited to their working memory: specially designed visual cues (such as prompt cards) may help in some cases.
  • Interviewers sometimes rephrase questions during the interview, reducing the level of standardisation. Computer-assisted personal interviewing may assist with this.
  • Rapport with respondents is generally higher than other modes
  • Typically higher response rate than other modes.
  • Can be extremely expensive and time-consuming to train and maintain an interviewer panel. Each interview also has a marginal cost associated with collecting the data.
  • Relatively few limits to the population, as long as an interviewer is granted access.

Types of questions

  1. Contingency question – A question that is answered only if the respondent gives a particular response to a previous question. This avoids asking questions of people that do not apply to them (for example, asking men if they have ever been pregnant).
  2. Matrix questions – Identical response categories are assigned to multiple questions. The questions are placed one under the other, forming a matrix with response categories along the top and a list of questions down the side. This is an efficient use of page space and respondents’ time.
  3. Closed-ended questions – Respondents’ answers are limited to a fixed set of responses. Most scales are closed-ended. Other types of closed-ended questions include:
    • Yes/no questions – The respondent answers with a "yes" or a "no".
    • Multiple choice – The respondent has several option from which to choose.
    • Scaled questions – Responses are graded on a continuum (example : rate the appearance of the product on a scale from 1 to 10, with 10 being the most preferred appearance). Examples of types of scales include the Likert scale, semantic differential scale, and rank-order scale (See scale for a complete list of scaling techniques.).
  4. Open-ended questions – No options or predefined categories are suggested. The respondent supplies their own answer without being constrained by a fixed set of possible responses. Examples of types of open-ended questions include:
    • Completely unstructured – For example, "What is your opinion on questionnaires?"
    • Word association – Words are presented and the respondent mentions the first word that comes to mind.
    • Sentence completion – Respondents complete an incomplete sentence. For example, "The most important consideration in my decision to buy a new house is . . ."
    • Story completion – Respondents complete an incomplete story.
    • Picture completion – Respondents fill in an empty conversation balloon.
    • Thematic apperception test – Respondents explain a picture or make up a story about what they think is happening in the picture

Question wording

The way that a question is phrased can have a large impact on how a research participant will answer the question.[10] Thus, survey researchers must be conscious of their wording when writing survey questions.[10] It is important for researchers to keep in mind that different individuals, cultures, and subcultures can interpret certain words and phrases differently from one another.[10] There are two different types of questions that survey researchers use when writing a questionnaire: free response questions and closed questions.[10] Free response questions are open-ended, whereas closed questions are usually multiple choice.[10] Free response questions are beneficial because they allow the responder greater flexibility, but they are also very difficult to record and score, requiring extensive coding.[10] Contrastingly, closed questions can be scored and coded much easier, but they diminish expressivity and spontaneity of the responder.[10] In general, the vocabulary of the questions should be very simple and direct, and most should be less than twenty words.[10] Each question should be edited for "readability" and should avoid leading or loaded questions.[10] Finally, if multiple items are being used to measure one construct, the wording of some of the items should be worded in the opposite direction to evade response bias.[10]

A respondent's answer to an open-ended question can be coded into a response scale afterwards, [11] or analysed using more qualitative methods.

Question sequence

Marketings

References

  1. ↑ Merriam-Webster's Online Dictionary, s.v. "questionnaire," http://www.merriam-webster.com/dictionary/questionnaire (accessed May 21, 2008)
  2. ↑ Dillman, Don A., Smyth, Jolene D., Christian, Leah Melani. 2014. Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method, 4th edition. John Wiley: Hoboken, NJ
  3. ↑ Lord, F. and Novick, M. R.(1968). Statistical theories of mental test scores. Addison – Wesley.
  4. ↑ Heise, D. R.(1969). Separating reliability and stability in test-retest correlation. American Sociological Review, 34, 93-101.
  5. ↑ Andrews, F. M. (1984). Construct validity and error components of survey measures: a structural modelling approach. Public Opinion Quarterly, 48, 409-442.
  6. ↑ Saris, W. E. and Gallhofer, I. N. (2014). Design, evaluation and analysis of questionnaires for survey research. Second Edition. Hoboken, Wiley.
  7. ↑ Timothy R. Graeff, 2005. "Response Bias," Encyclopedia of Social Measurement, pp. 411-418. ScienceDirect.
  8. ↑ Frauke Kreuter, Stanley Presser, and Roger Tourangeau, 2008. "Social Desirability Bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity", Public Opinion Quarterly, 72(5): 847-865 first published online January 26, 2009 doi:10.1093/poq/nfn063
  9. ↑ Allyson L. Holbrook, Melanie C. Green And Jon A. Krosnick, 2003. "Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias". Public Opinion Quarterly,67(1): 79-125. doi:10.1086/346010.
  10. 1 2 3 4 5 6 7 8 9 10 Shaughnessy, J.; Zechmeister, E.; Jeanne, Z. (2011). Research methods in psychology (9th ed.). New York, NY: McGraw Hill. pp. 161–175.
  11. ↑ Mellenbergh, G.J. (2008). Chapter 9: Surveys. In H.J. Adèr & G.J. Mellenbergh (Eds.) (with contributions by D.J. Hand), Advising on Research Methods: A consultant's companion (pp. 183–209). Huizen, The Netherlands: Johannes van Kessel Publishing.
This article is issued from Wikipedia - version of the 10/15/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.