Questionnaires and surveys are useful for determining the attitudes, characteristics, profiles, beliefs from a sample of people. In designing a questionnaire that focuses on open ended questions the main concern should be on determining the way the responses will be quantified.
Open ended questions are often used in research to clarify closed questions or as a means of generating ideas. In both cases it is generally assumed that the longer and more detailed the response to an open ended question the better (Graziano et al, 2000). However, relatively little is known about the effect of factors such as question wording or questionnaire design on the response to open ended questions.
There is a need to ask open ended questions when the researcher desires to probe deeper into a subject area. The use of open ended questions is more common in subjects which are relatively unrehearsed. Open ended questions are also of value, where response cannot be predicted, or maybe to the reduce potential for researcher induced bias. This method makes it possible to explore a range of ideas about a certain issue or topic.
Open ended questionnaire questions can be also be quantified but in more indirect way. “Converting raw, open ended data from large sample sizes into meaningful categories that the researcher can utilize to quantify the results presents a challenge” (Culp, 1998). Several strategies can be used to convert these responses into meaningful information. Using qualitative data analysis software is rapidly becoming feasible. Using software applications that support content analysis and data linking and those that offer advanced linguistic analysis. The content-analysis applications allow an analyst to assess the number of times a participant uses a particular word or phrase in written material or transcribed remarks.
By counting the frequency of words and noting the association of certain words, one can categorize themes and concepts. By thus “quantifying” the qualitative communication, an analyst can associate the resulting information with other quantitative data. A more sophisticated analysis is possible with linguistic analysis, which examines the semantics, syntax, and context of participant’s verbal communications. Linguistic analysis applications help the analyst identify the key ideas in a text, gain an indication of the relative importance of each idea, and then develop a prediction of a participants attitude and or behavior based on the context of the remarks” (Gale, 2005).
The author would also consider scoring qualitative open ended question with coded values. For example expression strong emotional response against an idea could be valued as a 3, while a moderate response could be scored as 2, and a neutral or vague response a 1. Another idea that could be utilized is coding using a Likert type scale based on how overtly strong an idea or opinion is expressed. This method could induce bias if the person doing the interview or scoring the questionnaire has a personal preference in this idea. The research question should not be made known to the person doing the scoring in order to reduce the possibility of bias. The author believes that bias could also be minimized if several individuals do the scoring and the score then be averaged.
The topic chosen by this author lends itself well to open ended questions as it solicits responses in terms of attitudes, opinions, and beliefs about the economy. The following open ended questions are an example of questions that might be used in an interview about this topic.
•How do you feel about the current state of the economy?•What do you think caused the recent economic downfall?•How has the economy affected your budget?The author would use scoring qualitative open ended question with coded values. For example, strong response could be valued as a 3, while a moderate response could be scored as 2 and a neutral or vague response a 1. The author would then do an analysis of the content based on the categorizations of opinions and attitudes. In a full scale research project, the author would use a mixture of quantitative and qualitative question in order to establish opinions, thoughts, or beliefs on the subject. This would aid in the analysis of attitudes on the topic. The validity of this type of research is only in the accurate analysis of the opinions and attitudes of the participants.
Culp, K. & Pilat, M. (1999). Converting feedback into quantifiable categories [Electronicversion]. Journal of Extension 37(4).
Dilts, R., Grinder, J., Bandler, R., Delozier, J. (1980). The Study of the Structure of SubjectiveExperience, Neuro-Linguistic Programming: Vol I. CA: Meta PublicationsGale, T (2005). Let me count the words: quantifying open ended interactions with customers,Cornell Hotel & Restaurant Administration Quarterly Pub 01-Aug-2005, RetrievedMay 3, 2009 fromwww://goliath.ecnext.com/coms2/gi_0198-245360/Let-me-count-the-words.htmSmith, R. A. & Davis, S. F. (2007), The Psychologist as Detective. New Jersey: PearsonEducation, Inc.
Graziano, Anthony M. & Raulin, Michael L. (2000) Research Methods: A Process ofInquiry. Massachusetts: Allyn & Bacon.