|Year : 2021 | Volume
| Issue : 2 | Page : 139-141
Design and development of objective questionnaires
Praveen K Nirmalan
Department of Research, AMMA Healthcare Research, Kochi, Kerala, India
|Date of Submission||17-Dec-2020|
|Date of Decision||18-Dec-2020|
|Date of Acceptance||19-Dec-2020|
|Date of Web Publication||21-Aug-2021|
Dr. Praveen K Nirmalan
AMMA Healthcare Research, Gurukul, Kochi, Kerala
Source of Support: None, Conflict of Interest: None
Measurements are a fundamental concept in healthcare. Questionnaires are integral to healthcare research and provide healthcare practitioners with objective tools for measurement of several aspects of health care. Questionnaires are developed based on certain contexts and validated for further use in larger populations. The primary goal of questionnaires is to provide valid measures of underlying constructs. The use of inappropriate questionnaires or poorly designed questionnaires affects the quality of data collected and the validity of the study. In this manuscript, we describe issues related to the design and development of questionnaires. These include conceptualization of target constructs and development of item pools, the basic principles of writing the items, formats and derivative versions of questionnaires, and translations. In the second part of this series, we will describe the concepts and process for validation of questionnaires.
Keywords: Construct, measurement, questionnaire development, scale development, test
|How to cite this article:|
Nirmalan PK. Design and development of objective questionnaires. Kerala J Ophthalmol 2021;33:139-41
Questionnaires are often used to collect information on several aspects of healthcare including knowledge, behaviors, attitudes, quality of life, and clinical aspects. Poorly designed and poorly developed questionnaires lead to the collection of poor-quality data that inevitably lead to misleading or wrong conclusions. Information from questionnaires often translates to actions that may impact upon clinical course or practices. Naturally, tests of hypothesized relations between theory-based constructs and their observable manifestations are essential during the development of the questionnaire to establish its validity.[1.2] Validity relates to the interpretation of questionnaires and describes the support provided by the evidence and theory underpinning the development of a questionnaire to interpret test scores and use the questionnaire., Establishing the validity of a questionnaire becomes important as the quality of these decisions are driven partly by the quality of information derived from the questionnaires.
Questionnaires validated for use in several populations or globally are available for several health care domains. However, as medical practice and sickness profiles evolve rapidly, it may not be possible to have a previously validated questionnaire for every aspect of healthcare. Questionnaires must be developed from scratch to obtain valid information about a new domain. The selection of several questions from other questionnaires or grouping of several questions does not create a questionnaire. The development of questionnaires involves a systematic process that can establish good construct validity and provide accurate information.
| Developing Questionnaires|| |
An initial step in the development of a questionnaire is to form a clear, specific idea of the conceptual model for the questionnaire. The writing of a precise statement with reasonable detail of the target construct is the first step. A new construct should either provide significant improvements over an existing questionnaire or complete a gap in the existing questionnaire. The next step is a detailed literature review that focuses on two elements- (a) are there already existing, validated questionnaires for similar constructs, and (b) to acquire enough knowledge that helps to understand the theoretical context of the questionnaire and to differentiate it from other similar constructs. The literature review focuses on the existing construct of interest, related constructs, and constructs that correlate either positively or negatively-with the construct of interest. For example, a construct of satisfaction may relate or correlate with depression, anxiety, stress, guilt, shame, fear, hope, etc. The literature review helps to specify the construct of interest and identify what the construct is, and equally important, what it is not.
The use of standardized, validated questionnaires that can compare results across multiple studies and populations is common in healthcare. The use of questionnaires across populations presume that the questionnaires must elicit comparable, valid results in all sampling frames and retain the context in which the questionnaire was developed.,,,,, However, the validity of questionnaires must be rechecked as contexts and responses may differ between and within populations. A reliable questionnaire yields consistent results across repeated samples and different studies over time. A standardized questionnaire is a written document administered such that the same precise questions are asked to all participants, in the same way, with responses recorded uniformly. Standardizing increases the reliability of the questionnaire. The literature review helps to understand the contextual basis for the development of the original questionnaire, its validity, reliability, and standardization in multiple settings, and to determine if the questions still retain significance in the context of the population to which the questionnaire is proposed to be administered.
| Item Pools|| |
The writing of the items in the questionnaire is the next important step. Item writing specifically aims to systematically include content that is possibly relevant and can increase the overall content validity of the questionnaire. A key element to item writing is a good understanding of the theoretical constructs and the major content areas of the questionnaire that is under development. We recommend starting with a larger pool of items across the breadth of the construct as it is easier to delete items than add them later. A larger pool of items allows for an adequate sample of questions pertinent to each content area. The creation of homogeneous subscales, also known as item composites or factored item dimensions, can be used to assess each content area., An empirical evaluation of questions added is essential as the questionnaire is developed.,, The development of a good questionnaire is an iterative process with repeated assessments of the conceptual basis and psychometric properties of each question in the construct. The repeated analysis during development may reveal that subscales are necessary, that certain content areas may need more questions, that questions must be rewritten, or that questions must be deleted or added for better content validity.
The specific wording of the questions is important. The responses to questionnaires depend on the ability of the respondent as well and physical, cognitive, mental, social, and linguistic elements must be considered. Use simple, clear, and precise language at a reading level that can be understood by many, avoiding slang and colloquialisms, and preferably focused on a single characteristic. It is preferable to avoid words such as commonly, regularly, and frequently.
The choice of format to present the questions is a factor that can influence responses. Two common response formats used in healthcare are the dichotomous (example: yes/no) and the Likert type of rating scales with three or more response options. Other response formats that are used include checklists (these are prone to response bias) and the visual analog or sliding scales that are almost such as numerical rating scales. The format of the question responses can constrain the responses that are possible, and hence it is necessary to consider the response formats very carefully.
Derivative versions of questionnaires, including short-form questionnaires, translations, and age group adaptations are sometimes necessary. Any derivative version must be revalidated before administration. The derivation of short forms does not imply simply the removal of certain items across all or some content areas. Shortening of questionnaires must maintain test information even while reducing the number of questions in a scale. The use of item response theory analytic methods is useful to shorten forms retaining the validity of the questionnaire.,,, Translations of validated questionnaires are often used. It is important to revalidate the translated versions before administration. Translations are usually done through a two-way process with translation from English to the regional language as the first step and a back translation from the regional to English as the second step. The ability of the questionnaire to retain the original contextual meaning is assessed and the construct validity of the questionnaire is reassessed. However, the effects of acculturation and the impact of language on cultural frames must be considered carefully. It is preferable not to use translated questionnaires if the construct validity of the translated version is not provided. This is an important consideration as Universities and Boards in India require healthcare students in India to use translated versions of validated questionnaires (if a questionnaire is used) as part of the mandatory student dissertation experience.
Another area of concern is the use of questionnaires across age groups. The questionnaires must be adapted and revalidated for use across ae groups. Thus, a questionnaire developed for use in an adult population must not be used for an adolescent or pediatric population without validation. These adaptations can be complex as parents may be respondents when questionnaires are administered to small children. There can be multi-sourced assessments (parents, teachers, nurses, etc) and questionnaires may be supplemented with behavioral observations.,, The influences of developmental changes on constructs and response to constructs including a possible cognitive decline in the aged must be considered.
Careful consideration must be given to the layout of the questions. The questionnaire must be in an adequately sized font, easy to read format, and with adequate spacing. Considerations at the development stage must include data analysis plans including the time, resources, and expertise for analysis. For instance, if you are designing a questionnaire with open-ended questions, you must be aware of or have support for the analysis of open-ended questions.
In conclusion, the use of questionnaires in healthcare research is routine practice. The development of a validated questionnaire is a systematic process that involves several steps and repeated analysis of the psychometric properties of the questionnaire. It is important to understand and provide due respect to the process of development of a questionnaire before it is used or adapted or translated for use. In the next part of this series, we will look at the process of validating questionnaires.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (AERA, APA, & NCME). Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychol Assess 2019;31:1412-27.
Howie JG, Heaney DJ, Maxwell M, Walker JJ. A comparison of a patient enablement instrument (PEI) against two established satisfaction scales as an outcome measure of primary care consultations. Fam Pract 1998;15:165-71.
Van Hook MP, Berkman B, Dunkle R. Assessment tools for general health care settings: PRIME-MD, OARS, and SF-36. primary care evaluation of mental health disorders. older americans resources and services questionnaire; short form-36. Health Soc Work 1996;21:230-4.
Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al
. Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual Life Res 2002;11:193-205.
Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: Bibliographic study of patient assessed health outcome measures. BMJ 2002;324:1417.
Dijkers M. Measuring quality of life: Methodological issues. Am J Phys Med Rehabil 1999;78:286-300.
Gilbody SM, House AO, Sheldon TA. Routinely administered questionnaires for depression and anxiety: Systematic review. BMJ 2001;322:406-9.
Comrey AL. Factor-analytic methods of scale development in personality and clinical psychology. J Consult Clin Psychol 1988;56:754-61.
Hogan RT. A socioanalytic theory of personality In: Page M, editor. Nebraska: Symposium on Motivation. Lincoln: University of Nebraska Press; 1982. p. 55-89.
Haynes SN, Richard DC, Kubany ES. Content validity in psychological assessment: A functional approach to concepts and methods. Psychol Assess 1995;7:238-47.
Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep 1957;3:635-94.
Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ 2004;328:1312-5.
Green DP, Goldman SL, Salovey P. Measurement error masks bipolarity in affect ratings. J Pers Soc Psychol 1993;64:1029-41.
Simms LJ, Zelazny K, Williams TF, Bernstein L. Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychol Assess 2019;31:557-66.
Simms LJ, Watson D. The construct validation approach to personality scale construction In: Robins RW, Fraley RC, Krueger RF, editors. Handbook of Research Methods in Personality Psychology. New York: Guilford Press; 2020. p. 240-58.
Smith GT, McCarthy DM, Anderson KG. On the sins of short-form development. Psychol Assess 2000;12:102-11.
Reise SP, Meijer RR, Ainsworth AT, Morales LS, Hays RD. Application of group-level item response models in the evaluation of consumer reports about health plan quality. Multivariate Behav Res 2006;41:85-102.
Carmona-Perera M, Caracuel A, Pérez-García M, Verdejo-García A. Brief moral decision-making questionnaire: A rasch-derived short form of the greene dilemmas. Psychol Assess 2015;27:424-32.
Schwartz SJ, Benet-Martínez V, Knight GP, Unger JB, Zamboanga BL, Des Rosiers SE, et al
. Effects of language of assessment on the measurement of acculturation: Measurement equivalence and cultural frame switching. Psychol Assess 2014;26:100-14.
De Los Reyes A, Augenstein TM, Wang M, Thomas SA, Drabick DA, Burgers DE, et al
. The validity of the multi-informant approach to assessing child and adolescent mental health. Psychol Bull 2015;141:858-900.
Linde JA, Stringer DM, Simms LJ, Clark LA. The schedule for nonadaptive and adaptive personality youth version (SNAP-Y): Psychometric properties and initial validation. Assessment 2013;20:387-404.
Putnam SP, Rothbart MK, Gartstein MA. Homotypic and heterotypic continuity of fine-grained temperament during infancy, toddlerhood, and early childhood. Infant Child Dev 2008;17:387-405.