• Users Online: 156
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
RESEARCH METHODOLOGY
Year : 2021  |  Volume : 33  |  Issue : 2  |  Page : 139-141

Design and development of objective questionnaires


Department of Research, AMMA Healthcare Research, Kochi, Kerala, India

Date of Submission17-Dec-2020
Date of Decision18-Dec-2020
Date of Acceptance19-Dec-2020
Date of Web Publication21-Aug-2021

Correspondence Address:
Dr. Praveen K Nirmalan
AMMA Healthcare Research, Gurukul, Kochi, Kerala
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/kjo.kjo_204_20

Rights and Permissions
  Abstract 


Measurements are a fundamental concept in healthcare. Questionnaires are integral to healthcare research and provide healthcare practitioners with objective tools for measurement of several aspects of health care. Questionnaires are developed based on certain contexts and validated for further use in larger populations. The primary goal of questionnaires is to provide valid measures of underlying constructs. The use of inappropriate questionnaires or poorly designed questionnaires affects the quality of data collected and the validity of the study. In this manuscript, we describe issues related to the design and development of questionnaires. These include conceptualization of target constructs and development of item pools, the basic principles of writing the items, formats and derivative versions of questionnaires, and translations. In the second part of this series, we will describe the concepts and process for validation of questionnaires.

Keywords: Construct, measurement, questionnaire development, scale development, test


How to cite this article:
Nirmalan PK. Design and development of objective questionnaires. Kerala J Ophthalmol 2021;33:139-41

How to cite this URL:
Nirmalan PK. Design and development of objective questionnaires. Kerala J Ophthalmol [serial online] 2021 [cited 2021 Nov 30];33:139-41. Available from: http://www.kjophthal.com/text.asp?2021/33/2/139/324222



Questionnaires are often used to collect information on several aspects of healthcare including knowledge, behaviors, attitudes, quality of life, and clinical aspects. Poorly designed and poorly developed questionnaires lead to the collection of poor-quality data that inevitably lead to misleading or wrong conclusions. Information from questionnaires often translates to actions that may impact upon clinical course or practices. Naturally, tests of hypothesized relations between theory-based constructs and their observable manifestations are essential during the development of the questionnaire to establish its validity.[1.2] Validity relates to the interpretation of questionnaires and describes the support provided by the evidence and theory underpinning the development of a questionnaire to interpret test scores and use the questionnaire.[1],[2] Establishing the validity of a questionnaire becomes important as the quality of these decisions are driven partly by the quality of information derived from the questionnaires.

Questionnaires validated for use in several populations or globally are available for several health care domains. However, as medical practice and sickness profiles evolve rapidly, it may not be possible to have a previously validated questionnaire for every aspect of healthcare. Questionnaires must be developed from scratch to obtain valid information about a new domain. The selection of several questions from other questionnaires or grouping of several questions does not create a questionnaire. The development of questionnaires involves a systematic process that can establish good construct validity and provide accurate information.


  Developing Questionnaires Top


An initial step in the development of a questionnaire is to form a clear, specific idea of the conceptual model for the questionnaire. The writing of a precise statement with reasonable detail of the target construct is the first step. A new construct should either provide significant improvements over an existing questionnaire or complete a gap in the existing questionnaire. The next step is a detailed literature review that focuses on two elements- (a) are there already existing, validated questionnaires for similar constructs, and (b) to acquire enough knowledge that helps to understand the theoretical context of the questionnaire and to differentiate it from other similar constructs. The literature review focuses on the existing construct of interest, related constructs, and constructs that correlate either positively or negatively-with the construct of interest. For example, a construct of satisfaction may relate or correlate with depression, anxiety, stress, guilt, shame, fear, hope, etc. The literature review helps to specify the construct of interest and identify what the construct is, and equally important, what it is not.

The use of standardized, validated questionnaires that can compare results across multiple studies and populations is common in healthcare. The use of questionnaires across populations presume that the questionnaires must elicit comparable, valid results in all sampling frames and retain the context in which the questionnaire was developed.[3],[4],[5],[6],[7],[8] However, the validity of questionnaires must be rechecked as contexts and responses may differ between and within populations. A reliable questionnaire yields consistent results across repeated samples and different studies over time.[2] A standardized questionnaire is a written document administered such that the same precise questions are asked to all participants, in the same way, with responses recorded uniformly.[2] Standardizing increases the reliability of the questionnaire. The literature review helps to understand the contextual basis for the development of the original questionnaire, its validity, reliability, and standardization in multiple settings, and to determine if the questions still retain significance in the context of the population to which the questionnaire is proposed to be administered.


  Item Pools Top


The writing of the items in the questionnaire is the next important step. Item writing specifically aims to systematically include content that is possibly relevant and can increase the overall content validity of the questionnaire. A key element to item writing is a good understanding of the theoretical constructs and the major content areas of the questionnaire that is under development. We recommend starting with a larger pool of items across the breadth of the construct as it is easier to delete items than add them later. A larger pool of items allows for an adequate sample of questions pertinent to each content area. The creation of homogeneous subscales, also known as item composites or factored item dimensions, can be used to assess each content area.[9],[10] An empirical evaluation of questions added is essential as the questionnaire is developed.[9],[11],[12] The development of a good questionnaire is an iterative process with repeated assessments of the conceptual basis and psychometric properties of each question in the construct. The repeated analysis during development may reveal that subscales are necessary, that certain content areas may need more questions, that questions must be rewritten, or that questions must be deleted or added for better content validity.

The specific wording of the questions is important. The responses to questionnaires depend on the ability of the respondent as well and physical, cognitive, mental, social, and linguistic elements must be considered.[13] Use simple, clear, and precise language at a reading level that can be understood by many, avoiding slang and colloquialisms, and preferably focused on a single characteristic. It is preferable to avoid words such as commonly, regularly, and frequently.

The choice of format to present the questions is a factor that can influence responses. Two common response formats used in healthcare are the dichotomous (example: yes/no) and the Likert type of rating scales with three or more response options. Other response formats that are used include checklists (these are prone to response bias[14]) and the visual analog or sliding scales that are almost such as numerical rating scales.[15] The format of the question responses can constrain the responses that are possible, and hence it is necessary to consider the response formats very carefully.[9]

Derivative versions of questionnaires, including short-form questionnaires, translations, and age group adaptations are sometimes necessary. Any derivative version must be revalidated before administration. The derivation of short forms does not imply simply the removal of certain items across all or some content areas. Shortening of questionnaires must maintain test information even while reducing the number of questions in a scale. The use of item response theory analytic methods is useful to shorten forms retaining the validity of the questionnaire.[16],[17],[18],[19] Translations of validated questionnaires are often used. It is important to revalidate the translated versions before administration. Translations are usually done through a two-way process with translation from English to the regional language as the first step and a back translation from the regional to English as the second step. The ability of the questionnaire to retain the original contextual meaning is assessed and the construct validity of the questionnaire is reassessed. However, the effects of acculturation and the impact of language on cultural frames must be considered carefully.[20] It is preferable not to use translated questionnaires if the construct validity of the translated version is not provided. This is an important consideration as Universities and Boards in India require healthcare students in India to use translated versions of validated questionnaires (if a questionnaire is used) as part of the mandatory student dissertation experience.

Another area of concern is the use of questionnaires across age groups. The questionnaires must be adapted and revalidated for use across ae groups. Thus, a questionnaire developed for use in an adult population must not be used for an adolescent or pediatric population without validation. These adaptations can be complex as parents may be respondents when questionnaires are administered to small children. There can be multi-sourced assessments (parents, teachers, nurses, etc) and questionnaires may be supplemented with behavioral observations.[21],[22],[23] The influences of developmental changes on constructs and response to constructs including a possible cognitive decline in the aged must be considered.[23]

Careful consideration must be given to the layout of the questions. The questionnaire must be in an adequately sized font, easy to read format, and with adequate spacing. Considerations at the development stage must include data analysis plans including the time, resources, and expertise for analysis. For instance, if you are designing a questionnaire with open-ended questions, you must be aware of or have support for the analysis of open-ended questions.

In conclusion, the use of questionnaires in healthcare research is routine practice. The development of a validated questionnaire is a systematic process that involves several steps and repeated analysis of the psychometric properties of the questionnaire. It is important to understand and provide due respect to the process of development of a questionnaire before it is used or adapted or translated for use. In the next part of this series, we will look at the process of validating questionnaires.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (AERA, APA, & NCME). Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.  Back to cited text no. 1
    
2.
Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychol Assess 2019;31:1412-27.  Back to cited text no. 2
    
3.
Howie JG, Heaney DJ, Maxwell M, Walker JJ. A comparison of a patient enablement instrument (PEI) against two established satisfaction scales as an outcome measure of primary care consultations. Fam Pract 1998;15:165-71.  Back to cited text no. 3
    
4.
Van Hook MP, Berkman B, Dunkle R. Assessment tools for general health care settings: PRIME-MD, OARS, and SF-36. primary care evaluation of mental health disorders. older americans resources and services questionnaire; short form-36. Health Soc Work 1996;21:230-4.  Back to cited text no. 4
    
5.
Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, et al. Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual Life Res 2002;11:193-205.  Back to cited text no. 5
    
6.
Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: Bibliographic study of patient assessed health outcome measures. BMJ 2002;324:1417.  Back to cited text no. 6
    
7.
Dijkers M. Measuring quality of life: Methodological issues. Am J Phys Med Rehabil 1999;78:286-300.  Back to cited text no. 7
    
8.
Gilbody SM, House AO, Sheldon TA. Routinely administered questionnaires for depression and anxiety: Systematic review. BMJ 2001;322:406-9.  Back to cited text no. 8
    
9.
Comrey AL. Factor-analytic methods of scale development in personality and clinical psychology. J Consult Clin Psychol 1988;56:754-61.  Back to cited text no. 9
    
10.
Hogan RT. A socioanalytic theory of personality In: Page M, editor. Nebraska: Symposium on Motivation. Lincoln: University of Nebraska Press; 1982. p. 55-89.  Back to cited text no. 10
    
11.
Haynes SN, Richard DC, Kubany ES. Content validity in psychological assessment: A functional approach to concepts and methods. Psychol Assess 1995;7:238-47.  Back to cited text no. 11
    
12.
Loevinger J. Objective tests as instruments of psychological theory. Psychol Rep 1957;3:635-94.  Back to cited text no. 12
    
13.
Boynton PM, Greenhalgh T. Selecting, designing, and developing your questionnaire. BMJ 2004;328:1312-5.  Back to cited text no. 13
    
14.
Green DP, Goldman SL, Salovey P. Measurement error masks bipolarity in affect ratings. J Pers Soc Psychol 1993;64:1029-41.  Back to cited text no. 14
    
15.
Simms LJ, Zelazny K, Williams TF, Bernstein L. Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychol Assess 2019;31:557-66.  Back to cited text no. 15
    
16.
Simms LJ, Watson D. The construct validation approach to personality scale construction In: Robins RW, Fraley RC, Krueger RF, editors. Handbook of Research Methods in Personality Psychology. New York: Guilford Press; 2020. p. 240-58.  Back to cited text no. 16
    
17.
Smith GT, McCarthy DM, Anderson KG. On the sins of short-form development. Psychol Assess 2000;12:102-11.  Back to cited text no. 17
    
18.
Reise SP, Meijer RR, Ainsworth AT, Morales LS, Hays RD. Application of group-level item response models in the evaluation of consumer reports about health plan quality. Multivariate Behav Res 2006;41:85-102.  Back to cited text no. 18
    
19.
Carmona-Perera M, Caracuel A, Pérez-García M, Verdejo-García A. Brief moral decision-making questionnaire: A rasch-derived short form of the greene dilemmas. Psychol Assess 2015;27:424-32.  Back to cited text no. 19
    
20.
Schwartz SJ, Benet-Martínez V, Knight GP, Unger JB, Zamboanga BL, Des Rosiers SE, et al. Effects of language of assessment on the measurement of acculturation: Measurement equivalence and cultural frame switching. Psychol Assess 2014;26:100-14.  Back to cited text no. 20
    
21.
De Los Reyes A, Augenstein TM, Wang M, Thomas SA, Drabick DA, Burgers DE, et al. The validity of the multi-informant approach to assessing child and adolescent mental health. Psychol Bull 2015;141:858-900.  Back to cited text no. 21
    
22.
Linde JA, Stringer DM, Simms LJ, Clark LA. The schedule for nonadaptive and adaptive personality youth version (SNAP-Y): Psychometric properties and initial validation. Assessment 2013;20:387-404.  Back to cited text no. 22
    
23.
Putnam SP, Rothbart MK, Gartstein MA. Homotypic and heterotypic continuity of fine-grained temperament during infancy, toddlerhood, and early childhood. Infant Child Dev 2008;17:387-405.  Back to cited text no. 23
    




 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Developing Quest...
Item Pools
References

 Article Access Statistics
    Viewed364    
    Printed6    
    Emailed0    
    PDF Downloaded37    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]