Original Research ARTICLE
The reliability of an instrument (QoDoS) for assessing quality of decision making during the lifecycle of medicines
- 1Centre for Innovation in Regulatory Science (CIRS), United Kingdom
- 2School of Life and Medical Sciences, University of Hertfordshire, United Kingdom
- 3Professor of Pharmacoepidemiology, University of Hertfordshire, United Kingdom
Introduction: The Quality of Decision-Making Orientation Scheme (QoDoS) was developed to provide organisations involved in submission, approval and reimbursement of new medicines with a tool to improve the quality of their decision-making processes and is considered the most promising tool for such purpose. This study aimed to further establish the measurement properties of the QoDoS by evaluating its reliability (internal consistency and test-re-test reliability) and relevance in the target population.
Methods: The study participants consisted of 55 individuals recruited from pharmaceutical companies, regulatory and HTA agencies. It was designed as a longitudinal study with participants assessed on two different occasions, at baseline (test 1) and then seven days later (test 2). Internal consistency reliability was assessed with Cronbach’s alpha and the test-retest reliability was evaluated using the intraclass correlation coefficients (ICC) based on absolute agreement, 2 way mixed-effects model for the four QoDoS domains. The relevance of the QoDoS was evaluated by applying cognitive debriefing using five short feedback questions following test 1.
Results: Test 1 was completed by 44 study participants (80% response rate) and test 2 was completed by 32 of the 44 individuals, resulting in a 73% response rate. Cronbach’s alpha coefficient was greater than 0.7 across all the domains for test 1 and test 2, ranging from 0.71 to 0.79, indicating good consistency of responses. For the overall score across all 47 items, the Cronbach’s alpha coefficient was 0.81 for test 1 and 0.86 for test 2, which is rated as very good. The four QoDoS domains showed moderate to strong reproducibility (ICC range: 0.63-0.86) with an overall ICC of 0.87. The outcome of the cognitive debriefing from the 43 respondents (98% response rate) confirmed the relevance (95% agreement), language clarity (95%) and completeness of items (86%); the clarity of the scaling (91%) as well as spontaneity of responses (95%).
Conclusion: These results provide strong support for the relevance and reliability of the QoDoS, which are key properties for future longitudinal and cross-sectional applications of the instrument when evaluating quality of decision making by those involved in the lifecycle of medicines.
Keywords: Measurement instrument, QoDoS, Quality decision making, Reliability, relevance, Validation, Research and development, Regulatory, Health technolgy assessment
Received: 14 Sep 2018;
Accepted: 08 Jan 2019.
Edited by:Tahir M. Khan, Institute of Pharmaceutical Sciences, University of Veterinary and Animal Sciences, Pakistan
Reviewed by:Imran Masood, Islamia University, Pakistan
Allah Bukhsh, Monash University, Australia
Juman A. Dujaili, Monash University Malaysia, Malaysia
Copyright: © 2019 Bujar, McAuslane, Walker and Salek. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Prof. Sam Salek, University of Hertfordshire, Professor of Pharmacoepidemiology, Hatfield, United Kingdom, firstname.lastname@example.org