Design, Conduct, and Use of Patient Preference Studies in the Medical Product Life Cycle: A Multi-Method Study

Objectives: To investigate stakeholder perspectives on how patient preference studies (PPS) should be designed and conducted to allow for inclusion of patient preferences in decision-making along the medical product life cycle (MPLC), and how patient preferences can be used in such decision-making. Methods: Two literature reviews and semi-structured interviews (n = 143) with healthcare stakeholders in Europe and the US were conducted; results of these informed the design of focus group guides. Eight focus groups were conducted with European patients, industry representatives and regulators, and with US regulators and European/Canadian health technology assessment (HTA) representatives. Focus groups were analyzed thematically using NVivo. Results: Stakeholder perspectives on how PPS should be designed and conducted were as follows: 1) study design should be informed by the research questions and patient population; 2) preferred treatment attributes and levels, as well as trade-offs among attributes and levels should be investigated; 3) the patient sample and method should match the MPLC phase; 4) different stakeholders should collaborate; and 5) results from PPS should be shared with relevant stakeholders. The value of patient preferences in decision-making was found to increase with the level of patient preference sensitivity of decisions on medical products. Stakeholders mentioned that patient preferences are hardly used in current decision-making. Potential applications for patient preferences across industry, regulatory and HTA processes were identified. Four applications seemed most promising for systematic integration of patient preferences: 1) benefit-risk assessment by industry and regulators at the marketing-authorization phase; 2) assessment of major contribution to patient care by European regulators; 3) cost-effectiveness analysis; and 4) multi criteria decision analysis in HTA. Conclusions: The value of patient preferences for decision-making depends on the level of collaboration across stakeholders; the match between the research question, MPLC phase, sample, and preference method used in PPS; and the sensitivity of the decision regarding a medical product to patient preferences. Promising applications for patient preferences should be further explored with stakeholders to optimize their inclusion in decision-making.


Design, Conduct and Use of Patient Preference Studies in the Medical Product Life Cycle: a Multi-method Study -Supplementary Material
o Participants that were already interviewed in semi-structured interviews o Multiple patients from one family (but if in the end not enough patients are included, siblings can still be welcomed) • Book a venue/ meeting room that is neutral (hotel, patient organization, or university, not company or government), easily accessible to patients and that allows for a circle seating around a table • If possible, provide the information sheet to all participants in advance via email a week in advance (otherwise only before focus group on paper) • Provide the participants with instructions to get to the venue and room • Provide your/assistants' phone number(s) to the participants 2. Day of the focus group • Have enough information sheets, consent forms and surveys 1 with you • Seating arrangements: circle seating around table • Provide writing material and paper • Set up the PowerPoint presentation (bring laptop) • Provide name cards (mentioning only the first name of participants), to put on the table • Provide water/coffee/tea + something to eat • Take two audio recorders (+ extra batteries) with you and place these on the table at opposite sides • Dress code for moderator and assistant(s): Business casual, pants and blouse is OK, no suits, no suit jackets/blazers, not too formal 3. Moderator and assistant team • A meeting between moderator and assistant should be planned before the focus group to go over the guide and discuss the moderator and assistant's tasks 3.1 Attitude of moderator: • Exercise mild unobtrusive control (moderate the discussion but do not interrupt too often) • Adequate knowledge of topic • Appears like the participants • Use purposeful small talk • Alert and free from distractions (put sound of phone off and do not have your phone on you) • Have the discipline of listening and apply active listening • Familiar with questioning route (know this protocol, the timing of slides + animations in slides and all questions very well) • Take into account the different types of participants and try to balance the conversation while addressing the obligatory topics: dominant talkers, shy participants, etc.
3.2 Tasks of the assistant: • Handles logistics (location of refreshments, bathrooms, emergency exits) • Collects consent forms and surveys • Takes careful notes (bring laptop) o On statements made o Nonverbal reactions (movements, attitudes, emotions) • Controls for equal participation by all participants and informs the moderator if some participants are not getting the chance to participate • Monitors audio recording equipment • Time management via discrete signs to the moderator • Gives a general summary of what has been said at the end of the focus group

II. Focus group agenda
Everything below in Italic is to be said to the participants, everything in black is guidance and if applicable can be told to the participants in your own words. In bold the timing of actions is given; however, this is an indication of time, it is more important to finish the two first topics than to rush through the focus group and complete all three topics. In bold also, the slides to display is given. 4. 00:00 Slide 1. Welcome the participants while they arrive • Create warm and friendly environment o Provide coffee and tea o Interact with participants, and stimulate interaction between participants • Make seating arrangements for participants according to their needs • Provide the information sheet and let them fill in the consent form and survey 5. 00:06 Slide 2. When all participants arrived and completed the forms, start the focus group with a general introduction: • Welcome, my name is (your first name) and I will be your moderator today. In addition, I brought (first name of assistants) to help me with the focus group • My role as moderator will be to guide the discussion • The discussion that we will have today is on your involvement, or actually your preferences and how these preferences can be used in making drugs or medical devices, and in making them available to you • Today's discussion is part of a large European project with similar objectives • We want to have this discussion with you as patients (and caregivers), since we think it is important to know how you feel about using your preferences • The opinions collected today will be used to write reports and publications to inform companies, health authorities, researchers and other parties on how to use patient preferences • The focus group will take 1 hour to 1.5 hours • There will be a break in the middle of the discussion of 5-10 minutes 6. 00:08. Explain the rules: •

II. Focus group agenda
Everything below in Italic is to be said to the participants, everything in black is guidance and if applicable can be told to the participants in your own words. In bold the timing of actions is given; however, this is an indication of time, it is more important to finish the two first topics than to rush through the focus group and complete all three topics. 4. 00:00 Welcome the participants while they call in on the teleconference • Try to get every participant's name directly and welcome them • If participants try to participate without having signed the consent form, ask them if they can sign the consent form on spot and send a scan per e-mail within a couple of minutes. If this is not possible, ask them to leave the teleconference • Wait until all participants have joined for 5 minutes from start of teleconference 5. 00:06 When all participants arrived and completed the forms, start the focus group with a general introduction: • Welcome, my name is (your first name), working as PhD researcher at (institute) and I will be your moderator today. In addition, also (first name of assistants) called in to help me with the focus group • My role as moderator will be to guide the discussion • The discussion that we will have today is on the assessment and use of information on patient preference studies when making decisions during the development of drugs or medical devices, and when making them available to patients • Today's discussion is part of a study within a large European project called IMI PREFER.

PREFER looks at how and when it is best to perform and include patient-preference in decision making during the drug life cycle. The end-result will be recommendations to support development of guidelines for industry, Regulatory Authorities and HTA bodies.
• We want to have this discussion with you as (name stakeholder), since we think it is important to know how you feel about using these preferences in the decision making at your institution(s) • The opinions collected today will be used to write reports and publications to inform companies, health authorities, researchers and other parties on how to use patient preferences • The focus group will take about 1.5 hours 6. 00:08 Explain the rules: • There are no right or wrong answers, only differing points of view

II.I Steps and rationale behind the choice of the analysis method
The steps we undertook to select our analysis method included the following: forew. by Michael Quinn Patton à Only explains the "long-table approach" (using scissors), not manageable 2. Next, we consulted a qualitative research expert from the University of Leuven (external to the PREFER project), who suggested 2 approaches based upon our research objectives: o Multiple case design (Miles and Huberman, 1994) à This approach focusses on performing within and across case analyses to reveal similarities and differences among the different FGDs, which why we thought it was appealing to our study. However, we rejected this approach as it is also a very time consuming one; o Thematic analysis (Howitt, 2016) à Selected as our method.

Rationale behind the selection of thematic analysis:
1. Thematic analysis focusses on what has been said rather than how it was said; 2. Thematic analysis is accessible to novice researchers, as it is less demanding than other methods (e.g. thematic analysis does not involve the same level of sophistication in data collection and theory building as grounded theory); 3. Useful technique when: a. Data collection has finished; thematic analysis, as opposed to grounded theory, does not have the requirement that the data being collected are reviewed part-way through the analysis and new approaches to data collection are initiated; b. Data consists of detailed textual material (e.g. focus group transcripts); c. A broad-brush approach to analysis is desired (as opposed to some fine-grained approaches which characterize some qualitative methods); "How to do thematic analysis" (p.168-175, Howitt, 2016) has formed the basis of this protocol.

Chosen method: Thematic analysis based upon "Qualitative Research Methods in Psychology" -Dennis Howitt
Core analyzing team ("CAT"): Eline van Overbeeke (EvO) and Rosanne Janssens (RJ) Major steps in thematic analysis: 1. Data familiarization 2. Initial coding generation 3. Search of themes 4. Review of themes 5. Theme identification and labelling Step-wise explanation:

Data familiarization
a. Howitt states that: i. Data familiarization can take place via several methods and will differ according to the details of the study. Ways of familiarizing with the data are: (i) being involved in the data collection stage, (ii) doing the transcription, (iii) playing the recordings repeatedly or (iv) re-reading the transcripts; ii.
Start think about what is happening in the data during this stage. These early thoughts may suggest ways in which the data might be coded; iii.
Use literal transcription (since thematic analysis is about what was said rather than how it was said); iv.
Novice researchers to do all the data collection and transcription themselves; b. Therefore, we will: i. Become familiar with the data by: 1. Being present during the FGD as moderator or assistant to the moderator. Because of practical limitations, 2 FGDs were moderated by an external partner, which make the steps below critical to become familiar with those FGDs. These external partners were actively involved during the set-up the research (e.g. during the set-up of the protocol and FGD questions) as we had several discussions with them to inform them and ask about their feedback about our focus group questions and protocol. Next, because of the amount of the focus groups (n=8) and the different languages used, we chose to outsource the transcription and translation of the focus groups to a professional transcription company (TodayTranslations). 2. Thoroughly reading the transcript several times; ii.
Have the transcription done literally (at verbatim) by the transcription company; iii.
Have the transcripts checked for completeness and accuracy by moderator and/or assistants present during FGD; iv.
Convene and discuss in order to agree together about what was said in that particular part of the FGD, should there be difficulties in understanding the transcript. Should there be any remaining difficulties in understanding a certain part of the FGDs where neither one of EvO or RJ was present, we will contact the moderator and/or assistant present during that FGD to discuss and agree upon what was said.

Initial coding generation
c. Howitt states that: i. This stage does not aim at identifying the themes that the research will generate; initial codes are nothing more than labels that will describe the content of 1 or 2 lines, they are not sophisticated analyses of the data. However, ideas as to what the themes might be can already occur (as during any stage of thematic analysis); ii.
There are no "rules" describing that initial coding has to be done line-byline. Coding frequency depends on circumstances, if every line is not possible then every 2/3 lines is "all right"; iii.
Best if these codes are based on an abstraction rather than something concrete, the more conceptual (i.e. the less concrete, the more abstract) the codes, the better the final themes; iv.
Researchers can choose: 1. To work through the entirety of data or a subset of the data selected because it deals with a topic/matter of interest to the researcher; 2. Between a theory-led or data-led approach; v.
During this stage, it may be appropriate to re-name codes that are covering the same meaning so they have the same wording; vi.
After the initial coding has been done, researchers should put together all of the transcript which has received a certain code. Reviewing all coded text of a certain code can reveal that: 1. A coding label is not accurate/precise enough and needs to be renamed; 2. New codes need to be formed as some of the data in a certain code "does not match"; 3. Certain codes need to be combined to one code as the coded text below two codes is too similar. d. Therefore, we will: i. Aim for 1 initial code every 2/3 lines; ii.
Use a data-led approach as described by Howitt, in which codes are primarily guided by careful analysis of what is in the data; iii.
Independently code the entirety of the data (n=8 FGDs), since there are two core analyzers; iv.
Convene after the independent coding of the data and perform steps vi.1, 2 and 3 together in order to agree upon the final list of initial codes

Search for themes based on initial codes
e. Howitt states that: i. This stage involves turning the initial codes into themes, which requires a lot of analytic work on the part of the researchers; ii.
Searching for themes involves searching for patterns among the initial codes; as they will probably notice that some codes are more related than others; iii.
Themes are the result of grouping and categorizing codes, which does not preclude that some codes might turn out to be very important and result in this code being an actual theme; iv.
Some themes may be very obvious from the initial codes, whereas sometimes methods of sorting might help, e.g. by writing down all the initial codes on separate slips of paper and creating piles of related codes. NVivo or Word might be used in this stage. f. Therefore, we will: i. Each independently search for themes based upon the initial codes; ii.
Convene, discuss and agree upon the themes that we independently found.

Review of themes
g. Howitt states that: i. At this stage, there is a set of tentative themes which help to understand the data; ii.
In the case that these themes are not fully defined or refined at this stage, it is essential to examine these themes against the original data; iii.
Reviewing of themes involves organizing the data around the set of themes just as previously the data was organized around the codes; iv.
The possible scenarios of this stage are: 1. Modifying or abandoning the theme if there is very little in the data supporting the theme; 2. Dividing or subdividing the theme if the data in one theme actually imply two different themes or sub-themes; 3. Find a new theme if some of the data you initially believed were part of the theme does not fit. If this is the case, a check for applicability of these themes to this data as well as the entire data set is advised. h. Therefore, we will: i. Divide the total number of transcripts among EvO and RJ; ii.
Separately go back to each of the assigned transcript and organize the text that was captured by the initial codes around our identified themes; iii.
Separately critically revise whether the theme should be abandoned, modified, (sub)divided or whether a new theme should be found; iv.
Any modification to our initial found themes will trigger a discussion between EvO and RJ and should this discussion lead to a modification of the initial list of themes, a check of the applicability of this modified list of themes to the entire data set will be done.

Theme definition and labelling
i. Howitt states that: i. Although it might be easy to give a label to a theme, it might be more difficult to define exactly what a theme is; ii.
It is important to be able to conceptually distinguish one theme from another; iii.
It is likely to continue developing sub-themes at this stage; iv.
It is important to talk with other people about your analysis at this stage and allow them to question you and throw in ideas of their own. j. Therefore, we will: i. Discuss and agree upon the final list of themes and sub-themes and our explanation to it. This will form the basis of the report (see 6.); ii.
Discuss within our CAT as well as with other persons how our themes differ from each other.
6. Report writing k. Howitt states that: i. The explanation and description of the themes in the final report of thematic analysis involve the selection of appropriate illustrations taken from the material which is associated with the theme; ii.
Criteria that may be applied for this selection are: 1. How 'typical' the material is of the data which belong to a particular theme; 2. How 'fit' the material is in relation to the theme; some excerpts might illustrate particular features of the theme better than others; 3. How 'eye-catching' the excerpt is; some data might be preferred to other excerpts as it is more vivid; 4. Some might prefer using excerpts from just one of the participants to get into more detail about that particular case iii.
It is helpful to indicate in the report the basis for your excerpt selection l. Therefore, we will: i. Explain and describe in the final report the themes we identified; ii.
Use appropriate excerpts to illustrate these themes; iii.
Apply criteria ii.1, 2, 3 to select the excerpts we will use to describe the themes.

III.I Patient focus group themes
Themes Definition 1. Healthcare issues that matter to patients All topics, issues, concerns, desires, needs, preferences important to patients in relation to their or others' healthcare 1.1 Medical product related issues All topics, issues, concerns, desires, needs, preferences important to patients in relation to medical products and their characteristics 1.2 Non-medical product related issues All topics, issues, concerns, desires, needs, preferences important to patients not directly in relation to medical products and their characteristics, but still relating to healthcare 1.2.1 Patient journey related issues All topics, issues, concerns, desires, needs, preferences important to patients relating to all the sequential steps in receiving healthcare • Relationship with recruiting person: LC and RA patients explained that the level of trust and relation they have with the person that asks them to participate influences their willingness to participate. More specifically, receiving this question from their treating physician, whom they trust, motivates them to participate: "To encourage participation I believe also patients' trust in their oncologist is important, because when they phoned me I trusted them so I was willing to participate." (PA5_LC_IT). • Self-perception and disease acceptance: According to DM participants however, young DM patients are often self-conscious about their illness and do not want to attend big meetings where severely affected DM patients are present, as it confronts them with their disease: "Young people, sort of teen onwards, they are very self-conscious. They are very aware that they have got an issue and with my two, they don't like discussing it and don't want to be in an environment with other people, particularly wheelchairs and things, it is just a turn off for them" (CA1_UK_DM). Similarly, some LC participants mentioned that a lack of disease acceptance might hinder participation: "There are people who hide their disease, they receive the treatment but don't say it" (PA7_IT_LC).
• Interaction with patients: LC and DM participants mentioned that they are motivated to participate because of the possibility of sharing information, comparing experiences and just to go out and meet people: "So if I have to say why I would encourage someone else to attend, it is because you meet other patients and you realize you are not alone" (PA3_IT_LC). However, this view was not supported by some DM participants, who argued that DM patients often do not want to attend big meetings (see below: self-perception and disease acceptance).
• Financial compensation: Diverging views were expressed about whether or not financial compensations would incentivize patients to participate in preference studies; while some CVD participants argued that financial compensations are not needed, because they have a personal interest and "gain" from participating in preference studies, and because of the convenience of participating in preference studies situated in hospitals: "But this is not something you should get paid for, because you don't make any effort" (PA2_RO_CVD), one RA participant argued that financial compensations should be provided if the preference study interferes with work and daily activities: "I think that if people are paid, because everyone is working and is busy and... I think that it is something like this, that is, that one gets a benefit from it personally" (PA1_SE_RA).
• Convenience: Participants from all disease areas mentioned different aspects impacting their participation in preference studies related to the convenience of the study. According to RA and DM patients, the time investment of participation should be kept at a minimal level and hence, questionnaires should be as short as possible. One RA patient added that longer questionnaires, interfering with work life, should be financially compensated. Physical constraints (such as sleepiness and digestive problems), a long travel distance and an inconvenient time during the day were barriers to participating mentioned by DM patients: "It is a physical constraint and that is not got to do with her willingness or not to participate, it is just the physical side, this continuing fatigue" (PA1_UK_DM). Some DM, CVD and LC patients suggested organizing preference studies in a hospital setting, as this would allow them to combine their participation with hospital stays or visits: "We did not give up a profession, a meal, a study, an activity of sorts to come here, we stay with you for 7 days or 10 days, for as long as we are hospitalised" (PA2_RO_CVD). Some DM participants added that besides hospitals being "the easiest place for them to go to" (CA1_UK_DM), they are also familiar to DM patients.
• Attractiveness: DM participants spoke about how some young DM patients dislike big events where they are confronted with their illness. In order to increase the attractiveness and hence, the participation of young DM patients into the preference study, some DM participants suggested de-formalizing the preference study: "If you could de-formalise it (…) something like a barbeque or a party (…) music, they all love music" (CA2_UK_DM). • Feedback on results: DM participants argued that receiving information on the study results would make it worthwhile for them to participate: "It would be nice to know, get some global feedback and individual feedback because it comes back to this apathy that for me, you start to think, 'Oh, it is not worth it'" (PA1_UK_DM). Further, this information ideally comes in the form of a short summary: "I don't want a massive amount. A few short summaries is what I would be looking for rather than the detail" (CA2_DM_UK).