Evidence of the content validity, acceptability, and feasibility of a new Patient-Reported Impact of Dermatological Diseases measure

1. Introduction

Dermatological conditions carry a substantial physical, psychological, and social burden for patients (1, 2). The stigma of living with a visible condition (3), symptoms including pain and itch (4, 5), and financial cost (6) partially explain this burden (7). Many dermatological conditions have associated comorbidities (8), further increasing the disease burden (9).

The Global Burden of Disease (GBD) project (10, 11) is the most comprehensive worldwide epidemiological study to date, providing burden and mortality estimates for health problems at global, national, and regional levels. These estimates are exceptionally influential as they provide the evidence-base for identifying patient need, determining resource allocation and research priorities globally. We, along with others in the dermatology community [(e.g., 12)], maintain that the GBD studies systematically underestimate the burden of dermatological conditions as they are evaluated according to symptoms that affect only the skin (itch, disfigurement) and do not include the broader psychological and social aspects such as depression, anxiety, stigma, and social isolation in their measure of impact (13–16).

The Global Research on the Impact of Dermatological Diseases (GRIDD) project, the first patient-initiated and led impact research project in dermatology, is collecting global data on the impact of living with dermatological conditions. These data will support local, national, and international advocacy work to prioritize dermatological conditions more accurately in the global health debate.

To address GRIDD’s aim, a comprehensive measure of the impact of dermatological conditions on the patient’s life is required. Our systematic review (17) evaluated the quality of existing dermatology-specific (can be used across conditions) patient-reported outcome measures (PROMs) against the gold-standard Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) criteria (18). PROMS, like all measurement instruments (e.g., thermometers, sphygmomanometers), must meet predefined criteria for measurement properties—validity, reliability, and responsiveness—to have confidence that the data they produce are accurate (19–21). None of the 36 existing dermatology PROMs identified in our review, including widely used measures such as Dermatology Life Quality Index (DLQI) (22) and Skindex (23–25), met the standards to be recommended for use according to their known measurement properties and could not capture the full impact of the dermatological condition on the patient’s life according to our conceptual framework of impact (26). The single most common reason for poor quality assessment was insufficient patient involvement during PROM development. This included, for example, inadequate sample sizes and inappropriate data collection methods (27, 28). Other systematic reviews of existing quality of life PROMs in the context of psoriasis (29), eczema (30), and acne (31) have found a similar lack of adequate dermatology-specific PROMs.

We are developing the Patient-Reported Impact of Dermatological Diseases (PRIDD) measure in close collaboration with patients and according to best practices in PROM development (Supplementary material 1) (18, 19, 32–34). PRIDD is designed for use with adults (≥18 years) with a dermatological condition worldwide and for use in research and clinical practice. Congruent with best practice in cross-cultural translation of PROMs, PRIDD is initially being developed and validated in English before being translated into other languages.

Content validity, “the degree to which the content of an instrument is an adequate reflection of the construct to be measured” (35), is considered the most important measurement property (28, 36). We began the content validity phase of PRIDD development by conducting a qualitative interview study with 65 patients from 29 countries representing 29 dermatological conditions (26) and identified 263 areas of impact that cut across conditions and global regions. This work formed, to our knowledge, the first conceptual framework of the impact of dermatological conditions on patients’ lives. In the second phase of PRIDD development, 1,154 patients from 65 countries representing 90 dermatological conditions participated in a global Delphi survey to prioritize the 263 items for inclusion in PRIDD (37). While existing dermatology PROMs have included a range of domains relevant to the construct of impact (17), no single PROM has unified all relevant domains outlined in the conceptual framework of impact. This demonstrates that, as the first measure to capture all aspects of the conceptual framework as a unified construct, PRIDD advances knowledge in and makes a unique contribution to dermatology.

Following best practice, the next phase in PRIDD development is to pilot test the measure with dermatology patients and make any necessary adjustments (38, 39). The purpose of the pilot is to rigorously test three aspects of content validity: comprehensiveness (all key aspects of impact are present), comprehensibility (items are understood by respondents as intended) and relevance (all items are relevant to the impact of dermatological conditions from the patients’ perspective) (39). The acceptability (whether patients are willing to complete the instrument) and feasibility (whether patients are able to complete the instrument) can also be tested.

Cognitive interviewing is a pilot testing method that use a semi-structured topic guide to direct the interview according to Tourangeau’s four-stage model of question response (40, 41) to obtain information about how participants interpret questions and choose their answers. The COSMIN group recommend the Three-Step Test-Interview (TSTI) (42, 43) method of cognitive interviewing as this combines the “think-aloud” (44) and “probing” techniques (36, 38), thereby offsetting the weaknesses of each and providing a deeper understanding of how questions are interpreted and answered (38).

The aim of the current study was to pilot-test PRIDD by qualitatively exploring whether the measure (items, structure, response options and recall period) is comprehensive, comprehensible, relevant, feasible, and acceptable to people with dermatological conditions through cognitive interviews.

2. Materials and methods

2.1. Design

We conducted a theory-led, qualitative study using the TSTI method of cognitive interviewing to pilot test PRIDD. This study was tested against the latest COSMIN guidance on the pilot testing of PROMs (27, 33, 45) and is reported according to the Cognitive Interviewing Reporting Framework (46). Ethical approval was obtained from Cardiff University School of Healthcare Sciences Ethics Committee (SREC:637).

2.2. Participants

Participants met the inclusion criteria if they were an adult (≥18 years) with a dermatological condition from anywhere in the world and spoke English sufficiently to take part in the interview and complete PRIDD independently (without a translator). Those who required translation to complete PRIDD were excluded as construct equivalence, the assumption that items in the translation measure the same construct in the same way as in the original language (47–49), could not be determined and, therefore, confidence in the evidence of content validity would be lacking. Children and proxies, such as family members or carers, were also excluded as they are not PRIDD’s target population.

Participants were drawn from PRIDD’s target population via the International Alliance of Dermatology Patient Organizations’ (GlobalSkin) global membership network using purposive sampling to achieve maximum variation according to dermatological condition and demographic factors: age, gender, and country of residence. Participants were directed to a secure online platform which included the participant information sheet (PIS), electronic consent form, and interview booking information. Twelve patient organizations were invited to recruit to the interviews; 8 (66.7%) agreed to participate. Reasons for non-participation included lack of staff capacity, scheduling conflicts, and non-response. Recruitment ceased at the point of data saturation; when there was sufficient evidence that most problems had been detected and/or resolved (46, 50).

2.3. Materials

We tested the first draft of PRIDD (Supplementary material 2), a 27-item, English language measure of the impact of a dermatological condition on the patient’s life over the last week. The conceptual framework of impact (26) depicts a reflective measurement model (Supplementary material 3). The first draft of PRIDD has five subscales: physical impact, psychological impact, social impact, daily life and responsibilities impact, and financial impact. All items are rated on a 5-point scale with scores of 0 (“never”), 1 (“rarely”), 2 (“sometimes”), 3 (“often”), and 4 (“always”), each with an additional “not relevant” option.

A topic guide was developed detailing the interview procedures, instructions, and questions (Supplementary material 4). Two versions of PRIDD were used during the interviews: one with the original item order and one with items in reverse order. This enabled us to test item order effects (i.e., whether the order in which the items are presented affects people’s responses) and establish whether items were understood independently of each other. An online platform was created to enroll potential participants in the study using the PIS and consent form and included a demographic questionnaire.

2.4. Procedure

Twelve interviews were conducted via Zoom video conferencing software across three rounds from 2 August to 1 September 2021 with one of four researchers (RP, RH, MVL, and NTG), at a mutually convenient time. All interviewers were trained in cognitive interviewing by the study co-investigator (CB) and had backgrounds in healthcare practice and/or research.

Interviews were approximately 1 h long and followed the three steps of the TSTI method (see Supplementary material 5). They were audio-recorded using a high-quality audio recorder (OLYMPUS WS-833) and transcribed verbatim. Transcripts were checked and anonymized by RP by being allocated participant identifiers (PIDs).

2.5. Analysis

Data collection and analysis were interrelated and concurrent, with analysis beginning after the completion of the first interview. Accordingly, generated themes and edits made to PRIDD were incorporated into subsequent interviews.

Quantitative data were uploaded to SPSS version 26 and sample characteristics were summarized for clinical and demographic variables. Qualitative data were exported to NVivo 12 qualitative data software package. RP independently analyzed the data. NTS reviewed the coding and results reporting for accuracy. Analysis followed the thematic analytical model of cognitive interviewing (51) (see Supplementary material 6). This produced a summary of each item’s performance that established the comprehensiveness, comprehensibility, and relevance of the items, providing evidence of content validity and informing evidence-based improvements.

3. Results

Eighteen people completed the online consent form and demographics questionnaire. Of these, three people were excluded because they were not sufficiently proficient in English to complete PRIDD independently and three did not respond to invitations to schedule an interview. In total, 12 people (response rate = 67%) across six dermatological conditions (Table 1) and four countries participated in an interview.

TABLE 1

Table 1. Participant characteristics.

Supplementary material 7 outlines the changes made to PRIDD between the three rounds of cognitive interviews. Evidence-based adjustments resulted in a 26-item version of PRIDD (Supplementary material 8).

3.1. General feedback

Participants praised the comprehensiveness, comprehensibility, and relevance of PRIDD to their lived experiences:

I’ve completed a lot of dermatological questionnaires, but I don’t think I’ve ever seen them all integrated like this in such a questionnaire … I’m very, very happy with this. It has stirred my heart … There are things here that I wanted to discuss with my dermatologist. PID5, Patient with hidradenitis suppurativa, Ireland

They were short questions, and they were quite easily answered. PID15, Patient with alopecia, UK

PRIDD also appeared to be acceptable and feasible for patients. The average time taken to complete the questionnaire was 4.11 min (SD = 1.35, range = 2.62–7).

The questions are so concise. You can quickly fill that in, in the waiting room. PID12, Patient with extensive linear porokeratosis, Ireland

While no participants found any of the items offensive or objectionable, they felt that others might be “uncomfortable” (PID9, Patient with psoriasis, UK) with item 26 (“it has been difficult to be intimate with a partner”) as it referred to intimacy, but stressed it was important to include. Instead, participants felt that completing PRIDD initiated a process of reflection on their experiences with their condition:

I wasn’t offended by any of them [items] … It actually made me aware of how much this is actually controlling my life again. PID11, Patient with discoid lupus, Ireland

A minority of participants, most with alopecia, questioned the focus on the negative impacts of dermatological conditions and felt that positive impacts should be included too.

I think it's sometimes nice to balance the negatives out with positives … in the past week, I've felt a lot of empowerment, I've felt a lot of like confidence, I've felt a lot of people praising me for something I've tried to hide away for so long, so it's not just negatives that you could capture as well, having a separate question saying I've felt confident, or I've felt empowered or something. PID15, Patient with alopecia, UK

Some participants wanted to further elaborate on items with qualitative data.

You could even go deeper than that … you could even have … a box to maybe put is there anything you'd like to add … that [you] feel is relevant … because everybody isn't the same PID16, Patient with Pityriasis Rubra Pilaris (PRP), UK

3.2. Feedback on instructions

Overall, the instructions appeared easy to comprehend as participants were able to summarize them accurately and succinctly.

It was asking me to answer the below questions based on my condition, how it's affected me in the last week, and answer them with what's relevant to me and my experience. … I felt the instructions were really clear … it's a fairly straightforward questionnaire PID15, Patient with alopecia, UK

However, some “did not read that part [instructions]” (PID11, Patient with discoid lupus, Ireland), which affected the validity of their answers, particularly in relation to the recall period. On this basis, several sections of the instructions were emboldened to draw respondents’ attention to the instructions and their most important aspects.

Some suggestions to improve clarity were provided. First, participants felt that the example “because you do not work” created confusion as it led participants to believe that the items should be answered in relation to both their dermatological condition and employment. As a result, this example was removed from the instructions.

Going through the questions in my head, I don’t know how work would have anything to do with the questions that were asked … I don’t even really think you need it all. PID14, Patient with psoriasis, Canada

Second, some participants suggested alternatives to the term “dermatological conditions,” feeling it was wordy. A minority of participants with conditions primarily affecting the skin suggested using ‘a simple word like skin’ (PID9, Patient with psoriasis, UK) instead. Others with conditions that did not primarily affect the skin such as alopecia felt that dermatology implied a focus on the skin and would prefer another word, but could not provide a suitable alternative:

Is there another word to say skin, which includes hair, nails, whatever, you know? I don't think there is, but that's the only thing that I would maybe look into, but I don't think there is a synonym. PID18, Patient with alopecia, UK

Because most participants found it acceptable and it is more inclusive than “skin,” the term ‘dermatological conditions’ was retained.

You need it to be applicable to several different conditions, not just one and … there is the difficulty. So with that in mind, your questions are brilliant. PID18, Patient with alopecia, UK

I am sure phrases like dermatological, I mean will be familiar to anyone with any sort of conditions PID13, Patient with psoriasis, UK

3.3. Feedback on the recall period

Participants were almost unanimous in their criticism of the one-week recall period. Many felt that a longer recall period was required to accurately reflect the impact that their dermatological condition has had on their lives, largely due to the relapse and remitting nature of many of these conditions:

Seven days isn’t long enough for someone with … [a] condition that they’ve no control over, and people can see it. Because that’s another thing like, lupus can flare, and it’ll go back down, and I can have three good weeks and then one really crap week where it’s just blown up on my face. So, I still think that the past week is too short a term to ask someone how it is. PID11, Patient with discoid lupus, Ireland

Participants also explained that many people do not engage in some of the experiences captured by the items (e.g., intimacy, life decisions and social activities) on a weekly basis and so expanding the recall period would likely increase item relevance to a higher proportion of patients and consequently more accurately capture disease burden:

If the timeframe had been three months, six months, a year, or your lifetime … the feedback would be very different. So, you talk about relationships, intimate with [a] partner, all of these kinds of things, you know, social interactions, if somebody hasn’t had a social interaction in the last week they’re going to say never, whereas if the timeframe is much larger, you’re going to get a more realistic feedback. PID12, Patient with extensive linear porokeratosis, Ireland

A 2-week and 1-month recall period were tested. These were generally more acceptable to participants than the 1-week recall. A 1-month recall period was adopted, having been suggested as an alternative to the 1-week recall period by multiple participants.

3.4. Feedback on the items

A summary of the evidence of comprehensibility, relevance, and problems detected for each item is presented in Supplementary material 9. Nine of the 27 items remained unchanged because they were easily understood, relevant to participants and distinct from other items.

Sixteen items were modified to align them more closely with the intended concept of interest outlined in the item definition list or to reduce conceptual overlap with other items.

One item, “I have felt dismissed or abandoned by others,” was deleted because it was not easily understood by participants and was felt to be highly similar to “I have been excluded, bullied or discriminated against.”

Overall, participants found the item ordering acceptable. The five domains of the conceptual framework were evident in the items, as participants correctly recognized categories of items, providing evidence in support of the suitability of the item ordering as well as the conceptual framework.

They seemed to be grouped together quite well and I think the order of them was fine. PID15, Patient with alopecia, UK

The order of seven items was changed to enhance understanding. For example, the item “my everyday choices have been affected (for example, choice of clothes, hairstyle or products)” was listed before the item “my life goals and choices have been affected (for example, career choice or having children)” to highlight that the latter does not include everyday choices, which some participants subsumed under life goals and choices.

Subgroup differences were found on four items. People with alopecia felt that the item “my treatment has caused practical problems (for example, by taking up time or being messy)” was not relevant to them as they had no treatments. They also differed from people with other conditions on three items as alopecia appeared to have a positive impact in terms of timesaving, reduced financial costs and feelings of attractiveness.

It positively impacted it [daily routine], because I don't have to mess about with my hair as much in the morning… it's a bit of a blessing. PID15, Patient with alopecia, UK

3.5. Feedback on the response options

Participants found the response options (“never,” “rarely,” “sometimes,” “often,” and “always”) to be appropriate, cover the full range of experience, and comprehensible.

I found it quite easy, I think it gives a good range of options. Obviously always and never are complete extreme [s] and then a couple of the intermediates of different intensity, is fine. I think it is a really good way of asking questionnaires and usually makes it quite easy to answer. PID13, Patient with psoriasis, UK

Some participants had difficulty interpreting the “not relevant” response (NRR) option. NRRs caused confusion, especially for items where the condition had been an obstruction to engaging in the relevant life area. This was most clearly discussed in relation to the item on intimacy, for example:

‘It's been difficult to be intimate with a partner’ … this is the question that I always struggle with. [I chose] not relevant because I've not got a partner, but that can also be always, because I haven't got a partner, it can be both … the reason why I've not got a partner is because it's been difficult … I think it's an important question … I think [psoriasis is] … probably the reason why I'm single. In my formative years between when I started getting psoriasis and in hospital, was years when all my mates were getting wives and babies and all that. All of a sudden it had just passed me by, it had gone, you know. And it's like all of a sudden, I'm 40 odd and I'm like all my mates are married and having kids and I seem to have missed that bit. PID1, Patient with psoriasis, UK

Others could not distinguish the NRR option from the ‘never’ response option.

There were some questions where it was never or … not relevant and I’m thinking … what did not relevant mean? … you might think not relevant is fairly self-explanatory but it’s not in my case … what’s not relevant? … I’ve ticked it and it’s not relevant because it never occurred … [so] you’d say never, wouldn’t you? PID9, Patient with psoriasis, UK

The edits to items 12 (“I have struggled to concentrate”) and 26 (“I have been prevented from or found it difficult to be intimate with another person”) reduced the need for the not relevant option as these items could now apply to all respondents, regardless of employment or relationship status. This option was no longer necessary and was therefore removed to simplify.

4. Discussion

This study tested the content validity, acceptability, and feasibility of PRIDD. It met the highest standards for cognitive interviews outlined by COSMIN (Supplementary material 10) (19, 27), providing high-quality evidence of the comprehensibility, comprehensiveness, and relevance of PRIDD from the target population. The study findings and resultant adjustments produced a 26-item version of PRIDD, ready for field testing, and psychometric testing.

4.1. Implications for measuring the impact of dermatological conditions on the patient’s life

4.1.1. The challenge of dermatology-specific PROMs

With the International Classification of Disease (ICD)-10 (52) classifying over 1,000 dermatological conditions, dermatology patients are a particularly heterogeneous group in relation to age and condition type, relative to other medical specialties. Unsurprisingly then, participants differed in their relation to the term “dermatological condition” but understood the rationale behind this and no alternate sufficiently inclusive terms were suggested.

While PRIDD appears to be relevant to people with dermatological conditions overall, some sub-group differences were found. The physical, psychological, and social impacts were generally consistent across conditions but practical impacts such as time and financial resources differed for people with alopecia. They emphasized the positive impacts of their condition, for example, regaining time lost to styling hair. Nevertheless, the feedback indicated a consensus that PRIDD was relevant and accepted across conditions. We believe this shows that, despite their inherent challenges, dermatology-specific PROMs are appropriate but need to be developed carefully with high levels of patient involvement throughout.

4.1.2. The conceptual framework of impact

The findings of this cognitive interview study support our conceptual framework of the impact of dermatological conditions (26). First, participants’ lived experiences encompassed the biopsychosocial nature of their conditions. Second, no new items or domains were added to PRIDD, and participants could identify which items corresponded to the underlying domains. Third, the data support our previous decision to remove an “impact of healthcare” domain from the original conceptual framework (37). One participant, for example, summarized this decision while reflecting on being dismissed by healthcare professionals saying, “that could be a whole … paper all by itself … that’s a whole different ball game if you get involved in that” PID14. Future work should quantitatively hypothesis test the conceptual framework to complete the evaluation of content validity.

Given the importance patients placed on the influence of healthcare in determining the impact of their condition in our previous work (26, 37), we suggest that these data could form the basis of a separate ‘quality of dermatological care’ measure.

4.1.3. Patient perspectives on issues with response options

We pilot tested a 5-point rating scale with an additional NRR option for each item. NRR options are common in dermatology. The DLQI, the most widely used PROM in dermatology (53), for example, uses a sum score of its 10 items, eight of which have a NRR option that is given the same zero score as “not at all” responses (22). This scoring method assumes that NRRs are due to a lack of disease burden and therefore have no impact on overall quality of life scores. However, recently several independent studies have shown that this interpretation is problematic (54–56) and revised scoring methods have subsequently been proposed (54, 57). Concurrent with these findings, we found the NRR to be problematic as participants differed in their interpretation of this option with consequences for the accuracy of their scores.

This calls into question the current use and scoring of NRRs in dermatology PROMs. Indeed, the emerging body of research on NRRs has shown that approximately 20%–76% of patients provide at least one NRR on the DQLI (54, 56, 58–60). The frequency of NRRs differs across socio-demographic groups with the elderly, females, those not working full time, and less educated patients reporting higher rates of NRRs than others (54). This is may be related to the consistent finding that the DQLI items with the highest rates of NRRs are those on impairment in work and school, sport, sexual relationships, and social activities (54–56, 61, 62), areas of life that may be less applicable to these particular subgroups. Content validity requires that the PROM is comprehensive, comprehensible, and relevant across the target population. Such high frequency and differing rates of NRRs suggest content validity issues with the DLQI as certain items are not relevant to a significant proportion of patients and subgroups groups (31). Indeed, since relevance is a central parameter of content validity (19, 27), the inclusion of items with NRRs, regardless of their rates, could be viewed as a fundamental threat to content validity.

Because NRRs are scored as 0, a higher yield of NRRs should be associated with a lower DLQI score. However, Rencz et al. (54) found the inverse; a greater number of NRRs was associated with a higher DQLI score, indicating poor construct validity. Researchers and clinicians have hypothesized that NRRs do not reflect actual lower quality of life burden but rather the opposite.

To our knowledge, this is the first study to provide the patient perspective on NRRs and clarify the above findings. First, we provide qualitative data to show that socio-demographic factors do play a role in the use of NRRs. For example, the item on work and study was not relevant to people who were retired. Second, our data suggest that low disease burden is not the only reason patients chose an NRR option. As hypothesized previously, NRRs can indicate a high disease burden. This is most clearly seen in items relating to romantic relationships and intimacy. Some participants felt that these items were not relevant to them as they were not in a relationship but explained that their condition was the cause of this. These findings challenge the use of NRRs in dermatology PROMs as they show that the assumption of low disease burden is not always correct.

We overcame the issues inherent in NRRs by following our participants’ recommendation to remove this option. As all the items pilot tested here had been prioritized for inclusion in PRIDD, it was clear that these were important impact concepts to patients and therefore we did not remove items to remove the NRR. Instead, we maximized the applicability of each item across the target population. For example, the item “It has been hard to work or study” was changed to “I have struggled to concentrate” as this tapped the underlying concept while being applicable regardless of employment status or age. We were also careful not to link items too closely to specific examples. To illustrate, the DLQI item on leisure and daily activities asks “how much has your skin interfered with you going shopping or looking after your home or garden?.” The true impact of the condition on the patient’s leisure and daily activities may be hidden by this question if shopping, housework, or gardening are not relevant to them. We used neutral wording, e.g., “my leisure time/activities have been negatively affected” to overcome this. We also reworded the item “It has been difficult to be intimate with a partner” to “I have been prevented from or found it difficult to be intimate with another person” to increase its relevance in light of the finding that NRRs to this item may be due to high disease burden. We believe that by removing the NRR option and rewording items to maximize their applicability across the target population, we have overcome the scoring limitations of the existing dermatology PROMs by using a more valid and reliable method.

4.1.4. Determining the recall period

Choice of recall period is an aspect of internal validity as a suboptimal recall period can introduce measurement error. There is no “gold standard” recall period for PROMs as “one size does not fit all” (63). The FDA guidance (64) on PROM development states a preference for items with short recall periods or those that ask patients to describe their current or recent state. Their rationale is twofold. First, longer recall periods are thought to undermine content validity because they rely on memory and therefore may introduce recall bias. Second, longer recall periods may be impractical in research or clinical practice with frequent data collection points or clinic visits due to overlapping periods. Hence, we initially assumed that a one-week recall period would be the most appropriate for PRIDD.

Study participants almost unanimously criticized the one- and two-week recall periods and proposed longer recall periods (e.g., 1–6 months, years, or lifetime), supporting the concept of Cumulative Life Course Impairment in dermatology (65–68). Participant’s views strengthen previous work suggesting that a shorter recall period likely underestimates the burden of long-term conditions (69), particularly those with a relapsing-and-remitting course (70), results in loss of information (71) and that patients can accurately recall over a longer period of time than the FDA guidance suggests (36), particularly when their issues are bothersome and memorable (69, 72). In response, we changed the recall period to 1 month, which is within the recommended range for PROMs of phenomena such as quality of life and is likely to reduce the risk of recall bias (63).

4.1.5. Acceptability and feasibility of PRIDD

During PROM development, a balance is evident between maximizing the information gained about the construct of interest and reducing respondent burden, meaning that every item in a PROM must earn its place (21). Where participant interviews demonstrated that items were not easily interpreted or clearly aligned with their underlying concept of interest, they were removed or edited during the interview rounds. In this way, this study provides evidence of the value of each of the 26 items.

PRIDD appears to be feasible for use, with participants taking an average of approximately 4 min to complete. The average time to complete PRIDD in research and clinical practice is likely to be lower because, in most cases, participants were thinking and responding aloud while completing PRIDD and edits were consequently made to improve the comprehensibility of the measure.

4.2. Strengths and limitations

Though it is an important step in the development and validation of new PROMs (27), the pilot testing of dermatology-specific PROMs is rare and, when conducted, is often of poor methodological quality (17). This is the first pilot study of a dermatology-specific PROM to both (a) be of high methodological quality and (b) show evidence sufficient content validity, acceptability, and feasibility, according to the COSMIN criteria (27).

The main strength of this pilot test is the use of qualitative methods. The interviews followed the COSMIN-recommended TSTI method of cognitive interviewing (38), eliciting data from a range of sources (i.e., observational, think aloud, and probing techniques). Our approach of asking participants to elaborate in detail regarding their understanding of each aspect of and item in PRIDD provided manifold definitions and examples of impact as well as identifying how participants understood each item. From these data, we could detect and resolve problems with item wording, ordering, and redundancies. Notably, we followed the COSMIN guidance by testing each aspect of PRIDD separately and in its final form, except for a minor adjustment to the instructions (27). PRIDD, therefore, is the first dermatology-specific PROM to meet the COSMIN standards for cognitive interviews (17).

Participants were sampled purposively through GlobalSkin’s unique global network to achieve diversity in terms of clinical (e.g., common and uncommon, inflammatory and non-inflammatory dermatological conditions) and demographic variables (e.g., age and gender). This strengthens the content validity of PRIDD for global use across dermatological conditions. However, our participants were mainly patient organization members and therefore may not represent the experiences or views of non-members. With a sample size of 12, we were able to demonstrate data saturation as no major problems that could be resolved were identified in the final round of interviews. Still, rarer problems or those pertaining to conditions not represented may have gone undetected. Five of the 12 participants had psoriasis and we were unable to recruit participants with other common conditions such as acne, eczema, and vitiligo or those who do not speak British, American, or Canadian English, reducing the transferability of PRIDD. The next stage of development is psychometric testing. Here, we will be able to further test PRIDD with a larger sample of patients representing a wider array of dermatological conditions and global regions and who speak other variations of English.

4.3. Implications for clinical practice

A PROM’s potential to advance person-centered care is contingent upon its applicability, comprehensiveness, and relevance to patients. PRIDD should not replace the discussion of the wider impact of the disease during the clinical consultation, but facilitate and frame patient-centered discussions (73). Indeed, participants in the current study and previous dermatology PROM development work [(e.g., 74)] expressed a desire to provide qualitative information alongside their response options to afford clinicians a deeper understanding of their lived experiences.

As Tourangeau’s (40, 41) cognitive theory demonstrates, the completion of a PROM requires cognitive processing. Clinicians should be aware that a patient’s literacy level, among other factors, may facilitate or create barriers to PROM completion. Consequently, patient-centeredness in the administration of PROMs should be paramount to avoid perpetuating health inequalities.

Consistent with the literature (75, 76), while developing PRIDD, participants have consistently expressed that the psychological aspects of dermatological conditions require more attention (26, 37). There is a need to establish effective psychological interventions and pathways to psychological support, improve clinicians’ skills and confidence to address psychological and social issues (77) and develop effective collaboration between dermatologists and mental health professionals including psychologists.

4.4. Implications for research

Some participants reported positive impacts of their dermatological conditions and expressed a desire for these to be measured as well as the negative impacts. The positive impacts reported—e.g. empowerment, confidence, and gratitude—are congruent with Tedeschi and Calhoun’s (78) work on post-traumatic growth and validate the analysis of our concept elicitation study which also discerned positive impacts (26). Because PRIDD focuses on the burden of dermatological conditions, positive impacts were not incorporated as this would violate the unidimensionality required of the measure. Given the importance of the various positive impacts to patients, the data gathered throughout the development of PRIDD could serve as the basis of a new, separate measure of the positive impact of dermatological conditions. Qualitative research to inform the development of psychological interventions typically focuses on the negative aspects of long-term health conditions, but it can be worthwhile to consult with people with positive experiences as they are well placed to provide input that may lead to effective interventions.

Researchers, clinicians, and regulatory agencies should choose measurement instruments based on their quality. Before PRIDD can be recommended for use in research and clinical practice, validation of the measurement properties (validity, reliability, and responsiveness) and interpretability information (i.e., Minimally Important Change) is required (36). While cognitive interviews allow patients to have greater input into the item refinement process than purely statistical methods allow, ideally, they would not be the sole method of item refinement. Several items were identified as having conceptual overlap in the current study. It will be important to test for item redundancy and data structure in the psychometric testing phase.

5. Conclusion

In this final step in the content validity phase of development, PRIDD was pilot tested through cognitive interviews with the target population. The data triangulated previous findings, recommendations, and the conceptual framework of impact. The results provide insight into how patients understood the items in PRIDD and shed light on the patient perspective on current debates regarding PROMs in dermatology. The resultant confirmation of the comprehensibility, comprehensiveness, relevance, acceptability, and feasibility of PRIDD provides evidence of content validity from the target population. The next step in the development of PRIDD is the psychometric testing phase.

Data availability statement

The datasets presented in this article are not readily available because, due to the nature of this research, participants of this study did not agree for their data to be shared publicly. Requests to access the datasets should be directed to RP, pattinsonr@cardiff.ac.uk.

Ethics statement

This study involves human participants was reviewed and approved by Cardiff University School of Healthcare Sciences Ethics Committee. The patients/participants provided their written informed consent to participate in this study.

Author contributions

RP: conceptualization, methodology, validation, formal analysis, investigation, resources, data curation, writing—original draft, writing—review and editing, visualization, supervision, and project administration. NT-S: methodology, validation, and writing—review and editing. RH, MV, and NT: investigation and writing—review and editing. JA: conceptualization, resources, writing—review and editing, project administration, and funding acquisition. AF: resources, writing—review and editing, and project administration. NC: methodology, writing—review and editing, and supervision. MA: conceptualization, methodology, writing—review and editing, and supervision. CB: conceptualization, methodology, validation, resources, writing—review and editing, and supervision. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by GlobalSkin as a part of its Global Research on the Impact of Dermatological Diseases (GRIDD) project.

Acknowledgments

We would like to thank GlobalSkin for their efforts in conceiving of the GRIDD initiative, project management, and recruiting participants for this study. We are thankful for the generous involvement of participants for sharing their expertise and experiences with us and acknowledge the support of the following patient organizations: Alopecia UK, Canadian Psoriasis Network, HS Ireland, Irish Skin Foundation, PRP Alliance, and Psoriasis Association. We also wish to acknowledge the input of those serving on the GRIDD project’s Scientific Advisory Board: Ramesha Bhat (Father Muller Medical College, India), Arnon D. Cohen (Clalit Health Services, Israel), Ncoza Dlova (Nelson R Mandela School of Medicine, South Africa), Toshiya Ebata (The Jikei University School of Medicine, Japan), Cristina Echeverría (ECHO Psoriasis, Argentina), Alice Gottlieb (Icahn School of Medicine at Mt. Sinai, US), Andrew Finlay (Cardiff University, UK), Luigi Naldi (Ospedale San Bortolo di Vincenza, Italy), Lone Skov (University of Copenhagen, Denmark), and Marc Yale (International Pemphigus Pemphigoid Foundation, US). The contents of this manuscript have previously appeared online in RP’s PhD thesis (79).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1020523/full#supplementary-material

References

1. Batchelor, JM, Ridd, MJ, Clarke, T, Ahmed, A, Cox, M, Crowe, S, et al. The eczema priority setting partnership: a collaboration between patients, carers, clinicians and researchers to identify and prioritize important research questions for the treatment of eczema. Br J Dermatol. (2013) 168:577–82. doi: 10.1111/bjd.12040

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Koo, J, and Lebwohl, A. Psycho dermatology: the mind and skin connection. Am Fam Physician. (2001) 64:1873–8.

PubMed Abstract | Google Scholar

3. Germain, N, Augustin, M, François, C, Legau, K, Bogoeva, N, Desroches, M, et al. Stigma in visible skin diseases—a literature review and development of a conceptual model. J Eur Acad Dermatol Venereol. (2021) 35:1493–504. doi: 10.1111/jdv.17110

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kini, SP, DeLong, LK, Veledar, E, McKenzie-Brown, AM, Schaufele, M, and Chen, SC. The impact of pruritus on quality of life: the skin equivalent of pain. Arch Dermatol. (2011) 147:1153–6. doi: 10.1001/archdermatol.2011.178

CrossRef Full Text | Google Scholar

5. Beiteke, U, Bigge, S, Reichenberger, C, and Gralow, I. Pain and pain management in dermatology. JDDG. J Dtsch Dermatol Ges. (2015) 13:967–87. doi: 10.1111/ddg.12822

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Seidler, EM, and Kimball, AB. Socioeconomic disability in psoriasis. Br J Dermatol. (2009) 161:1410–2. doi: 10.1111/j.1365-2133.2009.09464.x

CrossRef Full Text | Google Scholar

7. World Health Organization. Global report on psoriasis. Report No.: 9789241565189. Geneva: WHO (2016).

Google Scholar

8. Wakkee, M, and Nijsten, T. Comorbidities in dermatology. Dermatol Clin. (2009) 27:137–47. doi: 10.1016/j.det.2008.11.013

CrossRef Full Text | Google Scholar

9. Basra, MKA, and Shahrukh, M. Burden of skin diseases. Expert Rev Pharmacoecon Outcomes Res. (2009) 9:271–83. doi: 10.1586/erp.09.23

CrossRef Full Text | Google Scholar

10. Murray, CJ, Vos, T, Lozano, R, Naghavi, M, Flaxman, AD, Michaud, C, et al. Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990-2010: a systematic analysis for the global burden of disease study 2010. Lancet. (2012) 380:2197–223. doi: 10.1016/S0140-6736(12)61689-4

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Murray, CJL, Barber, RM, Foreman, KJ, Ozgoren, AA, Abd-Allah, F, Abera, SF, et al. Global, regional, and national disability-adjusted life years (DALYs) for 306 diseases and injuries and healthy life expectancy (HALE) for 188 countries, 1990–2013: quantifying the epidemiological transition. Lancet. (2015) 386:2145–91. doi: 10.1016/S0140-6736(15)61340-X

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Griffiths, CEM, van der Walt, JM, Ashcroft, DM, Flohr, C, Naldi, L, Nijsten, T, et al. The global state of psoriasis disease epidemiology: a workshop report. Br J Dermatol. (2017) 177:e4–7. doi: 10.1111/bjd.15610

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Seth, D, Cheldize, K, Brown, D, and Freeman, EE. Global burden of skin disease: inequities and innovations. Curr Dermatol Rep. (2017) 6:204–10. doi: 10.1007/s13671-017-0192-7

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Hollestein, LM, and Nijsten, T. An insight into the global burden of skin diseases. J Investig Dermatol. (2014) 134:1499–501. doi: 10.1038/jid.2013.513

CrossRef Full Text | Google Scholar

15. Hay, RJ, Johns, NE, Williams, HC, Bolliger, IW, Dellavalle, RP, Margolis, DJ, et al. The global burden of skin disease in 2010: an analysis of the prevalence and impact of skin conditions. J Investig Dermatol. (2014) 134:1527–34. doi: 10.1038/jid.2013.446

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Karimkhani, C, Dellavalle, RP, Coffeng, LE, Flohr, C, Hay, RJ, Langan, SM, et al. Global skin disease morbidity and mortality: an update from the global burden of disease study 2013. JAMA Dermatol. (2017) 153:406–12. doi: 10.1001/jamadermatol.2016.5538

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Pattinson, RL, Trialonis-Suthakharan, N, Gupta, S, Henry, AL, Lavallée, JF, Otten, M, et al. Patient-reported outcome measures in dermatology: a systematic review. Acta Derm Venereol. (2021) 101:adv00559. doi: 10.2340/00015555-3884

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Mokkink, LB, de Vet, HCW, Prinsen, CAC, Patrick, DL, Alonso, J, Bouter, LM, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. (2018) 27:1171–9. doi: 10.1007/s11136-017-1765-4

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Terwee, CB, Bot, SD, de Boer, MR, van der Windt, DA, Knol, DL, Dekker, J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. (2007) 60:34–42. doi: 10.1016/j.jclinepi.2006.03.012

CrossRef Full Text | Google Scholar

20. Souza, AC, Alexandre, NMC, and Guirardello, EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude. (2017) 26:649–59. doi: 10.5123/S1679-49742017000300022

PubMed Abstract | CrossRef Full Text | Google Scholar

21. DeVellis, RF. Scale development: theory and applications. 4th ed. Los Angeles, CA: SAGE (2017).

Google Scholar

22. Finlay, AY, and Khan, GK. Dermatology life quality index (DLQI)--a simple practical measure for routine clinical use. Clin Exp Dermatol. (1994) 19:210–6. doi: 10.1111/j.1365-2230.1994.tb01167.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Chren, MM, Lasek, RJ, Flocke, SA, and Zyzanski, SJ. Improved discriminative and evaluative capability of a refined version of Skindex, a quality-of-life instrument for patients with skin diseases. Arch Dermatol. (1997) 133:1433–40. doi: 10.1001/archderm.1997.03890470111018

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chren, MM, Lasek, RJ, Quinn, LM, Mostow, EN, and Zyzanski, SJ. Skindex, a quality-of-life measure for patients with skin disease: reliability, validity, and responsiveness. J Invest Dermatol. (1996) 107:707–13. doi: 10.1111/1523-1747.ep12365600

CrossRef Full Text | Google Scholar

25. Chren, MM, Lasek, RJ, Sahay, AP, and Sands, LP. Measurement properties of Skindex-16: a brief quality-of-life measure for patients with skin diseases. J Cutan Med Surg. (2001) 5:105–10. doi: 10.1177/120347540100500202

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Pattinson, R, Hewitt, RM, Trialonis-Suthakharan, N, Chachos, E, Courtier, N, Austin, J, et al. Development of a conceptual framework for a patient-reported impact of dermatological diseases (PRIDD) measure: a qualitative concept elicitation study. Acta Derm Venereol. (2022) 102:adv00823. doi: 10.2340/actadv.v102.2401

CrossRef Full Text | Google Scholar

27. Terwee, CB, Prinsen, CAC, Chiarotto, A, de Vet, HCW, Bouter, LM, Alonso, J, et al. (2018). COSMIN methodology for assessing the content validity of PROMs - user manual (version 1.0). Amsterdam, Netherlands: COSMIN Group.

Google Scholar

28. Prinsen, C, Mokkink, L, Bouter, L, Alonso, J, Patrick, D, Vet, H, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. (2018) 27:1147–57. doi: 10.1007/s11136-018-1798-3

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kitchen, H, Cordingley, L, Young, H, Griffiths, CE, and Bundy, C. Patient-reported outcome measures in psoriasis: the good, the bad and the missing! Br J Dermatol. (2015) 172:1210–21. doi: 10.1111/bjd.13691

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Gabes, M, Tischer, C, Apfelbacher, C, and Ebisawa, M. Measurement properties of quality-of-life outcome measures for children and adults with eczema: an updated systematic review. Pediatr Allergy Immunol. (2020) 31:66–77. doi: 10.1111/pai.13120

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Hopkins, ZH, Thiboutot, D, Homsi, HA, Perez-Chada, LM, and Barbieri, JS. Patient-reported outcome measures for health-related quality of life in patients with acne vulgaris: a systematic review of measure development and measurement properties. JAMA Dermatol. (2022) 158:900–11. doi: 10.1001/jamadermatol.2022.2260

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Apfelbacher, CJ, and Nelson, PA. Patient-reported outcome measures and qualitative research in dermatology: the quest for authenticity. Br J Dermatol. (2017) 176:285–7. doi: 10.1111/bjd.15251

CrossRef Full Text | Google Scholar

33. Mokkink, LB, Prinsen, CAC, Patrick, DL, Alonso, J, Bouter, LM, de Vet, HCW, et al. (2019). COSMIN study design checklist for patient-reported outcome measurement instruments. COSMIN. Available at: https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf.

Google Scholar

34. Mokkink, LB, Terwee, CB, Knol, DL, Stratford, PW, Alonso, J, Patrick, DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. (2010) 10:22. doi: 10.1186/1471-2288-10-22

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Mokkink, LB, Terwee, CB, Patrick, DL, Alonso, J, Stratford, PW, Knol, DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. (2010) 63:737–45. doi: 10.1016/j.jclinepi.2010.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Brod, M, Tesler, LE, and Christensen, TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res. (2009) 18:1263–78. doi: 10.1007/s11136-009-9540-9

CrossRef Full Text | Google Scholar

37. Trialonis-Suthakharan, N, Pattinson, R, Gandomkari, NT, Austin, J, Janus, C, Courtier, N, et al. Patient prioritisation of items to develop the patient-reported impact of dermatological diseases (PRIDD) measure: a global Delphi study. Manuscript submitted for publication.

Google Scholar

38. de Vet, HCW, Terwee, CB, Mokkink, LB, and Knol, DL. Measurement in medicine: a practical guide. Cambridge: Cambridge University Press (2011).

Google Scholar

39. Terwee, C, Prinsen, C, Chiarotto, A, Westerman, M, Patrick, D, Alonso, J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. (2018) 27:1159–70. doi: 10.1007/s11136-018-1829-0

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Tourangeau, R In: LJ Rips and KA Rasinski, editors. The psychology of survey response. Cambridge, U.K: Cambridge University Press (2000). doi: 10.1017/CBO9780511819322

CrossRef Full Text | Google Scholar

41. Tourangeau, R. Cognitive science and survey methods In: TB Jabine, ML Straf, JM Tanur, and R Tourangeau, editors. Cognitive aspects of survey design: building a bridge between disciplines. Washington, DC: National Academy Press (1984). 73–100.

Google Scholar

42. Hak, T, van Der Veer, K, and Jansen, H. The three-step test-interview (TSTI): an observation-based method for pretesting self-completion questionnaires. Survey Res Methods. (2008) 2:143–50. doi: 10.18148/srm/2008.v2i3.1669

CrossRef Full Text | Google Scholar

43. Jansen, H, and Hak, T. The productivity of the three-step test-interview (TSTI) compared to an expert review of a self-administered questionnaire on alcohol consumption. J Off Stat. (2005) 21:103–20.

Google Scholar

44. MWV, S, Barnard, YF, and Sandberg, J. The think aloud method: a practical guide to modelling cognitive processes. London: Academic Press (1994).

Google Scholar

45. Gagnier, JJ, Lai, J, Mokkink, LB, and Terwee, CB. COSMIN reporting guideline for studies on measurement properties of patient-reported outcome measures. Qual Life Res. (2021) 30:2197–218. doi: 10.1007/s11136-021-02822-4

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Boeije, H, and Willis, G. The cognitive interviewing reporting framework (CIRF): towards the harmonization of cognitive testing reports. Methodology. (2013) 9:87–95. doi: 10.1027/1614-2241/a000075

CrossRef Full Text | Google Scholar

47. Byrne, BM, and Campbell, TL. Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure: a look beneath the surface. J Cross-Cult Psychol. (1999) 30:555–74. doi: 10.1177/0022022199030005001

CrossRef Full Text | Google Scholar

48. Beaton, DE, Bombardier, C, Guillemin, F, and Ferraz, MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. (2000) 25:3186–91. doi: 10.1097/00007632-200012150-00014

CrossRef Full Text | Google Scholar

49. Hawkins, M, Cheng, C, Elsworth, GR, and Osborne, RH. Translation method is validity evidence for construct equivalence: analysis of secondary data routinely collected during translations of the health literacy questionnaire (HLQ). BMC Med Res Methodol. (2020) 20:130. doi: 10.1186/s12874-020-00962-8

CrossRef Full Text | Google Scholar

50. Patrick, DL, Burke, LB, Gwaltney, CJ, Leidy, NK, Martin, ML, Molsen, E, et al. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2--assessing respondent understanding. Value Health. (2011) 14:978–88. doi: 10.1016/j.jval.2011.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Miller, K, Willson, S, Chepp, V, and Ryan, JM. Analysis In: K Miller, editor. Cognitive interviewing methodology. Hoboken, NJ: John Wiley & Sons, Inc. (2014). 35–50.

Google Scholar

52. World Health Organization. ICD-10: International statistical classification of diseases and related health problems: tenth revision. Geneva: World Health Organization (2004).

Google Scholar

53. Maymone, MBC, Laughter, M, Vashi, NA, Jones, JD Jr, Hugh, J, Dunnick, CA, et al. The most cited articles and authors in dermatology: a bibliometric analysis of 1974-2019. J Am Acad Dermatol. (2020) 83:201–5. doi: 10.1016/j.jaad.2019.06.1308

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Rencz, F, Poór, AK, Péntek, M, Holló, P, Kárpáti, S, Gulácsi, L, et al. A detailed analysis of 'not relevant' responses on the DLQI in psoriasis: potential biases in treatment decisions. J Eur Acad Dermatol Venereol. (2018) 32:783–90. doi: 10.1111/jdv.14676

CrossRef Full Text | Google Scholar

55. Langenbruch, A, Radtke, MA, Gutknecht, M, and Augustin, M. Does the dermatology life quality index (DLQI) underestimate the disease-specific burden of psoriasis patients? J Eur Acad Dermatol Venereol. (2019) 33:123–7. doi: 10.1111/jdv.15226

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Barbieri, JS, Chiesa Fuxench, ZC, Shin, DB, and Takeshita, J. Frequency and influence of “not relevant” responses on the dermatology life quality index among adults with atopic dermatitis. Qual Life Res. (2021) 30:1705–13. doi: 10.1007/s11136-021-02770-z

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Rencz, F, Mitev, AZ, Szabó, Á, Beretzky, Z, Poór, AK, Holló, P, et al. A Rasch model analysis of two interpretations of 'not relevant' responses on the dermatology life quality index (DLQI). Qual Life Res. (2021) 30:2375–86. doi: 10.1007/s11136-021-02803-7

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Rencz, F, Brodszky, V, Gulácsi, L, Péntek, M, Poór, AK, Holló, P, et al. Time to revise the dermatology life quality index scoring in psoriasis treatment guidelines. J Eur Acad Dermatol Venereol. (2019) 33:e267–9. doi: 10.1111/jdv.15537

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Barbieri, JS, and Gelfand, JM. Evaluation of the dermatology life quality index scoring modification, the DLQI-R score, in two independent populations. Br J Dermatol. (2019) 180:939–40. doi: 10.1111/bjd.17419

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Gupta, V, Taneja, N, Sati, HC, Sreenivas, V, and Ramam, M. Evaluation of 'not relevant' responses on the dermatology life quality index (DLQI) and the DLQI-R scoring modification among Indian patients with vitiligo. Br J Dermatol. (2021) 184:168–9. doi: 10.1111/bjd.19424

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Hahn, HB, Melfi, CA, Chuang, TY, Lewis, CW, Gonin, R, Hanna, MP, et al. Use of the dermatology life quality index (DLQI) in a midwestern US urban clinic. J Am Acad Dermatol. (2001) 45:44–8. doi: 10.1067/mjd.2001.110880

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Mazzotti, E, Barbaranelli, C, Picardi, A, Abeni, D, and Pasquini, P. Psychometric properties of the dermatology life quality index (DLQI) in 900 Italian patients with psoriasis. Acta Derm Venereol. (2005) 85:409–13. doi: 10.1080/00015550510032832

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Stull, DE, Leidy, NK, Parasuraman, B, and Chassany, O. Optimal recall periods for patient-reported outcomes: challenges and potential solutions. Curr Med Res Opin. (2009) 25:929–42. doi: 10.1185/03007990902774765

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Food and Drug Administration (2009). Patient-reported outcome measures: use in medical product development to support labeling claims. U.S. Department of Health and Human Services.

Google Scholar

65. Warren, RB, Kleyn, CE, and Gulliver, WP. Cumulative life course impairment in psoriasis: patient perception of disease-related impairment throughout the life course: cumulative life course impairment in psoriasis: patient perspectives. Br J Dermatol. (1951) 164:1–14. doi: 10.1111/j.1365-2133.2011.10280.x

CrossRef Full Text | Google Scholar

66. Linder, MD, and Kimball, AB. Dermatological diseases and cumulative life course impairment. Germany: Karger (2013). doi: 10.1159/isbn.978-3-318-02404-3

CrossRef Full Text | Google Scholar

67. Bhatti, ZU, Salek, MS, and Finlay, AY. Major life changing decisions and cumulative life course impairment. J Eur Acad Dermatol Venereol. (2011) 25:245–6. doi: 10.1111/j.1468-3083.2010.03930.x

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Kimball, AB, Gieler, U, Linder, D, Sampogna, F, Warren, RB, and Augustin, M. Psoriasis: is the impairment to a patient's life cumulative? J Eur Acad Dermatol Venereol. (2010) 24:989–1004. doi: 10.1111/j.1468-3083.2010.03705.x

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Leidy, NK, and Vernon, M. Perspectives on patient-reported outcomes: content validity and qualitative research in a changing clinical trial environment. PharmacoEconomics. (2008) 26:363–70. doi: 10.2165/00019053-200826050-00002

CrossRef Full Text | Google Scholar

70. Patrick, DL, Burke, LB, Gwaltney, CJ, Leidy, NK, Martin, ML, Molsen, E, et al. Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 1—eliciting concepts for a new PRO instrument. Value Health. (2011) 14:967–77. doi: 10.1016/j.jval.2011.06.014

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Clarke, PM, Fiebig, DG, and Gerdtham, U-G. Optimal recall length in survey design. J Health Econ. (2008) 27:1275–84. doi: 10.1016/j.jhealeco.2008.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Norquist, JM, Girman, C, Fehnel, S, DeMuro-Mercon, C, and Santanello, N. Choice of recall period for patient-reported outcome (PRO) measures: criteria for consideration. Qual Life Res. (2012) 21:1013–20. doi: 10.1007/s11136-011-0003-8

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Pattinson, R, and Bundy, C. Self-management: PROM s can prompt support. Br J Dermatol. (2020) 183:418–9. doi: 10.1111/bjd.18894

CrossRef Full Text | Google Scholar

74. Howells, L. ‘A box the shape of me’: the challenge of developing and evaluating patient-centred outcomes for use in eczema clinical trials. Nottingham: Nottingham University (2020).

Google Scholar

75. British Association of Dermatologists (2011). National Survey on Psychodermatology services.

Google Scholar

76. All-Party Parliamentary Group on Skin (2020). Mental health and skin disease. London. All-Party Parliamentary Group on Skin.

Google Scholar

77. Nelson, PA, Kane, K, Chisholm, A, Pearce, CJ, Keyworth, C, Rutter, MK, et al. 'I should have taken that further'—missed opportunities during cardiovascular risk assessment in patients with psoriasis in UK primary care settings: a mixed-methods study. Health Expect. (2016) 19:1121–37. doi: 10.1111/hex.12404

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Tedeschi, RD, and Calhoun, LG. Posttraumatic growth: conceptual foundations and empirical evidence. Psychol Inq. (2004) 15:1–18. doi: 10.1207/s15327965pli1501_01

CrossRef Full Text | Google Scholar

79. Pattinson, R. The patient-reported impact of dermatological diseases (PRIDD) measure: a mixed methods measurement development and validation study. Cardiff, UK: Cardiff University (2022).

Google Scholar