The ENTER study (E-DetectioN Tool for Emerging Mental DisoRders): general population recruitment and data integrity in online screening for psychosis risk

Introduction: Effective detection of young people at clinical high-risk for psychosis (CHR-P) is one of the rate-limiting steps in improving outcomes through preventive treatment. ENTER (E-DetectioN Tool for Emerging Mental DisoRders) was developed to refine and increase the specificity of e-detection strategies to identify young people in the community who might be exhibiting emerging symptoms of psychosis. This paper aimed to outline the ENTER procedure and data validation process and the characteristics of the self-selected sample.

Methods: The ENTER study was conducted across sites in London, Glasgow (UK), and Pavia (Italy). Participants from the general population aged 12–35 years were recruited through universities, colleges, flyers, and social media. The online screener collected data on demographics, cognition, speech, environmental risk and protective factors, and frequent subthreshold features that characterise emerging psychotic disorders.

Results: A total of 8,009 participants completed the screener over a period of 3 years. However, only 2,540 responses (32%) were deemed valid. The mean age of the participants was 23 years. The majority were women (77%), identified as white (70%), and had some experience in higher education (82%). Nearly half of the valid sample (48%) scored ≥6 on the Prodromal Questionnaire (PQ-16).

Discussion: The proportion of participants scoring ≥6 on the PQ-16 is consistent with the findings from other European studies in the general population and in outpatient mental health settings. The procedures and sample characteristics reported here provide context for further analyses using the ENTER tool. In addition, the findings highlight the considerable challenge of fraudulent and inauthentic responses in online research—an issue that may have been amplified by the use of financial incentives and recruitment via social media.

Introduction

Psychotic disorders, such as schizophrenia, comprise one of the world’s leading causes of health-related disability (1) and are linked to a high personal burden to the affected individuals, their families, and their carers (2). The first episode of psychosis (FEP) typically occurs during adolescence or early adulthood (3) and is often preceded by a prodromal phase characterised by attenuated psychotic symptoms (4). Attention has therefore been directed towards the early detection of individuals meeting the clinical high-risk for psychosis (CHR-P) criteria (5). CHR-P individuals are typically between 14 and 35 years of age and accumulate various environmental, genetic, and individual risk factors for psychosis (6, 7). The CHR-P status is associated with an increased risk of developing a psychotic disorder (OR = 9.32), which peaks at 2–3 years following the initial clinical assessment and continues to gradually increase over time to a cumulative risk of 0.35 at 10 years of follow-up (8, 9).

Effective detection of young people who might benefit from specialised support is one rate-limiting step for improving outcomes through preventive interventions (10). Currently, the detection of CHR-P individuals is dependent on the available material and human resources of services (11). Psychosis risk assessments are commonly performed by trained personnel in help-seeking individuals who have already been referred or have self-referred to specialised CHR-P clinics (12). Evidence suggests that this is highly inefficient (10). A study of a local NHS Trust found that only 5% of individuals who had FEP had been detected by a local CHR-P service (13). In Australia, the Headspace initiative is a frontline youth mental health service (14, 15). However, despite their “one-stop shop” approach, estimates suggest that only 12% of individuals with FEP are detected during the CHR-P phase (16). As such, the preventive potential of CHR-P services remains largely untapped.

The delayed detection of prodromal symptoms is a major contributing factor to negative long-term outcomes in at-risk individuals (17, 18). Therefore, detecting young people in the community who are experiencing prodromal symptoms and signposting them to specialist treatment may be a step towards improving the outcomes of psychosis. This also aligns with UK clinical guidelines (19, 20), which require the prompt detection and treatment of CHR-P individuals. Promising future avenues for improving the screening and detection of CHR-P individuals in the general population are currently under investigation and include, among others, individualised transdiagnostic (21) or poly-environmental (22, 23) risk calculators, computerised assessments (24), and e-detection strategies aimed at the general population. Among these, e-detection approaches can allow for quick and easy implementation of risk enrichment and detection strategies via the use of web-based versions of self-report scales, such as the Prodromal Questionnaire (PQ-16) (25). This can potentially be used as an initial filter to identify individuals presenting with attenuated psychotic symptomatology who would benefit from a specialised assessment.

Online research has many other advantages, such as having a greater reach, not being limited by geographical location like specialised CHR-P clinics, having increased accessibility, and being able to recruit larger sample sizes in less time with fewer resources for both researchers and participants (26). For mental health research in particular, it can help recruit hard-to-reach and low-prevalence populations and can encourage disclosure of sensitive information, such as mental health symptoms, due to the increased anonymity (26, 27). On the other hand, online health research is faced with important challenges that call for proactive mitigation strategies, notably the risk of fraudulent responses that have been found to affect as low as 3% to as high as 94% of participants in studies (28).

The multi-site ENTER (E-DetectioN Tool for Emerging Mental DisoRders) study aimed at testing and refining an e-detection strategy for the identification of individuals at risk of psychosis in the general population. A longitudinal design was employed to investigate the ability of a self-administered screener (ENTER) to detect true CHR-P, as measured by clinical interview assessments. This method builds upon the Youth Mental Health Risk and Resilience Study (YouR-Study), which provided initial evidence for the feasibility of a web-based screening platform to detect CHR-P individuals in the general population (12). Expanding upon the YouR-Study, ENTER collected information on the following: 1) environmental risk and protective factors; 2) prodromal symptoms of bipolar disorder; and 3) cognitive and speech analysis. All factors have been previously associated with increased risk of psychosis. The aim of the ENTER study was to determine which factors from the screener improve the detection of CHR-P.

This paper outlines the ENTER procedure and recruitment method for automatic digital community screening for psychosis risk, as well as the general sociodemographic characteristics and the PQ-16 scores in the resulting self-selected sample. In addition, this paper describes the cleaning process employed for sample validation to ensure the integrity of the final ENTER study sample.

Methods

Ethics

For the UK sites, the study had NHS Research Ethics Committee approval, and all participants gave written informed consent for participation. The study was registered on ClinicalTrials.gov (NCT05127278). The Italian site adopted the same protocol, but was funded by a different grant and is therefore not listed on ClinicalTrials.gov. At the Italian site, the study was approved by the Ethical Committee of IRCCS San Matteo (no. 0044135/22).

UK site recruitment and study promotion

Participants were recruited from the general population in London and Glasgow. A combination of invitation e-mails through universities and colleges, flyers, and social media (e.g., Facebook and Instagram) was employed (see Supplementary Figure S1). Potential participants could obtain additional information about the study and a link to initiate the electronic consenting process, which, after completion, directed participants to the online screener section of the ENTER study.

A pilot version of the ENTER website was presented at the Young Person’s Mental Health Advisory Group (YPMHAG) at King’s College London, and feedback from young experts by experience was collected to inform the development of the website and the promotional materials.

The criterion for participant inclusion in the screener was age 12–35 years. Absence of a previous psychotic disorder was assessed at the interview stage. Participants were compensated with a £10 Amazon voucher for completing the 30-min screener. Recruitment started in February 2022 and ended in December 2024.

Italy site recruitment and study promotion

Recruitment strategies included e-mail invitations sent to university students, dissemination via internal WhatsApp groups of individual faculties, outreach through student associations, and the distribution of posters and brochures in common youth gathering places and general practitioners’ clinics. The study was also promoted via social media platforms (e.g., Instagram and LinkedIn). Participants at the Italy site in Pavia were recruited via e-mail invitations to students at universities in Pavia. In Italy, financial compensation to participants in clinical research is not permitted, except for reimbursement of direct expenses or documented loss of earnings as per the national implementation of EU Regulation no. 536/2014 and the official model approved by the Italian Coordination Centre of Ethics Committees (29). Therefore, unlike in the UK, participants at the Pavia site did not receive financial compensation.

Study design

Screener

The consent form and screener was built and managed through the online platform REDCap (30).

At enrolment, participants provided their age and site location before viewing the participant information sheet (PIS). The PIS explained that, after completion of the screener, eligible participants—the criteria were not disclosed to them—might be contacted by e-mail or phone to schedule an interview. At the bottom of the PIS, participants gave consent and submitted their contact details.

In the screener, participants were asked to complete demographic questions, the prodromal questionnaire (PQ-16) (31), the Psychosis Polyrisk Score (PPS) assessment (23), and the Digit Symbol Substitution Test (DSST) (32). Participants from the London and Italy sites also completed the self-rating version of the Bipolar Prodrome Symptom Scale (BPSS-AS-P) (33). The BPSS-AS-P scale was not included for the Glasgow site.

In the general population, the Cronbach’s alpha for the total score on the PQ-16 has been found to be 0.774 (31).

Participants were then directed to the online experiment builder platform Gorilla (www.gorilla.sc). Here, speech samples were collected using the eight images from the Thematic Apperception Test (TAT) as stimuli (34). When the image appeared, the accompanying text read, “Please describe what you see in this image. Please speak for the full minute—the recording has started,” and at 30 s, “How does this image make you feel?”

Across sites, participants were also asked the following question at the end of the PPS assessment: “How distressing did you find the process of completing this questionnaire?,” with the response options ranging from “not distressing at all” to “extremely distressing.”

Figure 1 shows the overall study design.

Figure 1

Figure 1. ENTER (E-DetectioN Tool for Emerging Mental DisoRders) study design. *The BPSS-AS-P (Bipolar Prodrome Symptom Scale) was not used at the Glasgow site.

Data cleaning

Given the prevalence of suspected fraudulent or inauthentic responses in web-based survey research, a multilayered data cleaning system was implemented to improve the integrity of the ENTER study. The detailed methodology can be found in the Supplementary Material. This manuscript uses the umbrella term “invalid data” to refer to entries that were judged to be potentially fraudulent or inauthentic, as well as lower-quality responses, such as those failing attention checks or not meeting the time thresholds.

Statistical analysis

Data visualisation and analysis were performed using R Statistical Software (v4.3.1; 35). The Kolmogorov–Smirnov test and the Mann–Whitney U test were performed on the PQ-16 data (see Supplementary Material).

Results

UK data cleaning

After passing CAPTCHA, 8,009 entries across all sites were submitted via the screener. One participant asked to have their data withdrawn.

The UK screener received 4,105 entries via the Glasgow site and 3,261 entries via the London site; 26 entries did not reach the question asking where they were recruited from. The ratio of valid to invalid entries was around 2:5 (Figure 2) (see Supplementary Material). The most common reason for the entries being excluded (83%) was that they were significantly incomplete (i.e., had not finished the DSST). A substantial number of entries (27%) were also ruled out due to completing the PQ-16 or DSST faster than it would be possible to make a considered decision. This may have been due to bots or people speeding through the screener. After data cleaning, the valid UK sample consisted of 2,110 entries.

Figure 2

Figure 2. UK validation process with the number of entries excluded at each stage. The entries in the automatically invalid box may fall into multiple categories, making them invalid. In the manually invalid category, the entries were categorised into one reason.

Italy data cleaning

The Italy screener received 616 entries. Similarly, the most common reason for entries being excluded in the Italy sample was that they were significantly incomplete (99%) (Figure 3). The only reason for invalidating automatically valid entries was because the participant had already completed the screener previously (3%). After data cleaning, the valid Italy sample consisted of 430 entries.

Figure 3

Figure 3. Italy validation process with the number of entries excluded at each stage. The entries in the automatically invalid box may fall into multiple categories, making them invalid. In the manually invalid category, the entries were categorised into one reason.

Recruitment of the valid sample

Participants were predominantly recruited through universities (Figure 4). The majority of the Italy sample (83%) did not respond to the recruitment question.

Figure 4

Figure 4. Frequency of the recruitment methods from participants that answered this question (n = 2,312).

Sample demographics

The demographic information for the valid sample is summarised in Table 1. One participant did not provide a date of birth.

Table 1

Table 1. Demographics of the valid sample.

PQ-16 scores

The across-site valid sample had a full range of PQ-16 scores, with a median score of 5 (Figure 5). The mean score was 5.7 (SD = 3.7). Approximately 48% (n = 1,213) reached the threshold (six items) to be invited to the assessment phase. The distress scores ranged from 0 to 43, with a mean of 8.6 (SD = 7.1) and a median of 7.

Figure 5

Figure 5. Frequency of the prodromal questionnaire (PQ-16) scores.

Screener experience

Just over half of the sample (53.6%) found that the screener was “not distressing at all.” Of the rest, 43.0% found it “mildly distressing,” 3.1% “very distressing,” and 0.2% “extremely distressing.” The remaining 0.1% did not answer the question.

Discussion

The aim of the ENTER study was to implement digital screening for emerging psychosis in the community and identify key factors from the screener that improve the detection of true CHR-P as assessed by the CAARMS. The specific aims of this paper were to outline the ENTER procedure, the recruitment methods for automatic digital community screening, and the data cleaning process and to report on the general sociodemographic characteristics and the results of the PQ-16 in the resulting self-selected sample.

The study screened 2,540 participants from sites in London (England), Glasgow (Scotland), and Pavia (Italy). The majority of the sample were of the female sex (77%), self-described white ethnicity (70%), and had some experience in higher education (82%). The mean age was 23 years (the range 12–35 years was set as an inclusion criterion). Just under half (45%) were migrants, either first or second generation. University invitations were the most effective recruitment strategy, accounting for 60% of the participants, which may explain the demographics of the sample.

Almost half of the sample (48%) scored ≥6 on the PQ-16, reaching the threshold set for CAARMS assessment. This was similar to the YouR-study, which found that 52% of its participants aged 16–35 years met the PQ-16 threshold (12). The Tone-P study also employed a web-based screening approach based on the YouR-study to identify CHR-P individuals and found that 43% of participants aged 18–35 years reached the PQ-16 threshold (36).

In addition, the percentage falls within the 35%–65% range reported by other European studies that recruited adolescents and young people from outpatient mental health and child and adolescent services, despite ENTER recruiting from non-clinical sources (37–41). The pretest risk in community samples is typically lower than that in help-seeking samples. The inclusion of many low-risk individuals can create a risk dilution effect, reducing the proportion of true CHR-P cases. To mitigate this and reduce false positives, community samples can be enriched by focusing on participants with a higher pretest risk, for example, those scoring ≥6 on the PQ-16 (42). The rationale for the PQ-16 threshold was based on the finding of Ising et al. (31) that a score of 6 or more on the PQ-16 has a high true-positive rate and specificity when differentiating people at ultra-high risk (UHR) of psychosis from those who are CAARMS-negative. There is some discussion as to whether this threshold is high enough and should take into account the distress scores, especially in non-help-seeking general population samples (37, 39, 41). Subsequent papers on ENTER will publish the CHR-P status of this sample.

This study also highlighted the significant challenge of potentially fraudulent or inauthentic responses in online community samples. Ultimately, in our UK sample, where the participants were paid, over 70% of the responses were deemed invalid and were excluded from the analysis. It is important to note that the main reason for data exclusion was significantly incomplete responses.

Incentive levels may have played a role; for example, ENTER provided a £10 Amazon voucher upon screening completion for the UK sample, while the YouR-study only offered entry into a prize draw. While financial remuneration can help remove barriers to access, it has been found that studies with incentives receive six times more fraudulent behaviour by participants than studies that do not include participant payments (26, 43). The focus of cleaning the data from the Italy sample was on ensuring that the clinical measures were complete and therefore the validation process is not directly comparable (see Supplementary Material). However, the Italian sample, which was not financially incentivised, had no participant responses invalidated due to not passing at least one attention check, providing a pattern of responses or inaccurate contact information.

The promotion of the study on social media may have further contributed to invalid responses, as this can attract participants primarily seeking incentives and facilitate the sharing of strategies to bypass validation measures (27, 44). In comparison, the YouR-study recruited through e-mail invitations sent to universities and colleges in Glasgow and Edinburgh and through flyers and posters, while the Tone-P study recruited through the undergraduate mailing lists of French and UK universities and Amazon’s Mechanical Turk. In ENTER, the top reported recruitment methods in the invalid sample in order (excluding those who did not finish the screener) were social media (33%, namely, via Facebook), friends (31%), and alleged university invitations (20%) (see Supplementary Material).

Given that the aim of the ENTER study was to identify key factors from the screener that improve CHR-P detection, it was paramount to undergo an extensive data cleaning process for future analyses, particularly with machine learning, to avoid the “garbage in, garbage out” problem. Whilst the detection of CHR-P enriched by an online screener is at the research stage, we encourage others conducting online research to proactively anticipate and implement practical considerations to mitigate the risk of fraudulent responses, in particular if monetary incentives are involved. In our experience, no single measure was single-handedly able to successfully identify and screen out all fraudulent respondents. As suggested in previous studies, a combination of automated and manual measures across the whole life span of the project appears to be the most effective strategy (45).

Overall, the results of this study, thus far, showed that online screening for psychosis risk in the general population is a feasible and promising strategy. In addition, real-world clinical settings may naturally mitigate many of the challenges outlined, as there will be a less intensive social media recruitment drive, no financial incentive, and people will be primarily motivated by help-seeking.

In conclusion, nearly half of the young people recruited from the general population who responded to an advert asking “Are you experiencing mental health difficulties?” reached a score of 6 on the PQ-16, which is consistent with the findings of other European studies. The procedures, data cleaning process, and sample characteristics reported here provide important context for future analyses and publications using the ENTER sample. This paper also contributes to addressing the ongoing challenge of fraudulent and inauthentic responses in online research.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by East Midlands - Leicester Central Research Ethics Committee (21/EM/0283) in the UK and Ethical Committee of IRCCS San Matteo (0044135/22) in Italy. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin.

Author contributions

PW: Conceptualization, Data curation, Formal Analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. AE: Conceptualization, Data curation, Investigation, Methodology, Writing – original draft, Writing – review & editing. MA: Conceptualization, Data curation, Investigation, Project administration, Supervision, Writing – review & editing. KH: Conceptualization, Data curation, Investigation, Project administration, Supervision, Writing – review & editing. XL: Conceptualization, Data curation, Writing – review & editing. TS: Conceptualization, Methodology, Supervision, Writing – review & editing. KD: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – original draft, Writing – review & editing. UP: Conceptualization, Methodology, Supervision, Writing – review & editing. PU: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing. PF-P: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research and/or publication of this article. This study was supported by Wellcome Trust (215793/Z/19/Z) and additional funding from the Academy for Medical Sciences through the Springboard Award (SBF007\100058).

Acknowledgments

Thank you to Bianca Bianciardi, Saskia Cooper, Elisa Figazzolo, Valentia Floris, Soumita Ramesh and Riccardo Stefanelli for investigation. Thank you to Rebecca Warner-Hodgkin for project administration. Thank you to Patrizia Di Campli San Vito and Shaun Macdonald for methodology.

Conflict of interest

PF-P has received research fees from Lundbeck and honoraria from Lundbeck, Angelini, Menarini, and Boehringer Ingelheim, outside of the current study.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer PB declared a shared parent affiliation with the author PFP to the handling editor at the time of the review.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2025.1665854/full#supplementary-material

References

1. James SL, Abate D, Abate KH, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. (2018) 392:1789–858. doi: 10.1016/S0140-6736(18)32279-7

PubMed Abstract | Crossref Full Text | Google Scholar

2. Fusar-Poli P, Estradé A, Stanghellini G, Venables J, Onwumere J, Messas G, et al. The lived experience of psychosis: a bottom-up review co-written by experts by experience and academics. World Psychiatry. (2022) 21:168–88. doi: 10.1002/wps.20959

PubMed Abstract | Crossref Full Text | Google Scholar

3. Solmi M, Radua J, Olivola M, Croce E, Soardo L, Salazar de Pablo G, et al. Age at onset of mental disorders worldwide: large-scale meta-analysis of 192 epidemiological studies. Mol Psychiatry. (2022) 27:281–95. doi: 10.1038/s41380-021-01161-7

PubMed Abstract | Crossref Full Text | Google Scholar

4. Klosterkötter J, Hellmich M, Steinmeyer EM, and Schultze-Lutter F. Diagnosing schizophrenia in the initial prodromal phase. Arch Gen Psychiatry. (2001) 58:158–64. doi: 10.1001/archpsyc.58.2.158

PubMed Abstract | Crossref Full Text | Google Scholar

5. Fusar-Poli P, Salazar de Pablo G, Correll CU, Meyer-Lindenberg A, Millan MJ, Borgwardt S, et al. Prevention of psychosis: advances in detection, prognosis, and intervention. JAMA Psychiatry. (2020) 77:755–65. doi: 10.1001/jamapsychiatry.2019.4779

PubMed Abstract | Crossref Full Text | Google Scholar

6. Oliver D, Reilly TJ, Baccaredda Boy O, Petros N, Davies C, Borgwardt S, et al. What causes the onset of psychosis in individuals at clinical high risk? A Meta-analysis of Risk and Protective Factors. Schizophr Bull. (2020) 46:110–20. doi: 10.1093/schbul/sbz039

PubMed Abstract | Crossref Full Text | Google Scholar

7. Radua J, Ramella-Cravaro V, Ioannidis JPA, Reichenberg A, Phiphopthatsanee N, Amir T, et al. What causes psychosis? An umbrella review of risk and protective factors. World Psychiatry. (2018) 17:49–66. doi: 10.1002/wps.20490

PubMed Abstract | Crossref Full Text | Google Scholar

8. Fusar-Poli P, De Micheli A, Signorini L, Baldwin H, Salazar de Pablo G, and McGuire P. Real-world long-term outcomes in individuals at clinical risk for psychosis: The case for extending duration of care. EClinicalMedicine. (2020) 28:100578. doi: 10.1016/j.eclinm.2020.100578

PubMed Abstract | Crossref Full Text | Google Scholar

9. Salazar de Pablo G, Radua J, Pereira J, Bonoldi I, Arienti V, Besana F, et al. Probability of transition to psychosis in individuals at clinical high risk: an updated meta-analysis. JAMA Psychiatry. (2021) 78:970–8. doi: 10.1001/jamapsychiatry.2021.0830

PubMed Abstract | Crossref Full Text | Google Scholar

10. Fusar-Poli P, Sullivan SA, Shah JL, and Uhlhaas PJ. Improving the detection of individuals at clinical risk for psychosis in the community, primary and secondary care: an integrated evidence-based approach. Front Psychiatry. (2019) 10:774. doi: 10.3389/fpsyt.2019.00774

PubMed Abstract | Crossref Full Text | Google Scholar

11. Salazar de Pablo G, Estradé A, Cutroni M, Andlauer O, and Fusar-Poli P. Establishing a clinical service to prevent psychosis: What, how and when? Systematic review. Transl Psychiatry. (2021) 11:1–14. doi: 10.1038/s41398-020-01165-x

PubMed Abstract | Crossref Full Text | Google Scholar

12. McDonald M, Christoforidou E, Van Rijsbergen N, Gajwani R, Gross J, Gumley AI, et al. Using online screening in the general population to detect participants at clinical high-risk for psychosis. Schizophr Bull. (2019) 45:600–9. doi: 10.1093/schbul/sby069

PubMed Abstract | Crossref Full Text | Google Scholar

13. Fusar-Poli P, Rutigliano G, Stahl D, Davies C, Bonoldi I, Reilly T, et al. Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry. (2017) 74:493–500. doi: 10.1001/jamapsychiatry.2017.0284

PubMed Abstract | Crossref Full Text | Google Scholar

14. Hetrick SE, Bailey AP, Smith KE, Malla A, Mathias S, Singh SP, et al. Integrated (one-stop shop) youth health care: best available evidence and future directions. Med J Aust. (2017) 207:S5–S18. doi: 10.5694/mja17.00694

PubMed Abstract | Crossref Full Text | Google Scholar

15. Rickwood D, Paraskakis M, Quin D, Hobbs N, Ryall V, Trethowan J, et al. Australia’s innovation in youth mental health care: The headspace centre model. Early Interv Psychiatry. (2019) 13:159–66. doi: 10.1111/eip.12740

PubMed Abstract | Crossref Full Text | Google Scholar

16. McGorry PD, Hartmann JA, Spooner R, and Nelson B. Beyond the “at risk mental state” concept: transitioning to transdiagnostic psychiatry. World Psychiatry. (2018) 17:133–42. doi: 10.1002/wps.20514

PubMed Abstract | Crossref Full Text | Google Scholar

17. Fusar-Poli P, Meneghelli A, Valmaggia L, Allen P, Galvan F, McGuire P, et al. Duration of untreated prodromal symptoms and 12-month functional outcome of individuals at risk of psychosis. Br J Psychiatry. (2009) 194:181–2. doi: 10.1192/bjp.bp.107.047951

PubMed Abstract | Crossref Full Text | Google Scholar

18. Zhang T, Cui H, Wei Y, Tang X, Xu L, Hu Y, et al. Duration of untreated prodromal psychosis and cognitive impairments. JAMA Netw Open. (2024) 7:e2353426. doi: 10.1001/jamanetworkopen.2023.53426

PubMed Abstract | Crossref Full Text | Google Scholar

19. NICE. Psychosis and Schizophrenia in Children and Young People: Recognition and Management (CG155) (2016). Available online at: https://www.nice.org.uk/guidance/cg155 (Accessed April 10, 2025).

Google Scholar

20. NICE. Psychosis and Schizophrenia in Adults: Prevention and Management (CG178 (2014). Available online at: https://www.nice.org.uk/guidance/cg178 (Accessed April 10, 2025).

Google Scholar

21. Arribas M, Oliver D, Patel R, Kornblum D, Shetty H, Damiani S, et al. A transdiagnostic prodrome for severe mental disorders: an electronic health record study. Mol Psychiatry. (2024) 29:3305–15. doi: 10.1038/s41380-024-02533-5

PubMed Abstract | Crossref Full Text | Google Scholar

22. Oliver D, Spada G, Englund A, Chesney E, Radua J, Reichenberg A, et al. Real-world digital implementation of the Psychosis Polyrisk Score (PPS): A pilot feasibility study. Schizophr Res. (2020) 226:176–83. doi: 10.1016/j.schres.2020.04.015

PubMed Abstract | Crossref Full Text | Google Scholar

23. Oliver D, Radua J, Reichenberg A, Uher R, and Fusar-Poli P. Psychosis polyrisk score (PPS) for the detection of individuals at-risk and the prediction of their outcomes. Front Psychiatry. (2019) 10:174. doi: 10.3389/fpsyt.2019.00174

PubMed Abstract | Crossref Full Text | Google Scholar

24. Mittal VA, Ellman LM, Strauss GP, Walker EF, Corlett PR, Schiffman J, et al. Computerized assessment of psychosis risk †. J Psychiatry Brain Sci. (2021) 6:e210011. doi: 10.20900/jpbs.20210011

PubMed Abstract | Crossref Full Text | Google Scholar

25. Savill M, D’Ambrosio J, Cannon TD, and Loewy RL. Psychosis-risk screening in different populations using the Prodromal Questionnaire - a systematic review. Early Interv Psychiatry. (2018) 12:3–14. doi: 10.1111/eip.12446

PubMed Abstract | Crossref Full Text | Google Scholar

26. Davies MR, Monssen D, Sharpe H, Allen KL, Simms B, Goldsmith KA, et al. Management of fraudulent participants in online research: Practical recommendations from a randomized controlled feasibility trial. Int J Eat Disord. (2024) 57:1311–21. doi: 10.1002/eat.24085

PubMed Abstract | Crossref Full Text | Google Scholar

27. Wang K, Zhao Y, Tan S, Zhang J, Li D, Chen J, et al. Semantic processing event-related potential features in patients with schizophrenia and bipolar disorder. Psych J. (2020) 9:247–57. doi: 10.1002/pchj.321

PubMed Abstract | Crossref Full Text | Google Scholar

28. Comachio J, Poulsen A, Bamgboje-Ayodele A, Tan A, Ayre J, Raeside R, et al. Identifying and counteracting fraudulent responses in online recruitment for health research: a scoping review. BMJ Evid-Based Med. (2025) 30:173–82. doi: 10.1136/bmjebm-2024-113170

PubMed Abstract | Crossref Full Text | Google Scholar

29. Agenzia Italiana Del Farmaco- Centro di Coordinamento Comitati Etici. Modello Rimborso spese e indennità per i partecipanti alla sperimentazione (2022). Available online at: www.aifa.gov.it/centro-coordinamento-comitati-etici (Accessed September 17, 2025).

Google Scholar

30. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, and Conde JG. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. (2009) 42:377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | Crossref Full Text | Google Scholar

31. Ising HK, Veling W, Loewy RL, Rietveld MW, Rietdijk J, Dragt S, et al. The validity of the 16-item version of the prodromal questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr Bull. (2012) 38:1288–96. doi: 10.1093/schbul/sbs068

PubMed Abstract | Crossref Full Text | Google Scholar

32. Wechsler D. Wechsler adult intelligence scale. 4th ed. Bloomington, MN: PsychCorp (2008).

Google Scholar

33. Van Meter A, Guinart D, Bashir A, Sareen A, Cornblatt BA, Auther A, et al. Bipolar prodrome symptom scale - abbreviated screen for patients: description and validation. J Affect Disord. (2019) 249:357–65. doi: 10.1016/j.jad.2019.02.040

PubMed Abstract | Crossref Full Text | Google Scholar

34. Murray HA and Cambridge MA. The Thematic Apperception Test Manual. Cambridge, Massachusetts: Harvard University Press (1943).

Google Scholar

35. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. (2023). Available at: https://www.R-project.org/.

Google Scholar

36. Dondé C, Palmer-Cooper E, Gauld C, Polosan M, and Alderson-Day B. Early auditory impairments as a candidate marker of attenuated sensory symptoms of psychosis. Prog Neuropsychopharmacol Biol Psychiatry. (2025) 136:111214. doi: 10.1016/j.pnpbp.2024.111214

PubMed Abstract | Crossref Full Text | Google Scholar

37. Lorenzo P, Silvia A, Federica P, Sara G, Ilaria S, Pupo S, et al. The Italian version of the 16-item prodromal questionnaire (iPQ-16): Field-test and psychometric features. Schizophr. Res. (2018) 199:353–60. doi: 10.1016/j.schres.2018.03.023

PubMed Abstract | Crossref Full Text | Google Scholar

38. de Jong Y, Boon AE, Mulder CL, and van der Gaag M. Do help-seeking adolescents report more psychotic-like experiences than young adults on the 16-item version of the prodromal questionnaire (PQ-16)? Early Interv Psychiatry. (2025) 19:e13597. doi: 10.1111/eip.13597

PubMed Abstract | Crossref Full Text | Google Scholar

39. de Jong Y, Mulder CL, Boon A, Coenders E, and van der Gaag M. Cross validation of the prodromal questionnaire 16-item version in an adolescent help-seeking population. Schizophr Bull Open. (2020) 1:sgaa033. doi: 10.1093/schizbullopen/sgaa033

PubMed Abstract | Crossref Full Text | Google Scholar

40. de Jong Y, Mulder CL, Boon AE, Deen M, van ‘t Hof M, and van der Gaag M. Screening for psychosis risk among adolescents in Child and Adolescent Mental Health Services: a description of the first step with the 16-item version of the Prodromal Questionnaire (PQ-16). Early Interv Psychiatry. (2018) 12:669–76. doi: 10.1111/eip.12362

PubMed Abstract | Crossref Full Text | Google Scholar

41. Pelizza L, Azzali S, Paterlini F, Garlassi S, Scazza I, Chiri LR, et al. Screening for psychosis risk among help-seeking adolescents: Application of the Italian version of the 16-item prodromal questionnaire (iPQ-16) in child and adolescent neuropsychiatry services. Early Interv Psychiatry. (2019) 13:752–60. doi: 10.1111/eip.12554

PubMed Abstract | Crossref Full Text | Google Scholar

42. Fusar-Poli P, Schultze-Lutter F, Cappucciati M, Rutigliano G, Bonoldi I, Stahl D, et al. The dark side of the moon: meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophr Bull. (2016) 42:732–43. doi: 10.1093/schbul/sbv162

PubMed Abstract | Crossref Full Text | Google Scholar

43. Bowen AM, Daniel CM, Williams ML, and Baird GL. Identifying multiple submissions in internet research: preserving data integrity. AIDS Behav. (2008) 12:964–73. doi: 10.1007/s10461-007-9352-2

PubMed Abstract | Crossref Full Text | Google Scholar

44. Pozzar R, Hammer MJ, Underhill-Blazey M, Wright AA, Tulsky JA, Hong F, et al. Threats of bots and other bad actors to data quality following research participant recruitment through social media: cross-sectional questionnaire. J Med Internet Res. (2020) 22:e23021. doi: 10.2196/23021

PubMed Abstract | Crossref Full Text | Google Scholar

45. Lawlor J, Thomas C, Guhin AT, Kenyon K, Lerner MD, and Drahota A. Suspicious and fraudulent online survey participation: Introducing the REAL framework. Methodol Innov. (2021) 14:20597991211050467. doi: 10.1177/20597991211050467

Crossref Full Text | Google Scholar