Abstract
Background:
Chronic dermatologic disorders can cause significant emotional distress. Sentiment analysis of disease-related tweets helps identify patients’ experiences of skin disease.
Objective:
To analyze the expressed sentiments in tweets related to alopecia areata (AA), hidradenitis suppurativa (HS), and psoriasis (PsO) in comparison to fibromyalgia (FM).
Methods:
This is a cross-sectional analysis of Twitter users’ expressed sentiment on AA, HS, PsO, and FM. Tweets related to the diseases of interest were identified with keywords and hashtags for one month (April, 2022) using the Twitter standard application programming interface (API). Text, account types, and numbers of retweets and likes were collected. The sentiment analysis was performed by the R “tidytext” package using the AFINN lexicon.
Results:
A total of 1,505 tweets were randomly extracted, of which 243 (16.15%) referred to AA, 186 (12.36%) to HS, 510 (33.89%) to PsO, and 566 (37.61%) to FM. The mean sentiment score was −0.239 ± 2.90. AA, HS, and PsO had similar sentiment scores (p = 0.482). Although all skin conditions were associated with a negative polarity, their average was significantly less negative than FM (p < 0.0001). Tweets from private accounts were more negative, especially for AA (p = 0.0082). Words reflecting patients’ psychological states varied in different diseases. “Anxiety” was observed in posts on AA and FM but not posts on HS and PsO, while “crying” was frequently used in posts on HS. There was no definite correlation between the sentiment score and the number of retweets or likes, although negative AA tweets from public accounts received more retweets (p = 0.03511) and likes (p = 0.0228).
Conclusion:
The use of Twitter sentiment analysis is a promising method to document patients’ experience of skin diseases, which may improve patient care through bridging misconceptions and knowledge gaps between patients and healthcare professionals.
Introduction
Twitter, with over 320 million users, allows close to real-time exchange of ideas about current affairs through microblogging that consists of up to 280 characters (1, 2). The use of sentiment analysis on Twitter posts in medicine was first published in 2009 (3). This technique is a subfield of natural language processing whose aim is to automatically classify the expressed sentiment in texts (4). Since then, it has been widely applied to predict disease outbreaks (5–8), prescription of drugs and adverse drug reactions (9–13), patient satisfaction (14), public perceptions (15), and many others (16, 17). Other features of Twitter such as “likes” and “retweets” enable users to share, to show appreciation, and to propagate information that can be used to monitor trends in public perceptions. Sentiment analysis on this large dataset can provide an overview of the moods and emotional outcomes that are associated with specific diseases and physiological status. This method has the advantage of covering larger populations and geographic areas compared to traditional questionnaire-based methods (18).
As more and more people are turning to social media for health advice, understanding the sentiments of social media posts has become increasingly relevant (19, 20), as patients frequently report a lack of opportunity to express their psychosocial needs (21, 22). However, analysis of social media data in dermatology remains underutilized. Because dermatologic diseases are linked to numerous mental, physical, and emotional stressors that may not be easily captured during clinical visits, we believe that leveraging social media posts can help elucidate the subjective experience of dermatologic disorders. Thus, the objectives of this study were (1) to analyze the expressed sentiments in tweets related to alopecia areata (AA), hidradenitis suppurativa (HS), and psoriasis (PsO) (9–25); (2) to compare the sentiments related to skin disorders with that related to fibromyalgia (FM), a chronic musculoskeletal disease (26) without cutaneous manifestations; and (3) to validate the use of social media analysis for disease surveillance.
Materials and methods
Data collection
We used the standard Twitter application programming interface (API) to collect tweets containing keywords or tags for the diseases of interest. For HS, these included #Hidradenitis, #Suppurativa, #HidradenitisSuppurativa, #HSawareness, and “Hidradenitis Suppurativa”; for AA, “Alopecia areata”, #AlopeciaAreata, Areata, and AAAwareness; for PsO, Psoriasis and #Psoriasis; and for FM, Fibromyalgia, #Fibromyalgia, and #ChronicFatigueSyndrome. Searches using the Twitter API were case insensitive. There was a 180 requests per minute limitation with the standard API limits, which was considered sufficient for this study. Requests to the Twitter API were made through the “retweet” package in R Studio. Tweets that were publicly available and written in English were collected every day for 1 month (from April 1st, 2022, to April 30th, 2022). For each tweet, we obtained data on the date and time of creation, the user’s publicly displayed name, device type, tweet body text, and like and retweet status. A subgroup analysis of private/individual vs. public/organizational accounts (both types of accounts were open to public access) was carried out to determine whether discrepancies in illness experience exist.
Sentiment analysis
To determine the expressed tones in each tweet, we used the AFINN lexicon developed by Finn Arup Nielsen and downloaded from the R “tidytext” package (27). The AFINN lexicon assigned a score between −5 (e.g., “bastard” and “twat”) and + 5 (e.g., “breathtaking” and “superb”) to each word, with negative scores suggestive of negative sentiment.
Statistical analysis
The sentiment of each post was determined by the summation of the sentiment score of each word in the post. Independent t-tests were used to compare the means and standard deviations (SD) of sentiment scores between the diseases of interest. A p-value less than 0.05 suggested statistical significance. The data were collected and analyzed with RStudio 2022.07.1 + 554 for Mac (Boston, USA).
Results
We identified 243, 186, 510, and 566 tweets related to AA, HS, PsO, and FM, respectively. The mean sentiment score was −0.239 ± 2.90. The median and mode were 0. The average scores [mean ± SD (range)] for AA, HS, and PsO were −0.021 ± 3.29 (−10- + 10), −0.341 ± 2.41 (−10- + 6), and −0.308 ± 2.86 (−17- + 14), respectively (Figure 1). There was no significant difference among the three disorders (p = 0.482). There were 2–3 times more tweets from private accounts than from public accounts for all diseases. Posts from public accounts were significantly more positive (−0.128 ± 2.95 vs. −0.731 ± 3.21, p = 0.0008), especially for AA (0.729 ± 2.50 vs. −0.458 ± 3.61, p = 0.0082). On average, there were 0.656 ± 2.26 retweets and 5.77 ± 54.7 likes for each post.
FIGURE 1
AFINN sentiment scores of tweets on different diseases.
Words in negative and positive tweets on dermatologic disorders
Figure 2 displayed the most frequent positive or negative words used in each specific disease. “Pain”, “bad”, and “hard” were used frequently in negative posts about HS and PsO; while “loss” was overwhelmingly presented in negative tweets on AA. “Anxiety”, “fear”, “wrong”, and “burden” were seen in posts on AA but not in posts on HS and PsO. Words expressing negative internal emotions such as “crying” were observed in posts on HS; words connoting external influence like “contagious” and “hate” were more commonly observed in posts on PsO than in posts on HS or AA. “Care” and “natural” were found in positive tweets related to all three diseases. Words describing a supportive system including “help”, “love”, “support”, and “god” were most frequently identified in positive posts on PsO.
FIGURE 2
AFINN lexicon-graded positive and negative words commonly used in tweets about different diseases.
The sentiment of tweets on dermatologic disorders and fibromyalgia
510 tweets about FM were identified. Like skin disorders, “pain”, “bad”, and “hard” were commonly seen in negative posts on FM. Besides, emotional terms used in AA tweets, like “anxiety”, “suffering”, “guilt”, and “sucks”, contributed to a significant portion of negative posts on FM. The average sentiment score was significantly lower for FM (−1.11 ± 33.47) than for the three skin disorders, AA, HS, and PsO (−0.239 ± 2.90, p < 0.0001). Unlike skin disorders, tweets from public (−0.953 ± 3.70) or private (−1.170 ± 3.39) accounts expressed similar negative sentiment (p = 0.5969).
Discussion
Findings of the present study provided an effective and efficient approach to measure sentiments surrounding AA, HS, and PsO via analysis on tweets. Words that have been given negative polarity, like “anxiety”, “pain”, and “crying”, are common in tweets related to AA, HS, and PsO. Sentiment regarding these three skin diseases is slightly polarized to the negative side, with less negative polarity compared to FM.
This study utilizes posts from popular social media to understand sentiments related to dermatologic disorders. The results seem to correlate well with previously documented psychiatric comorbidities. “Anxiety” was the most common emotional word in posts on AA. Patients with AA are particularly susceptible to generalized anxiety disorder (GAD) (28). A systematic review reported a 39–62% lifetime prevalence of GAD in patients with AA, giving an odds ratio (OR) of 7.28 compared to the general population. The ratio was higher than that for major depressive disorder (MDD) (OR = 5.87–6.77), social phobia (OR = 1.59–3.89), and paranoid disorder (OR = 4.4) (28). In contrast, for HS, words like “crying,” as well as aggressive words like “fuck” were commonly seen in tweets on HS, possibly reflecting the prevalence of bipolar disorders and MDD in this population. One meta-analysis on the psychiatric comorbidities of HS concluded that among the investigated psychiatric disorders, bipolar disorders (OR = 1.96) and MDD (OR = 1.75) were the most significantly increased comorbidities in patients with HS. Also in contrast to AA, for posts on PsO, we did not identify “anxiety” nor “bad” in the top 10 negative words. A recent meta-analysis reported a hazard ratio (HR) of 1.29–1.31 for anxiety in patients with PsO; on the contrary, the ratios were slightly lower than those found for AA and HS (29). The same study found that the OR for depression was 1.57 in patients with PsO (29). For comparison with all three skin disorders, tweets on FM were also examined. “Anxiety” and “guilt” were commonly used in negative posts on FM, for which patients with FM display a higher prevalence of GAD (20–80%) and MDD (13–63.8%) (30, 31). Thus, the approach adopted in the present study may be a powerful tool to conceptualize real-time emotional experience of dermatologic disorders, which may be used to predict or reflect their psychiatric comorbidities. Analyzing the psychological foundations of the affective lexicon allows for a better understanding of the emotional impact of diseases from patients’ perspectives and direct psychosocial interventions (32–34). Interestingly, the overall sentiment scores were neutral for the three dermatologic diseases and did not differ from one another. Despite their various health impact, previous studies suggested that the quality of life in dermatologic diseases was the most affected by the severity of diseases rather than the type of diseases (35–37). Our data may support this finding although we were not able to stratify sentiment scores by disease severity.
Besides emotional words, the dataset provided insight into other patient priorities. “Natural” and “care” were recurrent themes in all three diseases, suggesting growing interest in non-pharmacologic options. Words like “contagious” in tweets on PsO hinted at common misconceptions and could guide the development of future campaigns. Finally, a sentiment gap appeared between public and private accounts in tweets about skin disorders but not about FM. While a strong association between FM, depression, and anxiety is widely reported by lay media, many skin diseases were considered largely “cosmetic” and ignored for their emotional impact. Thus, this gap may reflect a failure of physicians and public organizations to identify occult emotional burdens. An empathetic and systematic approach may be beneficial and should be encouraged when caring for patients with dermatologic diseases. Furthermore, a previous study on tweets related to HS concluded that the analysis on social media data allowed the identification of some treatment beliefs not easily detected by traditional surveys (38). Collectively, these findings necessitated the presence of medical professionals and institutions to monitor and validate educational information on social media (39).
Despite continuous data collection for one month, the sample sizes were still small. In addition, we only analyzed one social media platform (i.e., Twitter), and therefore its external validity might be limited. A limitation specifically of Twitter API is the random selection of a number of tweets (set by users) during a period of time (set by users) from the pool of tweets using the specified hashtags/keywords. Twitter does not allow access to all qualified tweets with one search. Second, microbloggings on social media are usually used to express temporary emotions and may not adequately reflect long-term psychological status; and patients may be reluctant to publicly share either negative or positive experiences. Sentiment classification might fail when negation or irony are used. For example, profanity words can be used to modify a positive term, reversing their original polarity (14). Although irony may be indicated by emojis, previous studies did not show a significant improvement in sentiment classification with emoticons (40). Therefore, we did not include emojis in the analysis. Some people may use text embedded in images to trespass the 280-word limitation. These longer posts, which may be more personal, may be missed in the algorithm. Lastly, different sentiment lexicons can result in different results based on individual sensitivity and specificity. SentiStrength is another lexicon commonly used in health-related sentiment analysis (41, 42). That said, since AFINN was shown to have a similar or higher accuracy than SentiStrength, thus was preferred in this study (41).
Conclusion
The use of sentiment analysis on tweets is a promising method that can reflect psychological comorbidities, illness experience, and public perceptions of patients with dermatologic disorders. This technique has the potential to improve patient care by bridging misconceptions and knowledge gaps between patients and medical professionals.
Statements
Data availability statement
In the current study, the dataset regarding the reported results could be accessed via the publicly archived tool of Twitter application programming interface (https://help.twitter.com/en/rules-and-policies/twitter-api).
Author contributions
ITL, SEJ, STC, CK, and KSM contributed to the conceptualization, data analysis, writing, and editing of the manuscript. All authors contributed to the article and approved the submitted version.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Abbreviations
- AA
alopecia areata
- HS
hidradenitis suppurativa
- PsO
psoriasis
- FM
fibromyalgia
- API
application programming interface.
References
1.
YuHYangCCYuPLiuK. Emotion diffusion effect: Negative sentiment COVID-19 tweets of public organizations attract more responses from followers. Patel SKS, ed.PLoS One. (2022) 17:e0264794. 10.1371/journal.pone.0264794
2.
GruzdAMaiP. Going viral: How a single tweet spawned a COVID-19 conspiracy theory on Twitter.Big Data Soc. (2020) 7:205395172093840. 10.1177/2053951720938405
3.
EysenbachG. Infodemiology and infoveillance: Framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet.J Med Internet Res. (2009) 11:e11. 10.2196/jmir.1157
4.
ZunicACorcoranPSpasicI. Sentiment analysis in health and well-being: Systematic review.JMIR Med Inform. (2020) 8:e16023. 10.2196/16023
5.
SignoriniASegreAMPolgreenPM. The use of twitter to track levels of disease activity and public concern in the U.S. during the influenza A H1N1 pandemic.PLoS One. (2011) 6:e19467. 10.1371/journal.pone.0019467
6.
AramakiEMaskawaSMoritaM. Twitter catches the flu: Detecting influenza epidemics using twitter.Proceedings of the 2011 conference on empirical methods in natural language processing. (Stroudsburg, PA: Association for Computational Linguistics) (2011). p. 1568–76. 10.1080/07448481.2021.1973480
7.
MaKSIntegrating travel history via big data analytics under universal healthcare framework for disease control and prevention in the COVID-19 pandemic. J Clin Epidemiol. (2021) 130:147–8. 10.1016/j.jclinepi.2020.08.016
8.
MaKSTsaiSYBig data-driven personal protective equipment stockpiling framework under universal healthcare for disease control and prevention in the COVID-19 Era. Int J Surg. (2020) 79:290–1. 10.1016/j.ijsu.2020.05.091
9.
HansonCLCannonBBurtonSGiraud-CarrierC. An exploration of social circles and prescription drug abuse through twitter.J Med Internet Res. (2013) 15:e189. 10.2196/jmir.2741
10.
MaKSWeiJCChungWHCorrespondence to ‘Hypersensitivity reactions with allopurinol and febuxostat: a study using the Medicare claims data’. Ann Rheum Dis. (2022) 81:e107. 10.1136/annrheumdis-2020-218090
11.
MaKSChungWHHsuehYJChenSYTokunagaKKinoshitaSet alHuman leucocyte antigen association of patients with Stevens-Johnson syndrome/toxic epidermal necrolysis with severe ocular complications in Han Chinese. Br J Ophthalmol. (2022) 106:610–5. 10.1136/bjophthalmol-2020-317105
12.
ChiangCHChiangCHMaKSHsiaYPLeeYWWuHRet alThe incidence and risk of cardiovascular events associated with immune checkpoint inhibitors in Asian populations. Jpn J Clin Oncol. (2022) hyac150. 10.1093/jjco/hyac150
13.
MaKSSaeedHNChodoshJWangCWChungYCWeiLCet alOcular manifestations of anti-neoplastic immune checkpoint inhibitor-associated Stevens-Johnson syndrome/toxic epidermal necrolysis in cancer patients. Ocul Surf. (2021) 22:47–50. 10.1016/j.jtos.2021.06.010
14.
RamagopalanSWasiakRCoxAP. Using Twitter to investigate opinions about multiple sclerosis treatments: A descriptive, exploratory study.F1000Research. (2014) 3:216. 10.12688/f1000research.5263.1
15.
JiXChunSAWeiZGellerJ. Twitter sentiment classification for measuring public health concerns.Soc Netw Anal Min. (2015) 5:13. 10.1007/s13278-015-0253-5
16.
PaulMJDredzeM. Discovering health topics in social media using topic models. Lambiotte R, ed.PLoS One. (2014) 9:e103408. 10.1371/journal.pone.0103408
17.
McIverDJHawkinsJBChunaraRChatterjeeAKBhandariAFitzgeraldTPet alCharacterizing sleep issues using twitter.J Med Internet Res. (2015) 17:e140. 10.2196/jmir.4476
18.
GreavesFRamirez-CanoDMillettCDarziADonaldsonL. Use of sentiment analysis for capturing patient experience from free-text comments posted online.J Med Internet Res. (2013) 15:e239. 10.2196/jmir.2721
19.
NathCHuhJAdupaAKJonnalagaddaSR. Website sharing in online health communities: A descriptive analysis.J Med Internet Res. (2016) 18:e11. 10.2196/jmir.5237
20.
GabarronEDorronzoroERivera-RomeroOWynnR. Diabetes on twitter: A sentiment analysis.J Diabetes Sci Technol. (2019) 13:439–44. 10.1177/1932296818811679
21.
NelsonPABarkerZGriffithsCECordingleyLChew-GrahamCA. ‘On the surface’: A qualitative study of GPs’ and patients’ perspectives on psoriasis.BMC Fam Pract. (2013) 14:158. 10.1186/1471-2296-14-158
22.
NelsonPAChew-GrahamCAGriffithsCEMCordingleyLImpact Team.Recognition of need in health care consultations: A qualitative study of people with psoriasis.Br J Dermatol. (2013) 168:354–61. 10.1111/j.1365-2133.2012.11217.x
23.
WuKJTuCCHuJXChuPHMaKSChiuHYet alSeverity of periodontitis and salivary interleukin-1β are associated with psoriasis involvement. J Formos Med Assoc. (2022) 121:1908–16. 10.1016/j.jfma.2022.01.017
24.
WuMCMaKSChenHHHuangJYWeiJCRelationship between Helicobacter pylori infection and psoriasis: a nationwide population-based longitudinal cohort study. Medicine (Baltimore). (2020) 99:e20632. 10.1097/MD.0000000000020632
25.
HuangJWKuoCLWangLTMaKSHuangWYLiuFCet alCase report: In situ vaccination by autologous CD16+ dendritic cells and anti-PD-L 1 antibody synergized with radiotherapy to boost T cells-mediated antitumor efficacy in a psoriatic patient with cutaneous squamous cell carcinoma. Front Immunol. (2021) 12:752563. 10.3389/fimmu.2021.752563
26.
MaKSLaiJNVeeravalliJJChiuLTVan DykeTEWeiJCFibromyalgia and periodontitis: bidirectional associations in population-based 15-year retrospective cohorts. J Periodontol. (2022) 93:877–87. 10.1002/JPER.21-0256
27.
SilgeJRobinsonD. tidytext: Text mining and analysis using tidy data principles in R.J Open Source Softw. (2016) 1:37. 10.21105/joss.00037
28.
MirzoyevSASchrumAGDavisMDPTorgersonRR. Lifetime incidence risk of alopecia areata estimated at 2.1% by rochester epidemiology project, 1990–2009.J Invest Dermatol. (2014) 134:1141–2. 10.1038/jid.2013.464
29.
HedemannTLLiuXKangCNHusainMI. Associations between psoriasis and mental illness: An update for clinicians.Gen Hosp Psychiatry. (2022) 75:30–7. 10.1016/j.genhosppsych.2022.01.006
30.
FiettaPFiettaPManganelliP. Fibromyalgia and psychiatric disorders.Acta Bio-Medica Atenei Parm. (2007) 78:88–95.
31.
UçarMSarpÜKaraaslanÖGülAITanikNArikHO. Health anxiety and depression in patients with fibromyalgia syndrome.J Int Med Res. (2015) 43:679–85. 10.1177/0300060515587578
32.
OrtonyACloreGLFossMA. The referential structure of the affective lexicon.Cogn Sci. (1987) 11:341–64. 10.1207/s15516709cog1103_4
33.
CloreGLOrtonyAFossMA. The psychological foundations of the affective lexicon.J Pers Soc Psychol. (1987) 53:751–66. 10.1037/0022-3514.53.4.751
34.
IvenJGoudbeekMBKoolenRMF.The effect of emotional valence on word choice: How the valence levels of amusement and disgust influence word choice in spontaneous dialogues. (2017). *cp.
35.
AbediniRHallajiZLajevardiVNasimiMKarimi KhalediMTohidinikHR. Quality of life in mild and severe alopecia areata patients.Int J Womens Dermatol. (2018) 4:91–4. 10.1016/j.ijwd.2017.07.001
36.
SendrasoaFARazanakotoNHRatovonjanaharyVRaharolahyORanaivoIMAndrianarisonMet alQuality of life in patients with psoriasis seen in the department of dermatology, antananarivo, madagascar.BioMed Res Int. (2020) 2020:1–5. 10.1155/2020/9292163
37.
Schneider-BurrusSTsaousiABarbusSHuss-MarpJWitteKWolkKet alFeatures associated with quality of life impairment in hidradenitis suppurativa patients.Front Med. (2021) 8:676241. 10.3389/fmed.2021.676241
38.
WortsmanXRamírez−CornejoCFerreira−WortsmanCHowesR. Perspectives on the twitter presence of hidradenitis suppurativa.Int J Dermatol. (2021) 60:e402–3. 10.1111/ijd.15631
39.
MaKSChangHCKrupatETeaching evidence-based medicine with electronic databases for preclinical education. Adv Physiol Educ. (2021) 45:849–55. 10.1152/advan.00057.2021
40.
LeCompteTChenJ. Sentiment analysis of tweets including Emoji data.Proceeding of the 2017 international conference on computational science and computational intelligence (CSCI). (Piscataway, NJ: IEEE) (2017). p. 793–8. 10.1109/CSCI.2017.137
41.
GohilSVuikSDarziA. Sentiment analysis of health care tweets: Review of the methods used.JMIR Public Health Surveill. (2018) 4:e43. 10.2196/publichealth.5789
42.
Al-ShabiM. Evaluating the performance of the most important Lexicons used to Sentiment analysis and opinions Mining.IJCSNS. (2020) 20:1. 10.1155/2022/7612276
Summary
Keywords
sentiment analysis, Twitter, alopecia areata, hidradenitis suppurativa, psoriasis, mental health
Citation
Lee IT-L, Juang S-E, Chen ST, Ko C and Ma KS-K (2022) Sentiment analysis of tweets on alopecia areata, hidradenitis suppurativa, and psoriasis: Revealing the patient experience. Front. Med. 9:996378. doi: 10.3389/fmed.2022.996378
Received
19 July 2022
Accepted
26 September 2022
Published
31 October 2022
Volume
9 - 2022
Edited by
Ruano Juan, Hospital Universitario Reina Sofía, Spain
Reviewed by
Dillon Mintoff, University of Malta, Malta; Ilya Mukovozov, University of British Columbia, Canada
Updates
Copyright
© 2022 Lee, Juang, Chen, Ko and Ma.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Irene Tai-Lin Lee, heyttymonica@gmail.comKevin Sheng-Kai Ma, kevinshengkaima@g.harvard.edu
†These authors have contributed equally to this work and share first authorship
‡ORCID: Irene Tai-Lin Lee, orcid.org/0000-0001-6589-8403; Christine Ko, orcid.org/0000-0003-2270-2524; Kevin Sheng-Kai Ma, orcid.org/0000-0002-9394-4144
This article was submitted to Dermatology, a section of the journal Frontiers in Medicine
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.