Empirical Evaluation of Understandability and Usability of Health Handbooks Commonly Used in Japan

Objective: Various types of handbooks that summarize and record health information (health handbooks) have been used in Japan for many years. The purpose of this study was to explore ways to evaluate the understandability and usability of commonly used printed health handbooks in Japan. Methods: An internet search was performed to identify health handbooks used in Japan. The handbooks were then collected, searched, and evaluated for the quality and functionality of the health-information. The overall understandability was assessed using the Suitability Assessment of Material (SAM) and the Centers for Disease Control and Prevention Clear Communication Index (CCI), and overall usability was assessed using a purpose-user matrix. Results: A total of 14 health handbooks were extracted and analyzed. The median scores for SAM and CCI were 39 (min 25, max 45) and 81.5% (60%, 100%), respectively, and no significant association was observed between the two evaluation scores. Based on the matrix, the most common type of handbook was those designed for preventive purposes that the user or their family completed (n = 9). Conclusion: Our sampled health handbooks were used mostly for preventive purposes and their understandability varied. SAM and CCI assess different aspects of written materials and it is recommended they be used together when evaluating the understandability of health handbooks. To facilitate more effective use of health handbooks in public health activities, we suggest the content of handbooks be assessed by a purpose-user matrix.


INTRODUCTION
Japan is experiencing rapid aging of the population, and community-based care is required to effectively utilize limited health and medical resources (Cabinet Office, Government of Japan, 2017). In other words, self-management of one's health is required more than ever to promote healthy aging, maintain physical function, and manage chronic illness. Under the community-based integrated care system, coordination of not only hospital outpatient and inpatient units, but also welfare facilities, home-visit care services, and even mutual support activities among neighborhoods, is required throughout Japan (Arai et al., 2015;Hatano et al., 2017). In order to connect health personnel with patients and their families, there is an increasing need for patients and professionals to share and interactively use health information materials.
One source of multi-dimensional health information material that has been used in Japan for a long time is health handbooks that summarize health information in the form of brochures. For instance, the mother and child handbook (MCH handbook), which records the health information of mothers and children, is widely used in Japan. Health-related data is recorded by both the mother and the physician or care provider. MCH handbooks are provided at the time of pregnancy registration and are used mainly until the child enters school at the age of six. The health handbook serves as interactive educational material useful for improving communication between care providers and patients, and ultimately to achieve a maternal, newborn, and child health continuum of care (Takayanagi et al., 1993;Takeuchi et al., 2016). Internationally, the World Health Organization (WHO) newly published the first edition of WHO Recommendations on Home-based Records for Maternal, Newborn and Child Health (MNCH) in 2018, following the practical guide on Home-based Records in Immunization announced in 2015. The Japanese MCH handbook is listed as an example of home-based records and it has become known globally as a useful method of recording health services that an individual receives (World Health Organization, 2018a). Recent reviews of both qualitative and quantitative studies reported that the MNCH home-based records increased providers' feeling of connectedness with their clients, and improved antenatal care and monitoring of child immunizations and growth, thus reducing risks of pregnancy complications and cognitive delay in the child (Magwood et al., 2018;Magwood et al., 2019).
In addition to the MCH handbook, the "pharmacy handbook" is also commonly used in Japan. The pharmacy handbook was originally created with the objective of unified management of information on the patients' medication history, such as prescription, purchasing of general-purpose medicines, medical history, side effect, and allergy. Some have recommended increasing its use to improve medical safety through the tracking of care and prescriptions to avoid overlaps in treatments or medications (Wiedenmayer et al., 2006;Takamatsu et al., 2016;Tachi et al., 2018). In the United Kingdom, a similar tool, called "My Medication Passport," has been reported to serve as a platform for communication with healthcare providers and for sharing information with family (Barber et al., 2014). In addition, having a pharmacy handbook is an important source of medical information in the event of a natural disaster.
Considering that as much as 85% of Japanese people are reported to have limited health literacy, as compared to the reported proportion of 48% in Europe (Nakayama et al., 2015), health information needs to be easy to understand. Promotion of health literacy is a key concept in improving the use of health information among community residents or patients, as well as in assisting health care providers to better convey health information (Koh and Rudd, 2015). Health literacy is reportedly strongly linked with better patient outcomes, such as prognosis and management of health conditions (DeWalt and Hink, 2009;Berkman et al., 2011). Especially for older adults, receiving health information in a simple yet informative manner is important for health promotion (Andrus and Roth, 2002;Safeer and Keenan, 2005).
Various evaluation scales have been developed to evaluate and improve the understandability of written health materials, including the Suitability Assessment of Material (SAM) (Doak et al., 1985) and the Centers for Disease Control and Prevention (CDC) Clear Communication Index (CCI) (Baur and Prue, 2014). Many studies have used these indices to analyze patient educational materials focusing on specific diseases, such as rheumatic disease, cancers, and chronic suppurative otitis media (Rhee et al., 2013;Sakai, 2013;Okuhara et al., 2015). However, research regarding Japanese health information materials is limited and has been conducted in the local language (Noro, 2009;Sakai, 2011;Goto et al., 2018). More work is, therefore, needed to objectively analyze Japanese health handbooks using well-defined methods.
This study explored ways to evaluate understandability and usability, along with readability, of commonly used printed health handbooks in Japan.

MATERIALS AND METHODS
To ascertain the current status of health handbooks available in Japan, a Google search was conducted in the Japanese language using the word "handbook" in combination with other terms such as "health," "medicine," "lifestyle-related diseases," "elderly persons," and "home-based care." Health handbooks that appeared on the first five pages presented by the search engine were extracted. The information extracted was summarized in a table, including the title, publisher, format [i.e., paper, portable document format (PDF), etc.], and the method required to acquire the material (download, telephone request, etc.). Health handbooks presented in PDF format were downloaded, and publishers were contacted by telephone to request copies of materials in paper format.
The term "usability" is often used when evaluating digital information and refers to the degree to which users can utilize the product effectively (Walsh et al., 2017). In this paper, by referring to the introductory questions of the CCI, we clarified the target audience and the intended aim of the handbook to determine "usability." Further, "readability" refers to the assessment of words and sentences to measure how easy a material is to read (Brega et al., 2015). "Understandability" measures not only words and sentences, but also the layout and content (Brega et al., 2015).
First, the extracted health handbooks were assessed based on their layout (size and number of pages). Then, we assessed their contents (target audience, purpose, space for making entries, and information delivered to the target user). This evaluation framework was discussed and agreed upon by the study team by referring to the major components in SAM and CCI. Ratings were made as follows: "2" sufficient, "1" exists but insufficient, and "0" no such function exists, or, if it exists, the purpose is not being fulfilled. Additionally, a matrix was developed using purpose (disease prevention and disease management) and target users [patients or persons to whom the booklet was issued (handbook user), family members, healthcare professionals, etc.] in order to visually locate each health handbook within this purpose-user matrix.
Finally, two researchers evaluated the understandability using the following two tools: the Japanese version of the SAM (Doak et al., 1985) and the Japanese version of the CCI (Baur and Prue, 2014). Any discrepancies were discussed to reach consensus. As for readability assessment, we used a free software "Chuta's Toolbox" (a Japanese reading tutor) (Kawamura et al., 1997). The evaluation results were compared and matched after discussing discrepancies.
The SAM offers a systematic method for objectively evaluating the ease of understanding of a document in its entirety. The Japanese version of the SAM translated and adapted by Noro covers a total of 23 items that are designed to evaluate a document's contents, literacy demand, graphics, and cognitive and emotional considerations for its readers (Noro, 2009). Points (0, 1, 2) are assigned for each evaluation item. The maximum total score is 46, with higher scores indicating that the entire document is more understandable. Of note, the original (English) SAM scores expressed as percentages are interpreted as follows: 70-100% is considered as superior, 40-69% as adequate, and 0-39% as not suitable material.
The CCI is an index for determining effective communication via written materials developed by the CDC in the USA (Baur and Prue, 2014). The Japanese version of the CCI translated and developed by Goto and colleagues comprises 10 questions related to content, words, design, and scientific aspects (Goto et al., 2018). There are three index questions concerning written materials designed to change the behavior of communication targets, three index questions concerning materials that use numbers, and three index questions concerning materials explaining risks. Points (0, 1) are assigned for each evaluation item. A total point earned was divided by the total possible points that the material could have earned and multiplied by 100. The maximum score is 100%, with a higher score indicating that the document is more understandable. CDC's CCI manual recommends a score of 90% or higher is desirable.
Chuta's Toolbox software consists of the following four tools for studying the Japanese language: 1) a Japanese-to-Japanese dictionary, 2) a Japanese-to-English dictionary, 3) a vocabulary term checker for determining the difficulty of a vocabulary term using a standard Japanese-Language Proficiency Test, and 4) a kanji (Chinese character) checker (Kawamura et al., 1997). The vocabulary term checker was used in this study, with extractions from the CCI evaluation item "main message," using a five-point scale ranging from "very easy" ( 1) to "hard" ( 5).
Statistical analysis to assess an association between SAM and CCI was performed using SPSS (IBM SPSS Statistics for Windows, Version 24.0 Released 2016; IBM Corp: Armonk, NY). This analysis did not involve the use of any animal or human data; therefore, ethical approval from a review committee was not applicable for this study.

RESULTS
A total of 14 health handbooks were extracted and analyzed. Health handbooks were assigned a number from 1 to 14 ( Table 1). Most of the health handbooks analyzed were evaluated as satisfying each of the items of a health handbook's function (i.e., purpose, space for writing, and information that must be recorded within the health handbook). Regarding their size, only one health handbook was palm-sized, and most were either size A6 or A5. As for the format, some were divided into three separate booklets in accordance with the purpose (i.e., for basic information, for recording, and for advanced information). Almost all of the information in the health handbooks was "sufficient" or "exists but insufficient," but "no such function exists" was also seen.
As a result of the usability classification by the matrix, there were nine items designed for preventive purposes that the user or their families filled in; this comprised the largest number within a classification (Figure 1). There was an MCH handbook (No. 13) that was classified as preventive; however, it was meant to be completed by a person other than the user. There were health handbooks designed for management of a specific disease; one was a handbook completed by the users themselves (No. 10), and another was completed by other persons (No. 14).
Understandability of each health handbook was examined in detail so as to ascertain point-based scores ( Table 2). For SAM, the minimum score was 25 (54%) and the maximum was 45 (98%), with a median of 39 (85%). For CCI, the minimum score was 69% and the maximum 100%, with a median of 81.5%. For Chuta's Toolbox vocabulary term checker (readability assessment), the minimum was 1 and the maximum 5, with a median of 2.5. No significant association was found for the total scores obtained using the two different evaluation tools (r2 0.0082, p 0.738) (Figure 2).

DISCUSSION
In our matrix classification, we were able to visualize and clarify the characteristics of each health handbook; nine out of the 14 health handbooks aimed at prevention and targeted patients themselves or their families. As for the understandability and readability of these materials, the scores of SAM, CCI, and Chuta's Toolbox varied. In our previous assessment of five printed materials on breast feeding (Goto et al., 2018), SAM score in percentage ranged from 59 to 84% with a median score of 70%, and CCI score ranged from 40 to 80% with a median of 67%. Although the scores of the handbooks evaluated in the present study are slightly higher, considering that the recommended CCI score is a minimum of 90%, both results indicate the need for improvements. Since health information itself has the same function as medicine or medical treatment, it is necessary to carefully assess and improve the information before it is delivered (Tomlinson, 1986). Furthermore, we confirmed that no association was observed between the total scores of each of these respective tests, which implies that SAM and CCI assess different aspects of written materials. Both measures assess document content (clarification of main messages) and the understandability of the text and graphics, but SAM has more items about the layout and emotional considerations whereas CCI has more items about scientific data. Hence, our results suggest that use of SAM and CCI in combination will improve assessment, and in turn, improve the quality of health information material. In addition, when assessing health handbooks, we suggest   including the purpose, space for making entries, and information that must be recorded within the health handbook. While home-based records have been widely implemented for decades, evidence of their benefits has not previously been investigated. The WHO recently published a new guideline that addresses the gap between a wide use of home-based records and evidence of its benefits and harms. They stated that there was "insufficient evidence available to determine if any specific type, format or design of home-based records is more effective." The WHO guideline also indicated that policy makers should involve stakeholders when discussing the important considerations with respect to type, content, and implementation of home-based records (World Health Organization, 2018b). To make daily home-based records more effective, further analysis that applies an evaluation framework such as the present analysis is necessary.
Historically in Japan, the practice of prescription medication started in 1874, and maintaining records was mandated in 1909 by the Medical Practitioners' Act mainly for the purpose of guaranteeing the quality of services provided. Later, in 1947, the Ministry of Health and Welfare introduced and distributed the handbook for mothers and children. Similar to prescription records, the handbook was used as a tool to improve the registration of pregnancies and assess the health of newborns, along with the provision of food rations. It was then improved and named the Maternal and Child Health Handbook and served as a health promotion tool. The new version provided medical information, such as on antenatal care, child delivery processes, and immunization to the user, in order to improve the ownership of mothers about their own and children's health, and to facilitate communication between mothers and health care providers (Nakamura, 2010). Over time, there has been an important paradigm shift around medical records from provider-centered health care management to user-centered health promotion.
Given this background, the usefulness of home-based records has been widely accepted by Japanese citizens. Today, the Maternal and Child Health Handbook, in addition to various health handbooks, is used for disease management in medical institutions and communities. One recent study reported that a combination of individual nutrition guidance and blood pressure monitoring using a handbook was useful for controlling hypertension (Ohira et al., 2016). Another reported that a digitalized handbook and sharing of information from this handbook at a hospital was effective in managing diabetes (Hayashi et al., 2015). However, as recommended by the WHO, increased rigorous and scientific evaluation of handbook usage in Japan is necessary.
Our study presents a framework to classify handbooks as a basis of such evaluation, although it is not without limitations. Firstly, the number of cases that were analyzed was limited, and a more extensive search might be helpful. Health handbooks are issued by municipalities and organizations nationwide at their own discretion, so it is sometimes difficult to obtain these health handbooks. After health handbooks were identified online in the present study, the research team requested the cooperation of each municipality and organization that published the health handbooks. With increased access to digital materials in future years, a larger study could be more easily conducted, thereby enabling more detailed investigation of the differences among available indices (e.g., SAM and CCI) and further detailed evaluation of handbooks, including stratification by content. Secondly, a health handbook should be carried daily, used by patients and providers, and used over the long term, ideally for years; as such, handbook-specific usability is also an important factor in addition to understandability. Prior research has suggested that arranging verbal explanations in addition to easy-to-understand printed materials is associated with appropriate health behaviors (DeWalt and Hink, 2009). Health handbooks with health information material are useful tools to connect health service users and providers to help improve bilateral health communication. Further research is needed to investigate how such handbooks are used in various health and medical communication settings.

CONCLUSION
Our usability matrix demonstrated that the sampled health handbooks in Japan were used not only for disease management, but also for preventive purposes. In order to assess the understandability of such materials, it is recommended that SAM and CCI be used together to better understand the quality and function of health information from different viewpoints. For more effective use of a health handbook in public health activities, the handbook-specific usability also needs to be considered.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

AUTHOR CONTRIBUTIONS
MN and AG contributed to the study design and writing of the first draft. MN collected and analyzed data. EO and HM supervised the study and took part in revision of the paper.