eHealth literacy is a skill set that is becoming increasingly relevant as the Digital Age progresses (1, 2). When healthcare and public health services slowly moved to the digital sphere (3), the skills required to search for, identify, and access these services started changing incrementally (4). However, much like most of the world, these services transitioned to digital environments at the beginning of the COVID-19 pandemic (5). While innovations are typically taken up gradually by a population (6), society adapted acutely to the pandemic—resulting in digital and technological advances neither equally nor equitably permeating all layers of society across the globe and potentially widening existing inequalities (7–9). In fact, there are some indications that the migration to the digital environment was used as a short-term solution and that—when traditional health services reemerged—these were again preferred by those seeking healthcare (10).

eHealth literacy is a specific branch in the field of digital literacy (4). Given the transition toward digitalization globally, there have been attempts to define eHealth literacy. For instance, UNESCO frames digital literacy as a set of foundational skills needed to work in the digital world and is considered a catalyst for individuals to achieve health and social outcomes (11). Current training programs tend to focus heavily on technical skills required to use digital technologies, while cognitive and ethical considerations are disregarded entirely (12, 13). It takes ~5 years for the average individual to become sufficiently proficient in using digital technologies (12). However, this 5-year gap is not equal across all societal domains. For example, the medical sector has been slower to adapt to the digital world (14).

Recent work by Paige and colleagues created a tool with which to measure eHealth literacy (15). The Transactional eHealth Literacy Instrument (TeHLI) is a multi-dimensional instrument to measure increasing levels of eHealth literacy, based on the Transactional Model of eHealth Literacy (TMeHL) (16): functional (the ability to successfully read and write about health using technological devices), communicative (the ability to control, adapt, and collaborate communication about health with others in online social environments), critical (the ability to evaluate the relevance, trustworthiness, and risks of sharing and receiving health-related information on the Internet), and translational eHealth literacy (the ability to apply health-related information from the Internet in different contexts). However, given the novelty of this assessment tool, its application has only been focused on the general and middle age populations. However, as the driver population of eHealth utilization is the teenager and working population, more research is needed to test TeHLI on younger adults (15). This article aims to validate the TeHLI to see which components of the tool (how many components and which components included) would be the best fit statistically and whether the tool can be applied to groups of different characteristics (i.e., whether the different characteristics in terms of gender, education level, age group and other sociodemographic factors impact the effectiveness of the tool in any way) through exploratory and confirmatory factor analyses of survey data from Vietnamese young adults.

Table 2 shows the sociodemographic of the 236 participants in this study. The mean age of respondents was 21.3 ± 4.8 years, with the most common age group about above 24 years old (78.8%). The majority of those were female (71.2%) and 56.8% of participants had the education tertiary and upper.

Table 3 presents the results of the descriptive analysis for 18 items of the TeHLI. All the 18 items had a range of scores from 1 to 5. Out of 18 items, the maximum and mimimum scores was recored at 3.3 (0.9), and 3.3 (0.9), respectively. Skewness and Kurtosis coefficients were reported with ranging from −1.0 to −0.2 and 2.8 to 4.3, respectively. Table 3 also presented the reliability of the modified TeHLI. Correlation coefficients with other items in respective factors ranged between 0.7 and 0.8. Cronbach's alpha value of factor 1 “Functional,” factor 2 “Communicative,” factor 3 “Critical,” and factor 4 “Translational” were good at 0.82, 0.92, 0.92, and 0.90, respectively.

Table 4 further shows the results of Confirmatory Factor Analysis on four different models. The model with 1 factor of the TeHLI showed an RMSEA and CFI score of 0.157 and 0.806. In contrast, the model with four factors showed an RMSEA and CFI score of 0.116 and 0.907, respectively. As such, this model was used to conduct measurement invariance.

The results obtained from the Confirmatory Factor Analysis for the TeHLI are estimated in Figure 1. The standardized coefficients range from 0.87 to 1.1. The highest coefficient was found in 4 items (Q4, Q11, Q13, and Q17), while the lowest was Q8.

The values of ΔCFI were <0.01 when all models, with progressive restrictions, are compared across age groups, education, household economic (Table 5). These groups reported strict invariance. Besides, the configural model of gender, marital status, and location had fit statistics of RMSEA < 0.08, CFI > 0.9. However, in the metric model, the value of ΔCFI was higher than 0.01 at all three variables (gender, marital status, and location). Metric invariance was therefore not supported by the data.

This article aimed to validate the TeHLI to see which components of the tool (how many and which components included) would be the best fit statistically and whether the tool applies to groups of different characteristics. Overall, we found that a TeHLI divided into four factors (functional, communicative, critical, and translational) had the best statistical fit (RMSEA = 0.116; CFI = 0.907) and the highest level of internal consistency (Cronbach's α = 0.96). The individual items in the TeHLI have medium-to-high levels of correlation with other items in respective factors (r > 0.6). Upon further analysis, we found partial invariance of the TeHLI—only lacking support for metric invariance.

The study findings substantiate that the most optimal composition of the TeHLI consists of four factors: functional, communicative, critical, and translational eHealth literacy. In this model, Q1–Q4 measure functional eHealth literacy, Q5–Q9 measure communicative eHealth literacy, Q10–Q14 measure critical eHealth literacy, and Q15–Q18 measure transactional eHealth literacy. This tool composition is in line with the original dimensions of the TMeHL (16). The initial measurements of high internal consistency of the four domains by Paige and colleagues (Cronbach's α = 0.91, 0.92, 0.88, and 0.92 respectively) was also supported by our findings (Cronbach's α = 0.88, 0.92, 0.92, and 0.90 respectively) (15).

In terms of measurement invariance, the support for configural invariance indicates that the same construct is measured in different population groups. In the context of this study, functional, communicative, critical, and translational eHealth literacy are consistently measured when the TeHLI is deployed in different population groups. The support for scalar invariance indicates the ability to justify the comparison of means across population groups. It indicates that gender, education, marital status, age, location, and household economy do not influence the way participants to respond to the TeHLI to the point that would introduce measurement bias. The support for strict invariance indicates that the residual errors are equivalent across participants. In other words, using the TeHLI across population groups should not produce error margins that substantially differ from each other. However, metric invariance was only found in two groups (education [ΔCFI = 0.002] and household economy [ΔCFI = 0.005]; age group being precisely on the threshold [ΔCFI = 0.010]) and, therefore, only partial metric invariance is supported by this data (23), which suggests that the constructs measured have different meanings across some of the participant groups.

Finding partial support for metric invariance is reported to have minimal effects on the mean differences of a latent factor (24). Since two factor loadings reported metric invariance, there are grounds to assume the TeHLI can be used for comparisons across groups (25, 26). Simultaneously, there is also the question of whether partial invariance is sufficient as significant bias can still be introduced into study findings if the partial invariance is ignored (27, 28). As such, deploying a partially (including invariant items only) and a fully invariant model (including invariant and non-invariant items) and compare the results of interest arises as a viable solution to use the TeHLI across different population groups (29). The risk of this solution is that—if the results differ substantially—there is no clear path forward.

Some limitations apply to this study. The survey findings had a mild-to-moderate negative skewness. However, as extrapolation to the population is impossible given the lack of participants, this limitation does not significantly affect the study findings. The survey was also distributed only among the Vietnamese population. Therefore, the influence of culture could not be determined and needs to be examined in a future study that includes multiple countries or regions. Lastly, the current study sample did not reflect a balanced distribution of some socio-demographic characteristics and could lead to limit representativeness with respect to the Vietnamese youth population. Hence, further studies are required to explore the effect of the data imbalance on measurement invariances of the scales.

Ultimately, the TeHLI can be considered a valuable tool to measure different competency levels of eHealth literacy. As such, the TeHLI has the potential (if deployed correctly) to generate high-quality data that can be used to inform governments and change agents whether the general or a target population is capable of understanding, appraising, distinguishing, and acting appropriately on high- and low-quality health information on the Internet. As a result, it can make vital contributions in informing and guiding policy and practice sustainably into the Digital Age.

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

The protocol of this study was approved by the Institutional Review Board of Youth Research Institute, Vietnam (No 404QÐ*/TWÐ*TN-VCNTN) on 08/01/2020. The patients/participants provided their written informed consent to participate in this study.

TPTN: conceptualization, data collection, project management, and writing—review and editing. AD and HD: conceptualization, data collection, and project management. TV and RK: formal analysis, validation, writing—original draft, and writing—review and editing. BLHW, LB, GF, and PA: formal analysis, writing—original draft, and writing—review and editing. TTN: formal analysis, data collection, data analysis, and writing—original draft. CL and CH: conceptualization and writing—review and editing. RH: conceptualization, validation, and writing—review and editing. All authors contributed to the article and approved the submitted version.