Estimating the Healthiness of Internet Recipes: A Cross-sectional Study

A government’s response to increasing incidence of lifestyle-related illnesses, such as obesity, has been to encourage people to cook for themselves. The healthiness of home cooking will, nevertheless, depend on what people cook and how they cook it. In this article, one common source of cooking inspiration—Internet-sourced recipes—is investigated in depth. The energy and macronutrient content of 5,237 main meal recipes from the food website Allrecipes.com are compared with those of 100 main meal recipes from five bestselling cookery books from popular celebrity chefs and 100 ready meals from the three leading UK supermarkets. The comparison is made using nutritional guidelines published by the World Health Organization and the UK Food Standards Agency. The main conclusions drawn from our analyses are that Internet recipes sourced from Allrecipes.com are less healthy than TV chef recipes and ready meals from leading UK supermarkets. Only 6 out of 5,237 Internet recipes fully complied with the WHO recommendations. Internet recipes were more likely to meet the WHO guidelines for protein than other classes of meal (10.88 v 7% (TV), p < 0.01; 10.86 v 9% (ready), p < 0.01). However, the Internet recipes were less likely to meet the criteria for fat (14.28 v 24 (TV) v 37% (ready); p < 0.01), saturated fat (25.05 v 33 (TV) v 34% (ready); p < 0.01), and fiber (compared to ready meals 16.50 v 56%; p < 0.01). More Internet recipes met the criteria for sodium density than ready meals (19.63 v 4%; p < 0.01), but fewer than the TV chef meals (19.32 v 36%; p < 0.01). For sugar, no differences between Internet recipes and TV chef recipes were observed (81.1 v 81% (TV); p = 0.86), although Internet recipes were less likely to meet the sugar criteria than ready meals (81.1 v 83% (ready); p < 0.01). Repeating the analyses for each year of available data shows that the results are very stable over time.

One of the major contributing factors to such illnesses is poor dietary habits, in particular diets high in sugar, carbohydrates, and fat and diets low in fiber (4)(5)(6). Considerable research attention and practical intervention measures have been taken in an attempt to improve dietary choices. One such measure has been to encourage people to cook for themselves at home. For example, both the US Government's ChooseMyPlate initiative (7) and the UK Government's Change4Life program (8) advocate home cooking. Poorer cooking skills, less-frequent preparation of home-cooked food, and more frequent consumption of pre-prepared foods have been associated with restricted quality of diet and obesity (9)(10)(11). Further evidence supporting the endorsement of home cooking comes from research demonstrating that the amount of food consumed away from home is linked with higher Body Mass Index and lower fruit and vegetable intake (12), leading the authors of one study to conclude that strategies are needed to encourage more cooking among the general population (12). The healthiness of home cooking will, nevertheless, depend on what people cook and how they cook it. These two variables correlated with sociodemographic factors and obesity (13). Thus, the solution may be more complicated than simply getting people to cook-we need to understand sources of cooking inspiration and the role these play in the dietary decisions people make.
Howard et al. (14) investigated the nutritional properties of two common sources of food. They compared recipes published by well-known UK celebrity chefs with leading UK supermarket ready meals, which are often presented as being unhealthy (7,8). Ready meals, more commonly known as "TV dinners" in the United States, are pre-prepared main courses that can be reheated in their container, require no further ingredients, and need only minimal preparation before consumption. Surprisingly, Howard et al. (14) found that in some respects the ready meals were healthier than recipes. More ready meals than recipes met the WHO goals for fiber density and percentage of energy derived from carbohydrate and fat, although more ready meals than recipes exceeded the recommended sodium density. Thus, the source and, in particular, the content of the recipe seem to be more important than the type of meal (i.e., whether it is a ready meal or home cooked).
A further common source of cooking inspiration is the Internet (15). The food website, which, at the time of writing, claims to be the world's largest food-focused social network, is Allrecipes.com. The site has a community of 40 million home cooks accessing 3 billion pages annually across 19 sites in 24 countries with recipes available in 13 languages (16). The British version of Allrecipes.com was named as the Daily Mail's top pick for healthy eating websites, highlighting the "sophisticated search engine" and claiming that "… diabetics, coeliacs and even those specifically wanting to increase their fibre intake -are all catered for" (17).
The recommendation of a popular tabloid newspaper combined with government and media encouragement may persuade members of the public that cooking recipes sourced from the Internet is an approach likely to improve their diet, despite this no systematic study having comprehensively assessed the nutritional content of online recipes.
Using methods previously applied to evaluate the healthiness of ready meals and recipes published by celebrity chefs (14), we analyze the energy, protein, carbohydrate, fat, sugar, fiber, and salt content of recipes uploaded to Allrecipes.com and determine whether the nutritional content complied with national and international recommendations. We use the data collected from the previous analysis as a basis of comparison (14).

MaTerials anD MeThODs
We carried out a cross-sectional analysis of the nutritional content of 5,237 recipes from the food website Allrecipes. com. The global version of the site was chosen rather than the British version as the British version alone included insufficient data for analysis. This is unlikely to be a significant source of bias as the global version of the site is popular in Britain as well as worldwide: the web analytics service Alexa.com ranks Allrecipes.com as the most popular cooking community platform (18).
The data describing the Allrecipes.com recipes was downloaded in Summer 2015 and contains 7648 recipes published between the years 2000 and 2010 on the Allrecipes.com website as main dishes.
Rather than sampling from the dataset (as was done with the ready meals and celebrity recipes), we used all of the recipes available, which meet the criteria applied in the previously published analysis (14). This was important as we wished to draw comparisons with the meals analyzed in the previous work. The following restrictions were applied: • we included only main dishes consisting of at least 225 g per portion, matching the restriction used in Howard et al. 's analyses. • we restricted the recipes to those added before the end of year 2010 as these would have been the recipes available to users of the site at the time of Howard et al. 's analyses. • we chose only recipes for which reliable nutritional information could be provided (see below). Table 1 shows how these filters influenced the number of recipes analyzed. For the other categories of meal (TV chef recipes and supermarket ready meals), the data were provided by the lead author of an earlier study (14). Full details of the inclusion and exclusion criteria and sampling methodology for these recipes can be found in their paper (14). Ethical approval was not required for this analysis of published, publicly available information.

nutritional content of included internet recipes
The following nutritional information is available in our dataset about each recipe: year of publication; the recommended number of servings; and total energy (kJ), protein (g), carbohydrate (g), sugar (g), sodium (mg), fat (g), saturated fat (g), and fiber (mg) content.
The nutritional metadata was available via Allrecipes.com and collected during the main crawl. Allrecipes.com estimates the nutritional content for an uploaded recipe by matching the contained ingredients with those of the ESHA research database (19).
A small number of the main dish recipes (29 in total) collected have no nutritional information available. These recipes were excluded from our analyses. How these exclusion criteria influenced the number of recipes analyzed is shown in Table 1.

statistical analysis
Throughout our analyses, we make use of two internationally recognized standards for measuring the healthiness of meals: the World Health Organization (WHO) guidelines (20) and the UK FSA "traffic light" system for labeling food (6).
The WHO has defined 15 ranges of macronutrients, which should be considered in a daily meal plan. We follow the approach of Howard et al. (14) who chose the 7 most important (i.e., proteins, carbohydrates, sugars, sodium, fats, saturated fats, and fibers) and their corresponding ranges to determine a so-called WHO health score. The scale ranges from 0 to 7 (0 meaning none of the WHO ranges are fulfilled and 7 meaning all ranges are met). A recipe or meal plan with a WHO score of 7 is interpreted as being very healthy, whereas a score of 0 is seen as very unhealthy.
A similar approach is taken to derive an FSA traffic light labeling system score. The FSA score relates only to 4 macronutrients (sugar, sodium, fat, and saturated fat). The scale is green (healthy), amber, and red (unhealthy).
Following the procedure described in Howard et al.'s paper (14), for each meal, we calculated the nutritional content per portion by dividing the total content by the number of portions in the meal. Using the Mann-Whitney test, we compared the total content per portion between the Internet recipes and ready meals, as well as Internet recipes and recipes from TV chefs.
We calculated the percentage of energy derived from each macronutrient for each meal and used the Mann-Whitney test to compare the differences between the groups of ready meals and recipes, as well as recipes from the Internet and TV chefs. Using χ 2 tests, we compared the percentage of energy derived from macronutrients in the meals of different types with the nutrient intake goals for preventing diet-related chronic diseases recommended by WHO (6).
For each meal in different groups, we assigned a "traffic light" color for the four macronutrients (fat, saturated fat, sugar, and salt) according to a modified version of the 2007 FSA guidance on its recommended labeling scheme (20). Our modification was that, due to the fact that data on the proportion of sugar derived from such sources were not available, we did not include the criterion allowing a higher total sugar content in situations, where a high proportion of sugar is derived from natural sources. This mirrors the procedure applied in Howard et al. 's work (14). The traffic light system is used on the front of packaging to help consumers assess at a glance the fat, saturated fat, sugar, and salt content of meals, with the aim of helping them to make healthier dietary choices.
In addition to the statistical analyses performed by Howard et al., we calculated the same values for recipes published each year on Allrecipes.com. This allowed us to determine how stable the results are over time. Table 2 shows the nutritional content per portion of Internet recipes created by users in Allrecipes.com compared to TV chef recipes and supermarket ready meals. The TV chef recipes contain more energy (2,530 v 2,113 kJ (Internet) v 2,066 kJ (ready)) and more protein (37.46 v 29.50 g (Internet) v 27.85 g (ready)) than both Internet recipes and ready meals (all comparisons sig. p < 0.01). Internet recipes are significantly lower in terms of carbohydrate (35.20 v 49.48 g (TV) v 51.05 g (ready), p < 0.01) and sugar (5.50 v 8.25 g (TV) v   6.80 g (ready), p < 0.01, p = 0.02) content, but significantly higher than TV chef recipes in terms of sodium density (829 v 660 mg, p < 0.01). There is no significant difference between the sodium content of Internet recipes and ready meals, which are known for unhealthy high levels of salt (21,22) (829 v 800 mg, p = 0.59). In terms of fats and saturated fats, recipes from TV chefs contain the most (fat: 27.06 g, saturated fat: 9.20 g), but ready meals contain the least (fat: 17.20 g, saturated fat: 6.80 g) the difference between ready meals and Internet recipes being highly significant (fat: 17.20 v 24.80 g, p < 0.01; saturated fat: 6.80 v 8.80 g, p < 0.01). The fiber content of Internet recipes is low in comparison to the meals from the other sources (significantly lower than both TV chef meals 3.00 vs 3.44 g, p = 0.01 and ready meals 6.45 g, p < 0.01). Table 3 and Figure 1 summarize the percentage of recipes/ meals of different types, which meet the criteria established by the WHO in terms of the individual nutritional properties. In this case, the values are normalized with respect to the total number of calories in one portion. The data show that only 6 of the 5,237 Internet recipes (0.11%) and none of the sampled TV chef recipes or ready meals meet all 7 criteria. A small number of recipes of each type (5.9% of Internet recipes, 7% of TV chef recipes, and 1% of ready meals) do not meet any of the criteria at all. However, the majority recipes of all three types meet one or two criteria. A total of 46% of Internet recipes meet one criterion compared to 42% of TV chef recipes and 27% of ready meals. As the number of criteria to be met increases, the percentage of Internet recipes meeting these drops at a higher rate than the other types of meal. Figure 1 shows that Internet recipes and TV chef recipes met a similar mean number of WHO criteria (1.76 and 1.77, respectively; SDs 1.17 and 1.10, respectively). Ready meals met a mean of 2.37 criteria (SD 1.21). Using Mann-Whitney tests, the small difference between Internet recipes and TV chef recipes is non-significant (p = 0.64), but ready meals met significantly more WHO criteria than Internet recipes (p < 0.01). Table 4 shows the number and proportion of each meal type that met each nutrient-specific WHO goal. More Internet recipes meet the WHO guidelines for protein than other classes of meal (10.88 v 7% (TV), p < 0.01, 10.86 v 9% (ready), p < 0.01). However, fewer Internet recipes met the criteria for fat (14.28 v 24 (TV) v 37% (ready), p < 0.01), saturated fat (25.05 v 33 (TV) v 34% (ready); p < 0.01), and fiber (when compared to ready meals 16.50 v 56%; p < 0.01). In terms of salt, significantly more Internet recipes met the criteria for sodium density than ready meals (19.63 v 4%, p < 0.01), but significantly fewer than the TV chef meals (19.32 v 36%, p < 0.01). For sugar, no differences between Internet recipes and TV recipes can be observed (81.1 v 81% (TV), p = 0.86). However, Internet recipes significantly differ from ready meals (81.1 v 83% (ready), p < 0.01). Table 5 shows the traffic light assessment for the different recipe types according to modified Food Standards Agency guidelines (20). The FSA guidelines are based on macronutrient properties normalized by portion size. According to these guidelines, Internet and TV chef recipes have almost equal numbers of red labels (45 v 47% (TV)), but both TV recipes and ready meals have more green labels than Internet-sourced recipes (36 v 42 (TV) v 41% (ready)). Figure 2 shows these data averaged over all recipes of each type to provide simulated front of package labels for an average Internet recipe, an average TV chef recipe, and an average ready meal using a design based on FSA guidelines. Internet and TV recipes are classified as having high fat; all three categories are labeled as having low sugar, and both ready meals and Internet recipes are considered to contain medium salt.

resUlTs
Overall based on the FSA guidelines, the healthiest category is the ready meals and the unhealthiest category is Internet recipes, which is the same overall conclusion drawn when using the WHO guidelines.
To establish how stable these values are over time, we calculated the same statistics for each year of data available to us. Table 6 shows the nutritional content per portion of Internet recipes created by users in Allrecipes.com for the years 2000 to 2015. Similarly, Figure 3 presents the simulated front of package labels, based on the guidelines from the FSA (20), for an average Internet recipe created by Allrecipes.com users for each year during that time period. Overall both depict stable trends over time. The FSA labels for macronutrients, for the average recipe based on annually uploaded recipes, are the same for every year in the dataset. Figure 4 demonstrates the percentage of Allrecipes.com recipes meeting different numbers of WHO criteria overall, and Figure 5 depicts the same information at a macronutrient granularity. Both figures show limited annual variation but present an overall consistent trend.

DiscUssiOn
Internet recipes sourced from Allrecipes.com tend to be high in protein, fat, saturated fat, and sodium, low in carbohydrate, and   within the recommended range for sugar according to World Health Organization's nutritional guidelines for the avoidance of diet-related diseases (6). When compared to meals based on television chef recipes and ready meals from leading UK supermarkets, the Internet recipes were the least healthy. Significantly, fewer were within the recommended ranges for fat, saturated fat compared to meals from the other sources, and significantly fewer Internet recipes met the recommended range for fiber than ready meals. In terms of sodium density, significantly more Internet recipes met the criteria than ready meals, but significantly fewer than the TV chef meals. Internet recipes did, however, meet the criteria for protein content significantly more often than the other categories.
Internet recipes were also more likely to achieve red traffic light labels according to the criteria of the UK Food Standards Agency (FSA) (20). By investigating the criteria for each year of data collection, we revealed that the characteristics of recipes uploaded to the site are very stable on average. Thus, our findings, taken together with those of Howard et al., add weight to the argument that encouraging people to cook more at home might not, by itself, be enough to improve nutritional habits. This is because some common sources of cooking inspiration themselves promote unhealthy meals.

strengths and Weaknesses of the study
This study is the first to comprehensively evaluate the nutritional content of Internet-sourced main meals. To our knowledge, it is not only the largest study investigating the healthiness of online sourced recipes but also the first to systematically compare the nutritional properties of Internet recipes with those of other sources of cooking inspiration. Moreover, it is also the first investigation to provide insights on the nutritional content of Internet recipes over a long period of time (16 years in total). Although we examined the food website with the largest traffic volumes on the Internet, it is possible (and perhaps likely) that recipes from different Internet sources may have provided different results.
A second limitation of the study is that we have analyzed the Allrecipes.com collection as a whole. The collection is not subject to the same editorial or space constraints as a book or supermarket shelf, and thus no restrictions are placed on the recipes published on the site. It could be, therefore, that certain classes of recipes (e.g., Vegetarian, Asian, Italian, gluten-free, etc.) are over or underrepresented in the collection, which would bias the findings. Moreover, it could be that recipes, popular with users of the food portal, have different nutritional properties to unpopular recipes. Further analyses we performed confirm this (23). We now know that, for example, recipes accepted by users (i.e., recipes bookmarked most often and rated most highly) tend to be on average the least healthy. We also know that recipes associated with particular categories on the site are healthier than others according to the health guidelines we applied and users are not adept at distinguishing, which categories these might be. Although the new analyses were not performed with respect to comparing with other sources of meals as we have done here, they nevertheless endorse and strengthen the conclusions we have drawn based on the findings reported in this paper.   A further limitation of the study is that Allrecipes.com is an international site and a large percentage of the recipes are sourced from the US, whereas the TV chef recipes and ready meals were sourced from the UK. However, access statistics show that the site is among the most popular food sites in the UK, although the UK localized version (which was not available for the full duration of the period for which we wanted to draw comparison) has since overtaken the international version in terms of numbers of visits from UK users.
The datasets describing the supermarket ready meals and cookery books were created sometime ago and published in Howard et al. 's work (14) in 2012. All of the ready meals and books tested are still available for purchase at the time of writing.
It is important to acknowledge that the nutritional information for the three groups of meals was collated using different methods-for ready meals, the manufacturer's data was taken; for Internet recipes, Allrecipes.com data was taken; the TV chef recipes were analyzed using the WinDiets software (24). However, we would expect, based on the comparisons already reported in the medical literature (14), these methods to have broadly comparable results.
To evaluate the health properties of recipe and meals, we used metrics based on guidelines from the World Health Organization and the UK Food Standards Agency. These choices were driven by precedence in the literature (14). It could be argued that the WHO score might not be the best measure to determine the healthiness of an individual recipe or meal because it was designed to evaluate whole diet meal plans. Nevertheless, in our opinion, it is still a useful measure as it shows similar health trends to the FSA score but incorporates a wider range of macronutrients in the metric.
Our later analyses in Ref. (23) show that there is a strong and significant correlation between the WHO and FSA scores for Allrecipe.com recipes.
Systematic variation from the true macronutrient value of the foods could be a source of bias in this study. In the European Union, which at the time of writing includes the United Kingdom, the published nutritional data used for analysis of ready meals are permitted by law to vary by up to 20% from the true macronutrient values (25). Similarly, establishing the nutritional content of a recipe using ingredient mapping, as is done by Allrecipes.com, is known to be imperfect (26). However, in all cases, our analyses were based on the most accurate data currently available to the public.
We also concede that variation may exist in the way users of Allrecipes.com might use or consume recipes. For example, people may not follow all the steps and ingredients guides in the recipes, which may result in different nutritional intake. Similarly, recipes can be combined, perhaps with a side dish, which may or may not be likely with a ready meal. It is not possible with this kind of analysis to account for such a variation.
As Howard et al. (14) also conceded in their article, our analyses may have systematically mis-estimated the salt content within the recipe groups. Many recipes have the ingredient salt listed with the quantity marked as "a pinch" or "to taste. " Whereas Howard et al. ignored listed salt entries completely, we used the standard values calculated by Allrecipes.com. This website has fixed rules for these quantities, but in practice individual users may apply salt liberally or conservatively based on their own individual tastes. Therefore, the findings for salt, and in particular the comparison with TV chef recipes, should be interpreted with caution.
In our study, individual recipes were analyzed, but a healthy diet is created by combining a variety of food types. Past work has suggested combining recommended online recipes in a manner such that a created daily meal plan meets guidelines from official health organizations (27). This possibility is not reflected in our study.

comparison with Other studies
The primary investigation with which we can draw comparison is the study by Howard et al. (14). As reported above, with respect to the WHO and FSA guidelines, the Internet-sourced recipes were evaluated to be less healthy overall.
A 2010 study by Silva et al. (28) applied a healthy recipe index to 204 recipes featured on 2 Food Network shows and found that recipes were ranked as less than healthy by the index measure. Further analysis found that the recipes analyzed were excessive in energy, saturated fat, and sodium based on a 2,300-kcal diet.
While there is a general lack of research on the healthiness of online recipes, one relevant study comes from Schneider and colleagues, who investigated the nutritional properties of recipes (entrees and main dishes) sourced via popular online food blogs (29). The dishes were evaluated using US Department of Agriculture and US Department of Health and Human Services, Dietary Guidelines. The recipes analyzed met energy recommendations but were excessive in saturated fat and sodium.

Unanswered Questions and Future research
When investigating the nutritional content of recipes from food blogs, Schneider et al. (29) found that all risk-related nutrients of interest were significantly lower in vegetarian recipes compared with red meat and poultry recipes. One future research direction would be to investigate if this was the case in Allrecipes. com. Schneider et al. also discovered differences in sodium levels across seasons. A further line of research could similarly investigate temporal patterns in the nutritional properties of uploaded recipes.
It would, furthermore, be possible to repeat our analyses for other popular food sites, which can be crawled in a similar manner. Previous work has identified cultural differences in cooking habits via online recipe databases (30), and it would be interesting to investigate such cultural differences from the perspective of healthy nutrition.

Meaning of the study and implications
The Internet is a technology which the evidence suggests appeals to people as a source of cooking inspiration. Our analyses show that the content provided may not be healthy. As Howard et al. (31) suggest, the home cooking of nutritionally balanced recipes using primarily raw ingredients would likely be a nutritionally superior strategy relying on Internet recipes, recipes by television chefs, or ready meals.
Nevertheless, a problem associated with a technology may have a technological solution. For example, recommendation systems-such as those commonly used in online shops such as http://Amazon.com-have the potential to be used to find recipes of a similar type, but with different nutritional properties (31). Initial steps in this direction have been taken specifically for patients with diabetes (32).

aUThOr cOnTriBUTiOns
CT collected and analyzed the data. DE and CT drafted the manuscript, and both authors contributed to the study design and interpretation of results. SH commented on successive drafts of the manuscript and supported CT with the study design. CT and DE will act as guarantors.

FUnDing
This work was supported by the German Research Foundation (DFG) within the funding programme Open Access Publishing.