Opening Pandora's box: caveats with using toolbox-based approaches in mathematical modeling in biology

1 Classical mathematical modeling in biology

Mathematical modeling in biology is a process in which one constructs a mathematical equivalent (a.k.a., a model) of a given biological system/process [1]. As different artists would look at the same landscape and come up with different ways to represent it in a painting, different scientists would come up with different mathematical models for the same biological system/process. In that respect, mathematical modeling is as much an art as it is a technique [1]. The purpose of making a mathematical model may vary with the system, specific questions being asked, and scientists constructing the model. Models can be constructed to estimate biologically interesting parameters, for example, the rates at which T lymphocytes divide or die in response to a viral infection [2–4]. Or, as I have recently proposed, mathematical models can be used to rigorously determine which of several alternative hypotheses about specific biological system are consistent and which are not consistent with the data (so-called “strong inference in mathematical modeling,” [5]). Mathematical models can also be used to make predictions of how various interventions such as vaccination may impact dynamics of infectious diseases [6]. Mathematical models can also illustrate what types of outcomes one could expect from sometimes simple assumptions, e.g., how mathematical models with just one parameter can exhibit simple and complex behaviors [7]. Finally, the process of making a mathematical model (or a set of alternative models) in itself may in some cases be useful because in such a process one must outline basic details of the biological system that need to be incorporated in the model; and thus, such an exercise may identify areas where the understanding of biological processes is sufficient and where there is a need for additional experiments/measurements.

Given such a wide range of how mathematical modeling can be used, how does one learn to do mathematical modeling? While some may be lucky to start learning about mathematical modeling in high school, my experience was different, and perhaps as for many, my exposure to mathematical modeling, based primarily on differential equations, started in college, at Krasnoyarsk State University in Krasnoyarsk, Russia (now rebranded as Siberian Federal University). We first learned how to solve analytically different types of differential equations (ordinary and partial), and how autonomous systems of differential equations can be rigorously analyzed and their dynamics understood. A course in biophysics introduced how ODE-based mathematical models of various biological processes can be built and analyzed [8]. Because most such models were non-linear, understanding their behavior required different analytical techniques than finding the model's analytical solution. One of the memorable techniques I learned then was a separation of timescales in the model, driven by a relative difference in the model parameters [8, chapter 1]. For example, by modeling the dynamics of a substrate, enzyme, and a product, experimentally measured values of kinetic constants suggested that dynamics of the complex of substrate and enzyme is much more rapid compared to other components in the model allowing to derive the classical Michaelis–Menten relationship between speed of reaction and substrate concentration [8, chapter 2]. A very lucid description of time scales and the quasi-stability idea as it relates to enzyme kinetics has been also discussed by Segel [9, chapter 4]. While we also learned about numerical methods to solve ODE-based models we nearly never applied those in practice, and understanding of the dynamics of these models had to be done using pencil and paper. Such an approach allowed to rigorously understand mathematical models in question.

During my graduate studies at Emory University (Atlanta, USA) we learned many similar mathematical models in biology including classical ecological models (e.g., Lotka-Volterra models, [10]). Exposure to mathematical models of infectious disease dynamics, both within-host and epidemiological, from the classical textbook by Anderson and May [11], determined my career path for years to come. Many of the analyses including those by Anderson and May [11] were done using computers; however, many analytical techniques allowing to understand model behaviors were introduced and reinforced. Anderson and May [11] also introduced me to the world of how models can be compared to experimental data although in many instances this was done using only qualitative comparisons.

Interestingly, my first experiences in teaching mathematical modeling followed examples from my undergraduate and graduate years. As a post-doc at Utrecht University (Utrecht, Netherlands) I helped with teaching mathematical models to 1st year undergraduate students using an in house textbook [12]. In that course students learned basic mathematical models in various fields of biology and learned how these models can be analyzed using different analytical techniques such as finding steady states, determining their stability, etc. As a faculty member at the University of Tennessee, Knoxville we taught 1st year undergraduate students how to build analytical models to predict species dynamics or accumulation of interest in the retirement account along with methods for basic data analysis [13]. Graduate students in Mathematics department learned basics of mathematical modeling including ODE- and PDE-based models, again, using a variety of analytical techniques [14]. In these classes students typically learned so-called “forward” modeling whereby for a given biological system, one develops a mathematical model and uses the model dynamics/properties/predictions to understand the biological system in question. Solving an “inverse” problem whereby one attempts to identify the correct model given a set of data is typically more challenging. While there have been useful textbooks that focus on comparing models with experimental data [15], I evolved my own set of lectures on “inverse” modeling in biology. Inverse modeling typically involves computer programming since fitting the models to data cannot be easily done analytically; however, understanding behavior of the proposed mathematical models (to be fitted to data) has still been a critical component of my classes on mathematical modeling in biology. Overall, during my years of learning basics of mathematical modeling and my experience at teaching mathematical models to others, the key component has been to rigorously understand the developed mathematical models. Following such analyses it was relatively easy to answer the “why” question—why does the model do what it does?

2 Recent changes in mathematical modeling in biology

In the last few decades mathematical modeling in biology changed from what one could view as traditional approaches in modeling exemplified in classical textbooks [10, 11, 14]. Don't get me wrong—there are plenty of papers that use standard approaches of mathematical model analysis, e.g., positiveness of model solutions, number and stability of steady states, bifurcation analyses, etc. But in my area of expertise of within-host dynamics of infectious diseases and immunology, many if not most papers published in higher tier journals rarely focus on basic properties of mathematical models. Rather such models are compared to experimental data and are often large in size with tens to hundreds of variables and/or parameters. In fact, comparing models to data by using various fitting methods (e.g., least squares) has allowed mathematical modeling to be recognized as a valuable tool by experimentalists, e.g., in cancer biology [16]. Using mathematical modeling to discriminate between alternative mechanisms (i.e., strong inference [5, 17]) by using model comparison metrics such as Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) or similar can also help determine which directions research should take next [18–21]. Larger datasets involving longitudinal measurements, for example, virus concentration in the blood allows using advanced statistical methods such as non-linear mixed-effect modeling that can help estimate variability in model parameters between different individuals [22].

These novel developments in mathematical modeling in biology (large models, model comparisons, and non-linear mixed-effects modeling), while useful, generated a major problem: a blind, somewhat mechanical use of various tools that help researchers with these tasks. Indeed, because traditional training in mathematical modeling in biology typically deals with relatively small models and rarely with data, toolboxes that help with analyses of larger models or with comparisons of models to data may be desirable. And the number of various toolboxes for mathematical modeling in biology has increased dramatically in recent years [23]. The issue that I would like to highlight is when such toolboxes are used blindly, in a mechanical manner, i.e., without thorough understanding of the results of such analyses. Let me elaborate this point using three specific methodologies used in mathematical modeling in biology: (1) model sensitivity analyses, (2) model selection based on AIC, and (3) non-linear mixed-effect modeling.

2.1 Sensitivity analysis tools

From reviewing and reading recently published papers my impression is that in recent decades the average complexity of mathematical models in infectious disease biology and immunology (defined as average number of state variables and interactions) has gone up. This, however, makes intuitive sense—the amount of quantitative data on pathogen and immune response dynamics have increased and such an increase in data size naturally results in mathematical models that have more variables and parameters. In addition, our understanding of biological systems has also improved over time necessitating including more such details in the models. Such models often focus on transient dynamics and not steady states and are typically solved numerically. Many of the standard analytical tools for the analysis of such mathematical models taught in college are becoming relatively useless. Luckily, there have been development of many methods/tools that help with determining how various model parameters impact behavior of the mathematical model in question; these are sensitivity analysis tools. The methods may include local and global sensitivity analyses, may involve analytical or numerical methods based on Sobol indices, partial rank correlation coefficients, and other metics [24–28]. Papers presenting such methods in a systematic manner accrue many citations, and there are now multiple online tools or R libraries such as DAISY, sensobol, sensitivity, sensemakr allowing to automatically perform various sensitivity analyses [26, 29–31]. For example, Marino et al. [27] described how by using latin hypercube sampling and by calculating partial rank correlation coefficients one can evaluate the contribution of each model parameter to a specific output of the model (e.g., extracellular bacterial load); popularity of this methodology is exemplified by many citations (1590 on www.webofscience.com as of Dec 5, 2023; on average, a paper in PNAS is cited about 10 times/year).

While these methods can deliver ranking of model parameters in terms of their impact on various aspects of model dynamics, it is typically unclear why one parameter is more important than another. For example, Marino et al. [27] performed sensitivity analysis of the predator-prey models using several different methods and found that predator consumption rate of the prey β is more important than prey's growth rate α. However, why β is more important than α for the model dynamics was not explained. What is worse, a typical mathematical modeling paper using some of these toolbox-based sensitivity analyses does not even discuss whether estimated parameter sensitivities make sense and whether changing the relevant model output will influence ranking of the model parameters in terms of their impact on output. After submitting a mathematical modeling paper to a journal for a review I sometimes get feedback from reviewers that “you need to perform sensitivity analyses”—but do we? At the end, sensitivity analysis whatever way performed is supposed to clarify how the model outputs depend on model parameters, and as I described earlier, decades ago we used to do that with pencil and paper; now such analyses can be also done using numerically by simulating model outputs for different parameter values. Applying standard routines that spill out answers without explaining why a given model parameter has that specific influence on model output degrades the value of mathematical modeling.

A similar argument can be applied to a related area of parameter identifiability—a procedure that allows to determine which model parameters can be estimated from a particular type of data and which parameters cannot [32]. There are standard approaches proposed to address the question of parameter identifiability but the same argument of a mechanical application of toolbox approaches applies—determining that some parameters are identifiable and some are not requires an explanation of why that is the case. However, this is not typically done.

2.2 Model selection tools

Another interesting development in mathematical modeling in biology is availability of experimental data and the use of data to determine which mathematical models fit experimental data with better quality than some other models. There are different metrics of quality of model fit to the data but in the area of mathematical modeling in immunology and infectious disease dynamics AIC has been used commonly [18]. Due to a simple interpretation (“distance” from a true model), ability to rank models based on their AIC values, and availability of many tools calculating AIC (e.g., it is standard for many routines in R), use of model comparison tools such AIC is sometimes even required in publications. But here comes an issue—some studies including in both ecology and immunology compare from tens to hundreds of mathematical models and select the best (with lowest AIC value) without a deeper understanding why a given model is chosen as the best fit model [33, 34]. For example, one study looked at the distribution of sizes of cysts formed by the parasite Toxoplasma gondii in murine brains [33]. The authors considered how growth and death of cells in these cysts depends on the cyst's size and by testing 24 alternative models authors found that a model with a constant growth and removal rates fitted the data with nearly best quality. However, differences in many of the tested models were minor, with small changes in the terms for growth and/or removal, and why some of these alternative models fitted the data with nearly similar quality was not explained. Another study looked at differentiation of CD8 T cells during an infection with bacteria Listeria monocytogenes [34]. By tracking differentiation of individual naive CD8 T cells in vivo authors could document expression of several cell surface markers such as CD62L and CD27. By generating hundreds of alternative models of how CD8 T cells may differentiate during the infection and comparing model predictions with the data, the authors proposed that linear differentiation pathway (from naive cells to memory to effectors) was most consistent with the data [34]. However, why the best fit model was actually the best fit and why alternative models could not fit the data well was not explained.

The approach of generating a large number of alternative models that differ in minuscule elements and then of routinely comparing these models to data to find the best one is a common approach in ecology; this approach has been previously criticized as lacking an intelligent approach to modeling [35]. Ver Hoef and Boveng [35] even proposed that rather than testing alternative models, one should focus on a single model and by iterating the model, one may better understand if the model can fit the data well and why. Iterating or tinkering with the models is an extremely useful exercise—in some ways, it is the sensitivity analysis of the model (this can be also defined as an uncertainty analysis). However, focus on a single model is logically flawed because a single proposed model is likely to focus on a single underlying mechanism and there may be alternative mechanisms at play [36]. Rather, focusing on several alternative key mechanisms and testing these via comparing models with data—so-called strong inference in mathematical modeling [5], is clearly a more rigorous approach. Fitting different models to data and comparing quality of their fit is important but explaining why a given model fits well but another one fails to fit well is critical for mathematical modeling in the twenty-first century (e.g., [37]).

2.3 Non-linear mixed-effect modeling tools

There are now more studies in infectious diseases and immunology that are done in non-human primates and humans than decades ago. In such experiments it is typical to follow burden of different infections or immune responses over time in the same individual; for example, it is possible to track shedding of influenza virus or SARS-CoV-2 in nose/throat or antibodies against HIV in the blood in individual humans over time. In contrast with inbred strains of mice that may present very similar dynamics of an infection and/or immune response, monkeys and humans may display much greater variability in measurements, precluding sometimes using averages as a rigorous metric of the infection dynamics [38]. Recent advances, borrowed from pharmacokinetic studies of drugs in humans, involve using non-linear mixed-effect modeling to account for variability in different model parameters between individuals [39]. Fitting non-linear, ODE-based mathematical models to longitudinal data from multiple individuals has been dramatically assisted with the release of free-for-academics (but yet proprietary) program Monolix with a simple and intuitive interface and powerful computational methods to ensure fit convergence (https://monolix.lixoft.com/). There is a surge in papers that build relatively complex mathematical models that often cannot be fit to data from a single individual but by using the power of mixed-effect modeling approach (and Monolix) sometimes adequate fits of the model to data can be generated [40, 41]. However, (blind) usage of this tool can also result in conclusions that are not well-explained. For example, when fitting the models to data using mixed-effect modeling approach one must decide on which parameters should be fixed between individuals (fixed effects) and which should vary (random effects), what is the distribution of random effects (normal, lognormal, bimodal, etc.), and what are potential interactions between these parameters. Whether all parameters are identifiable when using mixed-effects modeling approach and how selecting the best model depends on the choice of fixed vs. random effects has not typically been rigorously evaluated. For example, Néant et al. [40] fitted a mathematical model of SARS-CoV-2 dynamics to data from patients with COVID19 sampled longitudinally. Even though the model fitted the data, the model included initial viral load and eclipse phase parameters that clearly could not be identified from the data because no initial viral loads were available (and that different eclipse phase durations are likely to be consistent with the data). Another study used better viral load measurements but could conclude how various elements of immunity contribute to the viral shedding pattern even though no immunological information was available in the patients [41]. Using a toolbox approach allows to get answers (“model fits the data”) but whether these answers make sense and why the model is able to fit the data is typically not discussed.

3 Bringing the “why” question back to mathematical modeling

In this Perspective I highlighted issues with a blind, mechanical use of several toolbox-based approaches to mathematical modeling in biology. While examples I cited came from immunology and infectious disease biology, I suspect that the use of toolbox approaches “without understanding why they give the answers they do,” probably occurs in other areas of biology.

But don't misinterpret my point—there is nothing wrong with using toolbox approaches per se. For example, years ago I used to code different numerical methods in Pascal or C to solve systems of ODEs but now we have plenty of well-designed routines to solve different types of equations. We often use ODE solvers without thinking as many libraries utilize very efficient numerical methods adequately. We rarely if ever check if our ODE solver gives us the correct solution of the model (e.g., by changing the method of solving the ODE numerically or by changing tolerance or other parameters of the solver) unless we get a warning (or an error) message that the produced numerical solution may be inaccurate. However, the numerical methods to solve ODEs matured over many decades, and the tools I discussed here do not have such a rich history and are likely to lead us astray unless we generate an understanding of why the results are what they are.

If sensitivity analysis tells us that one of the parameters is more important than another, a good approach would be to ask “Why is that the case? If I propose an explanation, how do I know this explanation is the correct one? What are other ways I can investigate if this parameter is truly important in the model?” When fitting alternative models to data and comparing them using AIC, we should ask “Why did the best fit model actually fit the data best? What is so special about that model? Why did the alternative model fail to fit the data? What did it miss? And how do I know that this was an important feature the model was missing?” Finally, when the model was fitted to data using mixed-effect modeling approach and fitted the data well, we should definitely ask “Did we have enough data to estimate all the model parameters? How do we know? Which features in the model, e.g., fixed vs. random effects, are critical in model fits of the data?”

When using such toolbox-based approaches we may be “getting something for nothing” (phrase coined to me by Louis Gross) which is probably impossible, there is always a price to pay. And the price may be that with using the tools we miss understanding of our model's behavior. Because of that there is a risk that insights generated by toolbox-based approaches are incorrect. So, in moving forward, let us all bring back the “why” question to the mathematical modeling in biology. Aiming beyond a blind, mechanical use of toolbox-based approaches in mathematical modeling in biology, let us bring back intuitive insights into model functioning. Let us remember that modeling is an art and not simply engineering [1].

Data availability statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

VG: Conceptualization, Funding acquisition, Writing—original draft, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was in part supported by an NIH grant (R01AI158963) to VG.

Acknowledgments

I would like to thank members of my lab (especially Viktor Zenkov) and Alan Hastings and Louis Gross for discussions on the topic of using toolbox approaches in mathematical modeling and feedback on earlier versions of the paper.

Conflict of interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Cross M, Moscardini AO. Learning the Art of Mathematical Modelling. Hoboken, NJ: John Wiley & Sons, Inc. (1985).

Google Scholar

2. De Boer RJ, Oprea M, Antia R, Murali-Krishna K, Ahmed R, Perelson AS. Recruitment times, proliferation, and apoptosis rates during the CD8(+) T-cell response to lymphocytic choriomeningitis virus. J Virol. (2001) 75:10663–9. doi: 10.1128/JVI.75.22.10663-10669.2001

PubMed Abstract | Crossref Full Text | Google Scholar

3. De Boer RJ, Homann D, Perelson AS. Different dynamics of CD4(+) and CD8(+) T cell responses during and after acute lymphocytic choriomeningitis virus infection. J Immunol. (2003) 171:3928–35. doi: 10.4049/jimmunol.171.8.3928

PubMed Abstract | Crossref Full Text | Google Scholar

4. Le D, Miller JD, Ganusov VV. Mathematical modeling provides kinetic details of the human immune response to vaccination. Front Cell Infect Microbiol. (2014) 4:177. doi: 10.3389/fcimb.2014.00177

PubMed Abstract | Crossref Full Text | Google Scholar

5. Ganusov VV. Strong inference in mathematical modeling: a method for robust science in the twenty-first century. Front Microbiol. (2016) 7:1131. doi: 10.3389/fmicb.2016.01131

PubMed Abstract | Crossref Full Text | Google Scholar

6. Thakkar K, Spinardi JR, Yang J, Kyaw MH, Ozbilgili E, Mendoza CF, et al. Impact of vaccination and non-pharmacological interventions on COVID-19: a review of simulation modeling studies in Asia. Front Publ Health. (2023) 11:1252719. doi: 10.3389/fpubh.2023.1252719

PubMed Abstract | Crossref Full Text | Google Scholar

7. May RM. Simple mathematical models with very complicated dynamics. Nature. (1976) 261:459–67.

PubMed Abstract | Google Scholar

8. Romanovskii I, Stepanova N, Chernavskii D. Mathematical Modeling in Biophysics. Moscow: Izdatel'stvo Nauka (1975). p. 344.

Google Scholar

9. Segel LA. Modeling Dynamic Phenomena in Molecular and Cellular Biology. Cambridge: Cambridge University Press (1984).

Google Scholar

10. Lotka AJ. Elements of Mathematical Biology. Mineola, NY: Dover Publications (1956).

Google Scholar

11. Anderson RM, May RM. Infectious Diseases in Humans: Dynamics and Control. Oxford: Oxford University Press (1991).

Google Scholar

12. De Boer RJ. Theoretical Biology: Ebook. (2007). Available online at: http://theory.bio.uu.nl/rdb/books/tb.pdf (accessed January 09, 2024).

Google Scholar

13. Bodine EN, Lenhart S, Gross LJ. Mathematics for the Life Sciences. Princeton NJ: Princeton University Press (2014). Available online at: http://books.google.com/books?id=Zz-OAwAAQBAJ (accessed January 09, 2024).

Google Scholar

14. Kot M. Elements of Mathematical Ecology. Cambridge: Cambridge University Press (2001). Available online at: http://books.google.com/books?id=7_IRlnNON7oC (accessed January 09, 2024).

Google Scholar

15. Mooney D, Swift R. A Course in Mathematical Modeling. Washington, DC: Mathematical Association of America (1999).

Google Scholar

16. Quaranta V, Weaver AM, Cummings PT, Anderson ARA. Mathematical modeling of cancer: the future of prognosis and treatment. Clin Chim Acta Int J Clin Chem. (2005) 357:173–9. doi: 10.1016/j.cccn.2005.03.023

PubMed Abstract | Crossref Full Text | Google Scholar

17. Platt JR. Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others. Science. (1964) 146:347–53.

PubMed Abstract | Google Scholar

18. Burnham KP, Anderson DR. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York, NY: Springer-Verlag (2002).

Google Scholar

19. Antia R, Ganusov VV, Ahmed R. The role of models in understanding CD8+ T-cell memory. Nat Rev Immunol. (2005) 5:101–11. doi: 10.1038/nri1550

PubMed Abstract | Crossref Full Text | Google Scholar

20. Ganusov VV, Milutinovic D, De Boer RJ. IL-2 regulates expansion of CD4+ T cell populations by affecting cell death: insights from modeling CFSE data. J Immunol. (2007) 179:950–7. doi: 10.4049/jimmunol.179.2.950

PubMed Abstract | Crossref Full Text | Google Scholar

21. Bresser K, Kok L, Swain AC, King LA, Jacobs L, Weber TS, et al. Replicative history marks transcriptional and functional disparity in the CD8+ T cell memory pool. Nat Immunol. (2022) 23:791–801. doi: 10.1038/s41590-022-01171-9

PubMed Abstract | Crossref Full Text | Google Scholar

22. Guedj J, Thiebaut R, Commenges D. Maximum likelihood estimation in dynamical models of HIV. Biometrics. (2007) 63:1198–206. doi: 10.1111/j.1541-0420.2007.00812.x

PubMed Abstract | Crossref Full Text | Google Scholar

23. Mac Gabhann F, Pitzer VE, Papin JA. The blossoming of methods and software in computational biology. PLoS Comput Biol. (2023) 19:e1011390. doi: 10.1371/journal.pcbi.1011390

PubMed Abstract | Crossref Full Text | Google Scholar

24. Saltelli A, Tarantola S, Chan KS. A quantitative model-independent method for global sensitivity analysis of model output. Technometrics. (1999) 41:39–56.

Google Scholar

25. Sobol IM. Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Math Comput Simul. (2001) 55:271–80. doi: 10.1016/S0378-4754(00)00270-6

Crossref Full Text | Google Scholar

26. Bellu G, Saccomani MP, Audoly S, D'Angi L. DAISY: a new software tool to test global identifiability of biological and physiological systems. Comput Methods Progr Biomed. (2007) 88:52–61. doi: 10.1016/j.cmpb.2007.07.002

PubMed Abstract | Crossref Full Text | Google Scholar

27. Marino S, Hogue IB, Ray CJ, Kirschner DE. A methodology for performing global uncertainty and sensitivity analysis in systems biology. J Theor Biol. (2008) 254:178–96. doi: 10.1016/j.jtbi.2008.04.011

PubMed Abstract | Crossref Full Text | Google Scholar

28. Campbell JE, Carmichael GR, Chai T, Mena-Carrasco M, Tang Y, Blake DR, et al. Photosynthetic control of atmospheric carbonyl sulfide during the growing season. Science. (2008) 322:1085–8. doi: 10.1126/science.1164015

PubMed Abstract | Crossref Full Text | Google Scholar

29. Cinelli C, Hazlett C. Making sense of sensitivity: extending omitted variable bias. J Royal Stat Soc Ser B Stat Methodol. (2020) 82:39–67. doi: 10.1111/rssb.12348

Crossref Full Text | Google Scholar

30. Puy A, Lo Piano S, Saltelli A, Levin SA. Sensobol: an R package to compute variance-based sensitivity indices. J Stat Softw. (2022) 102:1–37. doi: 10.18637/jss.v102.i05

Crossref Full Text | Google Scholar

31. Bertrand I, Sebastien Da Veiga AJ. Sensitivity: Global Sensitivity Analysis of Model Outputs. Available online at: https://CRAN.R-project.org/package=sensitivity (accessed January 09, 2024).

Google Scholar

32. Muñoz-Tamayo R, Puillet L, Daniel JB, Sauvant D, Martin O, Taghipoor M, et al. Review: to be or not to be an identifiable model. Is this a relevant question in animal science modelling? Animal. (2018) 12:701–12. doi: 10.1017/S1751731117002774

PubMed Abstract | Crossref Full Text | Google Scholar

33. Sullivan AM, Zhao X, Suzuki Y, Ochiai E, Crutcher S, Gilchrist MA. Evidence for finely-regulated asynchronous growth of Toxoplasma gondii cysts based on data-driven model selection. PLoS Comput Biol. (2013) 9:e1003283. doi: 10.1371/journal.pcbi.1003283

PubMed Abstract | Crossref Full Text | Google Scholar

34. Buchholz VR, Flossdorf M, Hensel I, Kretschmer L, Weissbrich B, Gräf P, et al. Disparate individual fates compose robust CD8+ T cell immunity. Science. (2013) 340:630–5. doi: 10.1126/science.1235454

PubMed Abstract | Crossref Full Text | Google Scholar

35. Ver Hoef JM, Boveng PL. Iterating on a single model is a viable alternative to multimodel inference. J Wildlife Manag. (2015) 79:719–29. doi: 10.1002/jwmg.891

Crossref Full Text | Google Scholar

36. Chamberlin TC. The method of multiple working hypotheses: with this method the dangers of parental affection for a favorite theory can be circumvented. Science. (1890) 15:92–6.

PubMed Abstract | Google Scholar

37. Avilov KK, Romanyukha AA, Belilovsky EM, Borisov SE. Mathematical modelling of the progression of active tuberculosis: insights from fluorography data. Infect Dis Modell. (2022) 7:374–86. doi: 10.1016/j.idm.2022.06.007

PubMed Abstract | Crossref Full Text | Google Scholar

38. Hooker KL, Ganusov VV. Impact of oseltamivir treatment on influenza A and B virus dynamics in human volunteers. Front Microbiol. (2021) 12:631211. doi: 10.3389/fmicb.2021.631211

PubMed Abstract | Crossref Full Text | Google Scholar

39. Siebinga H, de Wit-van der Veen BJ, Stokkel MDM, Huitema ADR, Hendrikx JJMA. Current use and future potential of (physiologically based) pharmacokinetic modelling of radiopharmaceuticals: a review. Theranostics. (2022) 12:7804–20. doi: 10.7150/thno.77279

Crossref Full Text | Google Scholar

40. Néant N, Lingas G, Le Hingrat Q, Ghosn J, Engelmann I, Lepiller Q, et al. Modeling SARS-CoV-2 viral kinetics and association with mortality in hospitalized patients from the French COVID cohort. Proc Natl Acad Sci USA. (2021) 118:e2017962118. doi: 10.1073/pnas.2017962118

PubMed Abstract | Crossref Full Text | Google Scholar

41. Owens K, Esmaeili-Wellman S, Schiffer JT. Heterogeneous SARS-CoV-2 kinetics due to variable timing and intensity of immune responses. medRxiv. (2023) 8:20. doi: 10.1101/2023.08.20.23294350

Crossref Full Text | Google Scholar