# Machine Learning in Model-free Mechanical Property Imaging: Novel Integration of Physics With the Constrained Optimization Process

^{1}F. L. Lizzi Center for Biomedical Engineering, Riverside Research, New York, NY, United States^{2}Department of Civil and Environmental Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States^{3}Department of Mechanical Science and Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States^{4}Beckman Institute of Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, Unites States^{5}Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, United States

The Autoprogressive Method (AutoP) is a fundamentally different approach to solving the inverse problem in quasi-static ultrasonic elastography (QUSE). By exploiting the nonlinear adaptability of artificial neural networks and physical constraints imposed through finite element analysis, AutoP is able to build patient specific soft-computational material models from a relatively sparse set of force-displacement measurement data. Physics-guided, data-driven models offer a new path to the discovery of mechanical properties most effective for diagnostic imaging. AutoP was originally applied to modeling mechanical properties of materials in geotechnical and civil engineering applications. The method was later adapted to reconstructing maps of linear-elastic material properties for cancer imaging applications. Previous articles describing AutoP focused on high-level concepts to explain the mechanisms driving the training process. In this review, we focus on AutoP as applied to QUSE to present a more thorough explanation of the ways in which the method fundamentally differs from classic model-based and other machine learning approaches. We build intuition for the method through analogy to conventional optimization methods and explore how maps of stresses and strains are extracted from force-displacement measurements in a model-free way. In addition, we discuss a physics-based regularization term unique to AutoP that illuminates the comparison to typical optimization procedures. The insights gained from our hybrid inverse method will hopefully inspire others to explore combinations of rigorous mathematical techniques and conservation principles with the power of machine learning to solve difficult inverse problems.

## 1 Introduction

Soft tissues are complex structures that exhibit non-linear, time-dependent elastic properties. Variations in mechanical properties can provide information pertaining to tissue health, including detecting and diagnosing lesions of the breast [6, 34], liver [26, 47, 56], and thyroid [9, 10, 54], monitoring thermal lesions during ablation therapy [41, 53], and characterizing atherosclerotic plaques [2, 3, 11]. Conventional medical imaging modalities are insensitive to this mechanical contrast. Elasticity imaging methods have been developed to fill this niche and directly evaluate the mechanical properties of soft tissue.

Elasticity imaging can be largely grouped into quasi-static or dynamic methods based on the type of mechanical stimulus applied to induce tissue motion. Comprehensive reviews of the various techniques can be found in several articles [33, 35, 38, 39, 46, 52]. We focus our discussion on quasi-static ultrasound elastography (QUSE), wherein local tissue displacements are estimated from RF echo frames acquired as an ultrasound probe is slowly pressed into the tissue surface. Quasi-static loading allows time for the force stimulus to propagate through the entire tissue section. The resulting force-displacement measurements at points provide an enormous amount of information pertaining to tissue material properties throughout the contiguous volume and at boundaries.

Estimating material properties from force-displacement measurements constitutes the inverse problem in QUSE. The majority of quantitative QUSE approaches are built upon model-based optimization methods. While effective, model-based methods are prone to modeling errors that can lead to artifacts in reconstructed images of pre-selected material property distributions. More importantly, mathematically defined constitutive models are inherently limited in their ability to capture material behavior. This limitation could preclude discovery of clinically relevant soft tissue material properties.

Data-driven inverse methods circumvent modeling errors by removing the constitutive model assumption and building a soft-computational model of material behavior from measurement data. The Autoprogressive Method (AutoP) is one such method. Unlike other data-driven approaches, AutoP combines physical modeling through finite element analysis (FEA) with artificial neural networks (ANNs) to extract the stress-strain relationship embedded within force-displacement measurements. In the context of elasticity imaging, ANNs characterizing the stress-strain behavior of soft tissues can be interrogated to infer material parameters that best summarize the learned mechanical properties.

Many of the research articles that employed AutoP have described the training process, but none have directly compared and contrasted its operation with the more typical model-based approaches. In this review, we aim to more thoroughly examine the operating principles behind AutoP to better highlight the fundamental differences when compared to model-based inverse methods. Of particular importance is the manner in which physical principles are directly exploited in AutoP to generate increasingly accurate estimates of stress and strain distributions from force-displacement measurements under arbitrary geometry and loading. We believe the working principles of AutoP can be applied as a data-driven, physics-guided alternative to other difficult boundary value inverse problems.

Section 2 reviews model-based inverse methods in QUSE to establish a basis for comparison. We focus on iterative methods utilizing FEA as the forward solver to keep the discussion concise and comparable to AutoP. Section 3 reviews AutoP and builds intuition of its operation by drawing parallels to typical optimization methods. We also cover a novel neural network architecture developed for QUSE and a regularization term uniquely suited for AutoP. Challenges with non-linear viscoelasticity imaging and potential approaches with alternative ANN structures are briefly discussed in Section 4. Data acquisition and displacement estimation details are omitted from our discussions, but are available in [24, 25]. Even though such details are important in practice and will affect the efficacy of QUSE methods, we wish to compare the operation of AutoP and model-based methods at a more conceptual level.

## 2 Model-Based Inverse Methods

Reconstructing a map of material properties from force-displacement measurements is an ill-posed inverse problem.^{1} By combining a prior assumption of the underlying mechanical behavior with a system model incorporating physical principles, model-based inverse methods reduce the solution space and estimate parameter values of a pre-selected constitutive model. The goal of QUSE methods is to find the set of spatially distributed material parameters

where

where *α* is a hyperparameter that controls how strongly the regularization term affects the solution.

### 2.1 Finite Element Analysis

In modern methods, the forward problem is usually solved through finite element analysis (FEA). A brief description of the FEA formulation for solid mechanics follows; more thorough treatments can be found in [12, 18]. The strong form of the governing equations for a solid continuum under quasi-static loading is

where

The finite element formulation expresses the boundary-value problem as a system of equations,

where ^{2} We chose to express

FEA solutions of a mechanical system must satisfy both equilibrium and compatibility requirements. The former is specified by Eq. 8 and relates stresses to forces. Compatibility, on the other hand, relates displacements to strains and, in simple terms, precludes elements in the mesh from overlapping or separating during deformation. These physical principles combined with object geometry and a valid constitutive model determine the set of admissible deformations under loading (Figure 1A). The importance of these principles in solving the inverse problem will be made clear in the following sections.

**FIGURE 1**. **(A)** Summary of model-based inverse methods. Deformations computed in the forward problem are dependent on the estimated material properties and physical principles. Existing model-based QUSE methods that utilize force measurements use the data as input directly to the cost function or to calibrate the magnitude of the modulus estimates after the optimization procedure. **(B)** Illustration of AutoP training process. Applying force and displacement measurements in separate FEAs exploits equilibrium and compatibility requirements to generate physically consistent stresses and strains.

### 2.2 QUSE as an Optimization Problem

Gradient or Hessian-based optimization techniques are typically applied to solve Eq. 1, wherein the parameter estimates are iteratively updated through the gradient of the error computed in Eq. 2. The solution process for QUSE can be generalized to five steps:

The diagram in Figure 1A provides a simplified illustration of the role of FEA within an optimization-based solution method while simultaneously highlighting the fact that the cost function directly affects estimated parameter values. Optimization methods often have intuitive geometric interpretations that aid in understanding the solution procedure. For example, consider the method proposed by Kallel and Bertrand [29]. Define the operator

where

Adopting an iterative, Hessian-based solution procedure with *k* indicating the current iteration, the Young’s modulus values can be found,

Geometrically, the curvature of the forward operator (the Hessian

In this form, the operator gradient forming the sensitivity matrix

**Algorithm 1** Optimization-based QUSE Inverse Method.

[1] Apply displacement BCs at the mesh boundary

[2] Compute the displacements over the whole scan plane through FEA

[3] Compare the computed displacements to the measured values (i.e., evaluate Eq. 2)

[4] Update the parameter estimates based on the error

[5] If solution converges, exit, otherwise return to [1].

QUSE methods utilizing only displacement measurements can provide quantitative parameter estimates up to a multiplicative constant, unless the parameter value(s) is known for some portion of the boundary or interior [4, 5]. That is, through compatibility requirements embedded within FEA, at best the distribution of relative parameter values can be inferred from displacement BCs. Incorporating measurements of surface force or boundary stresses can provide the additional information necessary to estimate parameter values exactly [51]. This can be seen directly from Eqs. 5,8: if no values of reaction forces corresponding to displacement BCs are known, there exists no means to calibrate the magnitude of forces in

Force and displacement measurements coupled with FEA are insufficient for overcoming the ill-posed nature of the inverse problem, hence the inclusion of a regularization term in Eq. 2. The choice of regularization depends on some presumed property of the solution; e.g., Tikhonov regularization is founded in energy arguments,

Model-based QUSE methods are effective when the applied load and true tissue material properties match the selected constitutive model and loading geometry. Choosing the wrong constitutive model can cause artifacts in the resulting elasticity images that corrupts or discards clinically relevant diagnostic information not described by the selected model parameters. The limited flexibility of model-based methods makes them ineffective at *discovering* material properties. Overcoming limitations of model-based methods and inferring mechanical behavior from measurement data without constraints imposed by constitutive model assumptions may allow investigators to find new clinically relevant material properties.

## 3 The Autoprogressive Method

Contrary to model-based inverse methods, the goal of AutoP is to describe the mechanical behavior of a material non-parametrically. By replacing the classic mathematically defined constitutive model with an artificial neural network, any arbitrary stress-strain relationship can be captured. This formulation replaces parameters in Eq. 1 with the connection weights of one or more ANNs^{3}. The resulting, trained ANN describing the stress-strain behavior is referred to as a neural network constitutive model (NNCM). One example of a NNCM architecture is shown in Figure 2A (left). The NNCM displayed is a feed-forward, fully connected ANN that accepts a strain vector as input and returns a stress vector at the output. Alternative network architectures have been proposed by other investigators, but this form is the basis for most AutoP applications thus far. Even considering the various NNCM architectures utilized within AutoP, to the best of our knowledge all have comprised at most two hidden layers with up to a few dozen nodes each. Compared to deep-learning methods, NNCMs are very small ANNs.

**FIGURE 2**. **(A)** A material property network **(left)** is the basis for many NNCM architectures previously used in AutoP. The combination of a spatial network **(right)** operating in tandem with a MPN is referred to as a CaNNCM. **(B)** Illustration of the self-learning property of AutoP.

Existing efforts to build model-free descriptions of material behavior typically rely on experimental test data acquired from a sample in a well-defined loading scenario, such as strip extensiometry, to obtain stress-strain data from force-displacement measurements. Recent investigations in data-driven mechanics use this type of measurement data directly (i.e., no NNCM) to solve computational mechanics problems [31, 32, 37, 50]. Alternatively, stress-strain measurement data can be used to train NNCMs for later use in computational problems (see reviews [14, 15, 44, 55]). More recently, researchers have developed physics-informed deep neural networks (PINNs) that embed physical laws into the cost function used in backpropagation-based training [40].

Some measure of success has been achieved with each of these methods; however, the resulting soft-computational constitutive model, in whichever applicable form, is fully defined by the measurement data acquired from a homogeneous sample. That is, the models are trained once and deployed. When used for inference against input data not within the domain of the training set, the solution becomes unreliable, analogous to modeling errors in model-based inverse methods. Furthermore, these methods rely on the existence of stress-strain training data. Unfortunately, the complexity of soft tissues precludes excision of geometrically precise, homogeneous samples for measurement in well-defined loading scenarios. Nor do *ex vivo* load tests provide a good measure of soft tissue properties *in vivo*. In short, current data-driven methods fail to extract relevant information from the available force-displacement measurements acquired in a normal clinical setting.

The Autoprogressive Method [17] circumvents limitations with model-based and other data-driven methods through a tight coupling of NNCMs with FEA and knowledge of both internal and external object geometry. It has been successfully used to build soft-computational mechanical models of many different materials in a multitude of loading scenarios, including soils [19, 20], concrete [27, 28], steel [30, 57], red blood cells [48, 49], and the cornea [16]. AutoP has even been applied to thermal constitutive models [1]. More recently, we have been developing an adaptation of AutoP for QUSE using both boundary and interior displacement measurements [22]. The caveat that internal geometry must be known precludes most clinical imaging applications and is addressed by introducing a new type of NNCM, discussed in Section 3.1.

AutoP is a method for generating information from measurements. When applied to inverse problems in continuum mechanics, the goal is to recover the stress and strain fields induced by measured forces and displacements in a model-free way. Replacing the mathematical constitutive model with an ANN is paramount to provide the flexibility inherent to said networks, but alone is not a sufficient strategy. The two key components that make AutoP work are 1) the application of force and displacement BCs in separate FEAs—FEA_{σ} and FEA_{ε}—to obtain physically consistent estimates of stresses and strains through known physical principles and 2) NNCMs participate in the generation of their own training data by acting as the material model in the assembly of the system of equations in Eq. 6. The former point is analogous to model-based inverse methods wherein the parameter estimates in the current iteration affect the displacements computed in the forward problem. For clarity, the FEA equations can be rewritten as

We introduce the operator

Imposing measured forces and displacements in separate FEAs exploits stress equilibrium and compatibility requirements. In FEA_{σ}, measured forces are applied as force BCs ^{4}. Stress equilibrium, defined in Eq. 3, relates forces to stresses and we therefore claim the stresses _{σ} are physically consistent approximations of the true stress field. In a separate FEA_{ε}, corresponding displacement measurements are applied as displacement BCs _{ε} are physically consistent approximations of the true strain field. The AutoP training process is summarized in Figure 1B and can be broken into the five steps listed in Algorithm 2.

**Algorithm 2** AutoP Training of NNCMs.

[0] Linear-elastic NNCM pretraining

[1] Apply force BCs in FEA_{σ}, compute

[2] Apply displacement BCs in FEA_{ε}, compute

[3] Train NNCM(s)

[4]If displacement error converged, exit, otherwise return to [1].

Linear-elastic pretraining is used only to provide NNCMs with a physically consistent starting state. Unlike model-based reconstruction methods that can be sensitive to parameter initialization [4], investigations with AutoP have demonstrated that linear-elastic pretraining does not constrain the mechanical properties learned by the NNCMs.

Convergence in AutoP is determined by comparing the values of a cost function to predefined limits. In most AutoP applications thus far, the cost function has assumed the form [17].

Aside from the use of the _{σ} and FEA_{ε}, respectively, adjust the connection weights so that the material properties described by the NNCMs more accurately resemble the true material properties. We also note that the because the cost function does not directly influence NNCM training, adjustments to

It is not immediately clear why the combination of the stresses and strains from FEAs solved under separate, but conjugate, boundary conditions should lead the ANNs to the correct material properties. We will demonstrate that it is in fact an implicit displacement error that guides the learning process. Define _{σ}, including any imposed BCs. Likewise, _{ε}. By definition of a NNCM,

To better understand this interaction, assume the NNCM describes a linear-elastic material so that _{σ} as _{ε} are supplied as input to the network. Training a NNCM with a backpropagation-based method consists of minimizing a cost function, in this case

where _{σ} and FEA_{ε} whereas

If we consider the first iteration of training when

and reverts back to Eq. 24 after the first training iteration and the NNCM weights

The preceding argument does not imply NNCMs trained in AutoP are limited to learning linear-elastic material properties. Rather, the case of linear materials was chosen as an example to best highlight the effect of displacement error in NNCM training. We emphasize that, similar to model-based methods, the displacement error is a driving force in the solution of the inverse problem as approached through AutoP. However, in the case of AutoP, the effect of displacement error emerges from the inconsistency between stresses and strains computed in FEA_{σ} and FEA_{ε}. It is a characteristic that arises out of the equilibrium and compatibility requirements rather than an explicitly defined error measure to direct the parameter search.

Much like optimization-based methods, the parameter estimates in the current iteration—the weights of the ANNs—affect the solution of the forward problem. Moreover, the error when comparing the forward problem solution to experimental measurements affects the parameter updates. What sets AutoP apart from typical model-based and machine learning techniques is its self-learning paradigm guided by physical principles. When applied to QUSE, the result is a patient-specific soft-computational material model. Figure 2B aims to illustrate the self-learning property of AutoP. Imposing measurement data as BCs in separate FEAs forces the material properties learned by the ANNs to satisfy physical principles and be self-consistent.

### 3.1 Cartesian NNCMs

The discussion of AutoP thus far has assumed that the finite element mesh conformed to both internal and external object geometry with a different NNCM assigned for each unique material. Eq. 18 implies this fact in that *a priori* nor segmented into discrete regions with homogeneous material properties. As a consequence, the goal of AutoP must be adjusted to infer both material properties and geometry from the set of force-displacement measurements.

Two significant changes were made toward this goal. First, the structure of the NNCM was changed by adding a second network that interacted with the existing NNCM and was responsible for learning spatial information. We deemed these networks Cartesian neural network constitutive models (CaNNCMs) [23–25]. The CaNNCM architecture is illustrated in Figure 2A. To identify the individual subnetworks, we refer to the one which learns the “average” stress-strain relationship of the entire object as the material property network (MPN, Figure 2A left) and the network that maps a Cartesian coordinate input to a spatially varying value as the spatial network (SN, Figure 2A right). A thorough description of CaNNCM theory of operation can be found in [23, 24]. Unlike NNCMs previously utilized with AutoP, a single CaNNCM is capable of characterizing the heterogeneous material properties of an entire object. The FEA equations are adjusted slightly to accommodate this change:

Spatial information is encoded within a set of spatial scaling values

**Algorithm 3** AutoP Training of CaNNCMs.

[0] Pretrain CaNNCM as homogeneous, linear-elastic material

[1] Apply force BCs in FEA_{σ}, compute

[2] Apply displacement BCs in FEA_{ε}, compute

[3] Train MPN

[4a] Update

[4b] Train SN

[5]If displacement error converged, exit, otherwise back to [1].

CaNNCMs work well in QUSE because information pertaining to the interior geometry can be gleaned from internal displacement measurements available through US imaging. In unpublished work, we found that CaNNCMs trained in AutoP using only boundary information were unable to recover internal structure away from the surface. Part of the shortcoming is due to the lack of surface force distribution measurements. Rather, the force measured experimentally is the sum of forces over the face of the US transducer that provides information about the mean stress distribution over the surface. Coupling these boundary data with prior knowledge of internal geometry is sufficient for a set of NNCMs to learn the heterogeneous material properties of an object. Therefore, in order for CaNNCMs to learn both material properties and geometry, some measurement data containing spatial information must be provided.

Interestingly, we found that incorporating internal displacements in FEA_{ε} was insufficient for accurately estimating material property distributions, particularly when measurements were acquired by loading from a single direction [25]. The reason can be seen from Eq. 27. By making the stress term explicitly dependent on spatial position, additional force measurements populating

An alternative strategy is to add a regularization term to augment the available information. Our solution was to add additional physical constraints during the calculation of

The purpose of the spatial values is to further reduce the error in Eq. 21 by storing the information not captured by the spatially independent MPN weights.

Implementing Eq. 28 in AutoP as defined will lead to erroneous material property distribution estimates. Perhaps the most intuitive explanation starts by noting the CaNNCM is pretrained as a homogeneous, linear-elastic material. Then, when computing spatial values in Step 4a, the _{σ} contain no spatial information.

We introduced *σ*-matching regularization to inject spatial information into Eq. 28 by imposing the physical constraint that conjugate force-displacement BCs imposed in separate FEAs should produce the same stress fields,

Note that the stress terms are constant, hence the multiplication by *σ*-matching term, we can show that the displacement error is influencing the result:

Adding the regularization term to the cost function for computing

A significant difference between regularization terms typically used in model-based inverse methods and 33) is the appearance of *σ*-matching within the *σ*-matching term to the existing cost function would have no effect on the computed spatial values. Adding *σ*-matching as we have in Eq. 33 can be interpreted as adjusting the (constant) stresses _{σ} with spatial information encoded within displacement measurements through the displacement error.

We demonstrated in [25] the significant improvement in material property estimates when *σ*-matching was incorporated into AutoP training, even with a relatively sparse set of force-displacement measurements. Contrary to typical forms of regularization, *σ*-matching is based on a physical principle that will always be true, not a prior assumption about the property the solution should exhibit. Furthermore, it is uniquely suited to AutoP because of the way equilibrium and compatibility requirements are exploited by FEA_{σ} and FEA_{ε}. Other constraints on physical properties will likely be incorporated into AutoP as CaNNCMs are further developed for imaging the non-linear and time-dependent properties of soft tissues.

## 4 Non-linear Elasticity Imaging with AutoP

The preceding discussion of NNCMS and AutoP was limited to objects under loads inducing small deformation. Mathematically, the strain tensor in the limit of infinitesimal deformation is expressed as

Probing the non-linear mechanical properties of deep structures in QUSE requires application of large loads to sufficiently deform distal tissues. In these scenarios, a large deformation framework must be invoked wherein the strain tensor includes geometrically non-linear terms. As an example, one potential definition of geometrically non-linear strain is

The problem then arises of selecting an appropriate non-linear and/or viscoelastic constitutive model that describes tissue behavior and maximizes the diagnostic information in QUSE images. Complexity is further increased when considering the likelihood that different tissues or tissue states (i.e., healthy tissue, benign lesions, and malignant tumors) are best described by different material models. Current model-based approaches assume homogeneity of the underlying constitutive model for the entire imaged tissue section and estimate the spatial variation of the corresponding model parameters. There exists no model-based QUSE inverse method capable of *discovering* the set of constitutive models—and estimating values of the corresponding parameters—that adequately describes a heterogeneous group of tissues. The combination of AutoP with CaNNCMs can fill this void.

AutoP has been applied to build soft-computational models of complex materials undergoing small and large deformation when the total geometry was known (e.g. [19, 27, 28, 30, 45, 58]). To achieve non-linear, time-dependent material property imaging with AutoP, CaNNCMs must be modified to 1) accommodate geometrically non-linear deformation to facilitate non-linear elasticity imaging and 2) incorporate additional stress-strain state information as network input to properly capture a heterogeneous distribution of complex mechanical behaviors. Prior successes with AutoP lead us to believe there exists an adaptation to the CaNNCM architecture and/or AutoP that will accomplish both tasks.

The difficulty of this problem can be understood by comparing the information required for a NNCM to compute a stress vector for materials of varying complexity. For linear-elastic materials, the NNCM output is defined as

Projecting further, adjustments to the CaNNCM architecture will also be necessary to characterize a heterogeneous distribution of non-linear, viscoelastic materials. If the operating principle of linear-elastic CaNNCMs is retained, the stress-strain response would take the form

## 5 Discussion

Quantitative elasticity imaging is a challenging problem. Much of the difficulty arises from the limited measurement data that can be acquired during a clinical US imaging exam. Model-based inverse methods address the ill-posed inverse problem by constraining the solutions through geometric and constitutive model assumptions, well-defined force loads, and application of regularization to the solution. While effective, deviation of measurement data from the model assumptions can introduce large artifacts in the reconstructed material property images, potentially corrupting diagnostic information, and the regularization method is almost never grounded in physical laws. Furthermore, parameters associated with the pre-selected constitutive model may only capture a portion of the tissue mechanical properties, missing information that could be important to clinical decision making.

AutoP and NNCMs are a fundamentally different approach to the inverse problem in elasticity imaging. Rather than constraining the solution through model assumptions, AutoP extracts material property information from force-displacement measurements through a combination of physical modeling and machine learning. In this review, we presented AutoP in a new way to highlight similarities of the procedure with well-known optimization methods. Specifically, we demonstrated how the principles of stress equilibrium and strain compatibility are exploited through application of measurement data as BCs in separate FEAs, from which naturally arises an implicit displacement error that drives NNCM training.

The primary benefit of AutoP over model-based inverse methods is the potential to *discover* 3-D mechanical properties of soft tissues *in vivo*. Full computational modeling of applied force loads means measurements need not be acquired in a constrained manner; e.g., uniaxial compression by a large platen. Nor is 3-D displacement data necessary to reconstruct material properties throughout a volume, as we demonstrated in [25]. However, the use of arbitrary force-displacement measurements as input to AutoP significantly increases the computational load. Furthermore, we expect intelligent sampling strategies will need to be developed in order to maximize the information content of measurement data and reduce the computational burden of AutoP. Good sampling strategies will likely be essential for effectively imaging non-linear, time-dependent material properties.

One particularly exciting prospect of CaNNCMs and AutoP is the ability to identify mechanical properties from a heterogeneous distribution of material types. Classic model-based inverse methods must first assume the form of the constitutive model for the entire tissue section. The solution is the spatial variation of corresponding parameters that best fits the measurement data. AutoP reverses this order to first build a non-parametric, soft-computational model that describes the spatially varying material properties encoded in force-displacement data. Parameter estimation occurs *after* relevant information has been extracted from measurements. Successfully developing this capability would represent a paradigm shift in not only QUSE, but inverse identification of material properties in general.

AutoP is at its core a method to solve boundary value inverse problems. It is naturally suited for elasticity imaging because the physical principles of stress equilibrium and strain compatibility are built into the forward model. These principles relate the available force-displacement measurement data to the desired stress and strain distributions. By combining the physical principles through forward modeling with the flexibility of ANNs, virtually any type of continuum material can be described non-parametrically. AutoP could be applicable to other inverse problems with well-defined physical laws relating measurement data to the desired quantities. Nevertheless, AutoP is just one example of integrating physical modeling with machine learning to build powerful methods of solving inverse problems.

## Author Contributions

The manuscript was written by CH with substantial intellectual input from each author.

## Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

## Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

## Acknowledgments

Research reported in this publication was supported by NCI and NIBIB of the National Institutes of Health under Award Numbers R01 CA168575 and R21 EB023402. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

## Footnotes

^{1}Inverse problems and image science are expansive fields of study. Some foundational textbooks with comprehensive coverage of these topics are [7, 8].

^{2}Only the linear form of the FEA is expressed in Eq. 6 for clarity. When the system includes non-linear materials or deformation, a Taylor expansion of Eq. 6 reformulates the problem in an iterative, incremental form.

^{3}Node biases would also comprise the set of parameters if included in the network architecture.

^{4}There exist certain geometric displacement BCs that are applied in all FEAs. For our examples, such BCs are the bottom surface of the scanned object being “pinned” and having zero displacement

## References

1. Aquino W, Brigham JC. Self-learning Finite Elements for Inverse Estimation of thermal Constitutive Models. *Int J Heat Mass Transfer* (2006) 49:2466–78. doi:10.1016/j.ijheatmasstransfer.2006.01.031

2. Baldewsing RA, Danilouchkine MG, Mastik F, Schaar JA, Serruys PW, van der Steen AFW. An Inverse Method for Imaging the Local Elasticity of Atherosclerotic Coronary Plaques. *IEEE Trans Inform Technol Biomed* (2008) 12:277–89. doi:10.1109/titb.2007.907980

3. Baldewsing RA, Schaar JA, Mastik F, van der Steen AFW. Local Elasticity Imaging of Vulnerable Atherosclerotic Coronary Plaques. In Atherosclerosis, Large Arteries and Cardiovascular Risk. Karger Publishers (2007). p. 35–61. doi:10.1159/000096719

4. Barbone PE, Bamber JC. Quantitative Elasticity Imaging: what Can and Cannot Be Inferred from Strain Images. *Phys Med Biol* (2002) 47:2147–64. doi:10.1088/0031-9155/47/12/310

5. Barbone PE, Gokhale NH. Elastic Modulus Imaging: on the Uniqueness and Nonuniqueness of the Elastography Inverse Problem in Two Dimensions. *Inverse Probl* (2004) 20:283–96. doi:10.1088/0266-5611/20/1/017

6. Barr RG, Destounis S, Lackey LB, Svensson WE, Balleyguier C, Smith C. Evaluation of Breast Lesions Using Sonographic Elasticity Imaging. *J Ultrasound Med* (2012) 31:281–7. doi:10.7863/jum.2012.31.2.281

9. Bhatia KSS, Tong CSL, Cho CCM, Yuen EHY, Lee YYP, Ahuja AT. Shear Wave Elastography of Thyroid Nodules in Routine Clinical Practice: Preliminary Observations and Utility for Detecting Malignancy. *Eur Radiol* (2012) 22:2397–406. doi:10.1007/s00330-012-2495-1

10. Bojunga J, Herrmann E, Meyer G, Weber S, Zeuzem S, Friedrich-Rust M. Real-time Elastography for the Differentiation of Benign and Malignant Thyroid Nodules: a Meta-Analysis. *Thyroid* (2010) 20:1145–50. doi:10.1089/thy.2010.0079

11. Dahl JJ, Dumont DM, Allen JD, Miller EM, Trahey GE. Acoustic Radiation Force Impulse Imaging for Noninvasive Characterization of Carotid Artery Atherosclerotic Plaques: a Feasibility Study. *Ultrasound Med Biol* (2009) 35:707–16. doi:10.1016/j.ultrasmedbio.2008.11.001

12. De Borst R, Crisfield MA, Remmers JJ, Verhoosel CV. *Nonlinear Finite Element Analysis of Solids and Structures*. John Wiley & Sons (2012).

13. Doyley MM. Model-based Elastography: a Survey of Approaches to the Inverse Elasticity Problem. *Phys Med Biol* (2012) 57:R35–R73. doi:10.1088/0031-9155/57/3/r35

14. Ghaboussi J. Advances in Neural Networks in Computational Mechanics and Engineering. In Advances of soft computing in engineering. Springer (2010). p. 191–236. doi:10.1007/978-3-211-99768-0_4

16. Ghaboussi J, Pecknold D, Hashash Y, Kwon T. Accurate Determination of Intraocular Pressure and Characterization of Mechanical Properties of the Cornea. *US Patent* (2011) 8(070):679.

17. Ghaboussi J, Pecknold DA, Zhang M, Haj-ali RM. Autoprogressive Training of Neural Network Constitutive Models. *Int J Numer Meth Engng* (1998) 42:105–26. doi:10.1002/(sici)1097-0207(19980515)42:1<105::aid-nme356>3.0.co;2-v

19. Hashash YMA, Marulanda C, Ghaboussi J, Jung S. Systematic Update of a Deep Excavation Model Using Field Performance Data. *Comput Geotechnics* (2003) 30:477–88. doi:10.1016/s0266-352x(03)00056-9

20. Hashash YM, Ghaboussi J, Fu Q, Marulanda C (2006). Constitutive Soil Behavior Representation via Artificial Neural Networks: a Shift from Soil Models to Soil Behavior Data. In GeoCongress 2006: Geotechnical Engineering in the Information Technology Age. 1–6.

21. Hashash YMA, Jung S, Ghaboussi J. Numerical Implementation of a Neural Network Based Material Model in Finite Element Analysis. *Int J Numer Meth Engng* (2004) 59:989–1005. doi:10.1002/nme.905

22. Hoerig C, Ghaboussi J, Insana MF. An Information-Based Machine Learning Approach to Elasticity Imaging. *Biomech Model Mechanobiol* (2017) 16:805–22. doi:10.1007/s10237-016-0854-6

23. Hoerig C, Ghaboussi J, Insana MF. *Cartesian Neural Network Constitutive Models for Data-Driven Elasticity Imaging* (2018). arXiv preprint arXiv:1809.04121.

24. Hoerig C, Ghaboussi J, Insana MF. Data-driven Elasticity Imaging Using Cartesian Neural Network Constitutive Models and the Autoprogressive Method. *IEEE Trans Med Imaging* (2019) 38:1150–60. doi:10.1109/tmi.2018.2879495

25. Hoerig C, Ghaboussi J, Insana MF. Physics-guided Machine Learning for 3-D Quantitative Quasi-Static Elasticity Imaging. *Phys Med Biol* (2020) 65:065011. doi:10.1088/1361-6560/ab7505

26. Huwart L, Peeters F, Sinkus R, Annet L, Salameh N, ter Beek LC, et al. Liver Fibrosis: Non-invasive Assessment with Mr Elastography. *NMR Biomed* (2006) 19:173–9. doi:10.1002/nbm.1030

27. Jung S, Ghaboussi J. Characterizing Rate-dependent Material Behaviors in Self-Learning Simulation. *Comp Methods Appl Mech Eng* (2006) 196:608–19. doi:10.1016/j.cma.2006.06.006

28. Jung S, Ghaboussi J. Inverse Identification of Creep of concrete from *In Situ* Load-Displacement Monitoring. *Eng Structures* (2010) 32:1437–45. doi:10.1016/j.engstruct.2010.01.022

29. Kallel F, Bertrand M. Tissue Elasticity Reconstruction Using Linear Perturbation Method. *IEEE Trans Med Imaging* (1996) 15:299–313. doi:10.1109/42.500139

30. Kim J, Ghaboussi J, Elnashai AS. Hysteretic Mechanical-Informational Modeling of Bolted Steel Frame Connections. *Eng Structures* (2012) 45:1–11. doi:10.1016/j.engstruct.2012.06.014

31. Kirchdoerfer T, Ortiz M. Data Driven Computing with Noisy Material Data Sets. *Comp Methods Appl Mech Eng* (2017) 326:622–41. doi:10.1016/j.cma.2017.07.039

32. Ladevèze P, Néron D, Gerbaud P-W. Data-driven Computation for History-dependent Materials. *Comptes Rendus Mécanique* (2019) 347:831–44. doi:10.1016/j.crme.2019.11.008

33. Larin KV, Sampson DD. Optical Coherence Elastography - OCT at Work in Tissue Biomechanics [Invited]. *Biomed Opt Express* (2017) 8:1172–202. doi:10.1364/boe.8.001172

34. Liu B, Zheng Y, Huang G, Lin M, Shan Q, Lu Y, et al. Breast Lesions: Quantitative Diagnosis Using Ultrasound Shear Wave Elastography-A Systematic Review and Meta-Analysis. *Ultrasound Med Biol* (2016) 42:835–47. doi:10.1016/j.ultrasmedbio.2015.10.024

35. Mariappan YK, Glaser KJ, Ehman RL. Magnetic Resonance Elastography: a Review. *Clin Anat* (2010) 23:497–511. doi:10.1002/ca.21006

36. Nenadic IZ, Urban MW, Greenleaf JF, Gennisson J-L, Bernal M, Tanter M. *Ultrasound Elastography for Biomedical Applications and Medicine*. John Wiley & Sons (2019).

37. Nguyen LTK, Rambausek M, Keip M-A. Variational Framework for Distance-Minimizing Method in Data-Driven Computational Mechanics. *Comp Methods Appl Mech Eng* (2020) 365:112898. doi:10.1016/j.cma.2020.112898

38. Nightingale K. Acoustic Radiation Force Impulse (ARFI) Imaging: a Review. *Cmir* (2011) 7:328–39. doi:10.2174/157340511798038657

39. Parker KJ, Doyley MM, Rubens DJ. Imaging the Elastic Properties of Tissue: the 20 Year Perspective. *Phys Med Biol* (2010) 56:R1–R29. doi:10.1088/0031-9155/56/1/r01

40. Raissi M, Perdikaris P, Karniadakis GE. Physics-informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations. *J Comput Phys* (2019) 378:686–707. doi:10.1016/j.jcp.2018.10.045

41. Righetti R, Kallel F, Stafford RJ, Price RE, Krouskop TA, Hazle JD, et al. Elastographic Characterization of HIFU-Induced Lesions in Canine Livers. *Ultrasound Med Biol* (1999) 25:1099–113. doi:10.1016/s0301-5629(99)00044-7

42. Rus Carlborg G, Faris I, Torres J, Callejas A, Melchor J, et al. Why Are Viscosity and Nonlinearity Bound to Make an Impact in Clinical Elastographic Diagnosis? *Sensors* (2020) 20. doi:10.3390/s20082379

43. Scherzer O, Grasmair M, Grossauer H, Haltmeier M, Lenzen F. *Variational Methods in Imaging*. Springer (2009).

44. Shahin MA, Jaksa MB, Maier HR. State of the Art of Artificial Neural Networks in Geotechnical Engineering. *Electron J Geotechnical Eng* (2008) 8:1–26.

45. Sidarta DE. *Neural Network-Based Constitutive Modeling of Granular Material*. Ph.D. thesis. University of Illinois at Urbana-Champaign (2001).

46. Singh MS, Thomas A. Photoacoustic Elastography Imaging: a Review. *J Biomed Opt* (2019) 24:1–15. doi:10.1117/1.jbo.24.4.040902

47. Sinkus R, Daire J-L, Vilgrain V, E. Van Beers B. Elasticity Imaging via MRI: Basics, Overcoming the Waveguide Limit, and Clinical Liver Results. *Cmir* (2012) 8:56–63. doi:10.2174/157340512799220544

48. Song H, Hashash YMA. Characterization of Stress-Strain Behaviour of Red Blood Cells (RBCs), Part I: Methodology Development and Application to Healthy RBCs. *Inverse Probl Sci Eng* (2015) 23:913–31. doi:10.1080/17415977.2014.954111

49. Song H., Hashash Y. M. A. Characterization of stress-strain behaviour of red blood cells (RBCs), part II: response of malaria-infected RBCs. *Inverse Problems in Science and Engineering* (2015b) 23:413–424. doi:10.1080/17415977.2014.922072

50. Stainier L, Leygue A, Ortiz M. Model-free Data-Driven Methods in Mechanics: Material Data Identification and Solvers. *Comput Mech* (2019) 64:381–93. doi:10.1007/s00466-019-01731-1

51. Tyagi M, Goenezen S, Barbone PE, Oberai AA. Algorithms for Quantitative Quasi-Static Elasticity Imaging Using Force Data. *Int J Numer Meth Biomed Engng* (2014) 30:1421–36. doi:10.1002/cnm.2665

52. Vappou J. Magnetic Resonance− and Ultrasound Imaging−Based Elasticity Imaging Methods: A Review. *Crit Rev Biomed Eng* (2012) 40:121–34. doi:10.1615/critrevbiomedeng.v40.i2.30

53. Varghese T, Shi H. Elastographic Imaging of thermal Lesions in Liver *In-Vivo* Using Diaphragmatic Stimuli. *Ultrason Imaging* (2004) 26:18–28. doi:10.1177/016173460402600102

54. Xu J-M, Xu X-H, Xu H-X, Zhang Y-F, Zhang J, Guo L-H, et al. Conventional US, US Elasticity Imaging, and Acoustic Radiation Force Impulse Imaging for Prediction of Malignancy in Thyroid Nodules. *Radiology* (2014) 272:577–86. doi:10.1148/radiol.14132438

55. Yagawa G, Okuda H. Neural Networks in Computational Mechanics. *Arco* (1996) 3:435–512. doi:10.1007/bf02818935

56. Yeh W-C, Li P-C, Jeng Y-M, Hsu H-C, Kuo P-L, Li M-L, et al. Elastic Modulus Measurements of Human Liver and Correlation with Pathology. *Ultrasound Med Biol* (2002) 28:467–74. doi:10.1016/s0301-5629(02)00489-1

57. Yun GJ, Ghaboussi J, Elnashai AS. Development of Neural Network Based Hysteretic Models for Steel Beam-Column Connections through Self-Learning Simulation. *J Earthquake Eng* (2007) 11:453–67. doi:10.1080/13632460601123180

Keywords: Inverse problem, elasticity, ultrasound, finite element analysis, neural network constitutive model

Citation: Hoerig C, Ghaboussi J, Wang Y and Insana MF (2021) Machine Learning in Model-free Mechanical Property Imaging: Novel Integration of Physics With the Constrained Optimization Process. *Front. Phys.* **9**:600718. doi: 10.3389/fphy.2021.600718

Received: 30 August 2020; Accepted: 29 June 2021;

Published: 30 July 2021.

Edited by:

Jing Guo, Charité–Universitätsmedizin Berlin, GermanyReviewed by:

Jean-Paul Mazellier, Institut de Chirurgie Guidée par l’image de Strasbourg, FranceSimon Chatelin, UMR7357 Laboratoire des sciences de l'Ingénieur, de l'Informatique et de l'Imagerie (ICube), France

Copyright © 2021 Hoerig, Ghaboussi, Wang and Insana. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Cameron Hoerig, choerig@riversideresearch.org