<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article article-type="research-article" xmlns:xlink="http://www.w3.org/1999/xlink">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Pain Res.</journal-id>
<journal-title>Frontiers in Pain Research</journal-title><abbrev-journal-title abbrev-type="pubmed">Front. Pain Res.</abbrev-journal-title>
<issn pub-type="epub">2673-561X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fpain.2022.966034</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Pain Research</subject>
<subj-group>
<subject>Hypothesis and Theory</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>A Bayesian model for chronic pain</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes"><name><surname>Eckert</surname><given-names>Anna-Lena</given-names></name>
<xref ref-type="corresp" rid="cor1">&#x002A;</xref><uri xlink:href="https://loop.frontiersin.org/people/1856381/overview"/></contrib>
<contrib contrib-type="author"><name><surname>Pabst</surname><given-names>Kathrin</given-names></name><uri xlink:href="https://loop.frontiersin.org/people/1942524/overview" /></contrib>
<contrib contrib-type="author"><name><surname>Endres</surname><given-names>Dominik M.</given-names></name><uri xlink:href="https://loop.frontiersin.org/people/55148/overview" /></contrib>
</contrib-group>
<aff><addr-line>Theoretical Cognitive Science Group, Department of Psychology</addr-line>, <institution>Philipps-University Marburg</institution>, <addr-line>Marburg</addr-line>, <country>Germany</country></aff>
<author-notes>
<fn fn-type="edited-by"><p><bold>Edited by:</bold> Lejian Huang Northwestern University, United States</p></fn>
<fn fn-type="edited-by"><p><bold>Reviewed by:</bold> William Sedley Newcastle University, United Kingdom Daniel S. Barron Brigham and Women&#x0027;s Hospital and Harvard Medical School, United States</p></fn>
<corresp id="cor1"><label>&#x002A;</label><bold>Correspondence:</bold> Anna-Lena Eckert <email>eckertan@staff.uni-marburg.de</email></corresp>
<fn fn-type="other" id="fn001"><p><bold>Specialty Section:</bold> This article was submitted to Pain Research Methods, a section of the journal Frontiers in Pain Research</p></fn>
</author-notes>
<pub-date pub-type="epub"><day>16</day><month>09</month><year>2022</year></pub-date>
<pub-date pub-type="collection"><year>2022</year></pub-date>
<volume>3</volume><elocation-id>966034</elocation-id>
<history>
<date date-type="received"><day>13</day><month>06</month><year>2022</year></date>
<date date-type="accepted"><day>25</day><month>08</month><year>2022</year></date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2022 Eckert, Pabst and Endres DM.</copyright-statement>
<copyright-year>2022</copyright-year><copyright-holder>Eckert, Pabst and Endres</copyright-holder><license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution License (CC BY)</ext-link>. The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license>
</permissions>
<abstract>
<p>The perceiving mind constructs our coherent and embodied experience of the world from noisy, ambiguous and multi-modal sensory information. In this paper, we adopt the perspective that the experience of pain may similarly be the result of a probabilistic, inferential process. Prior beliefs about pain, learned from past experiences, are combined with incoming sensory information in a Bayesian manner to give rise to pain perception. Chronic pain emerges when prior beliefs and likelihoods are biased towards inferring pain from a wide range of sensory data that would otherwise be perceived as harmless. We present a computational model of interoceptive inference and pain experience. It is based on a Bayesian graphical network which comprises a hidden layer, representing the inferred pain state; and an observable layer, representing current sensory information. Within the hidden layer, pain states are inferred from a combination of priors <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM1"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, transition probabilities between hidden states <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM2"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and likelihoods of certain observations <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM3"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. Using variational inference and free-energy minimization, the model is able to learn from observations over time. By systematically manipulating parameter settings, we demonstrate that the model is capable of reproducing key features of both healthy- and chronic pain experience. Drawing on mathematical concepts, we finally simulate treatment resistant chronic pain and discuss mathematically informed treatment options.</p>
</abstract>
<kwd-group>
<kwd>chronic pain</kwd>
<kwd>interoception</kwd>
<kwd>graphical models</kwd>
<kwd>computational psychiatry</kwd>
<kwd>Bayesian inference</kwd>
<kwd>belief-propagation</kwd>
<kwd>predictive coding</kwd>
<kwd>free energy</kwd>
</kwd-group>
<contract-sponsor id="cn001">Excellence Program of the Hessian Ministry for Science and the Arts</contract-sponsor>
<counts>
<fig-count count="6"/>
<table-count count="1"/><equation-count count="296"/><ref-count count="84"/><page-count count="0"/><word-count count="0"/></counts>
</article-meta>
</front>
<body>
<sec id="s2" sec-type="intro"><label>1.</label><title>Introduction</title>
<p>Pain has an undeniable function in daily life. By signalling potential or actual bodily damage and directing both attention and behavior, it ensures the individual&#x2019;s physical integrity in the long run (<xref ref-type="bibr" rid="B1">1</xref>). When touching a hot plate, for example, the individual will withdraw their hand immediately to avoid further tissue damage, and seek to soothe the acquired damage.</p>
<p>The same can not be said about chronic pain, where pain persists beyond the presence of acute injury (<xref ref-type="bibr" rid="B1">1</xref>). With around 20&#x0025; of adults and children reporting symptoms of chronic pain, it is a highly prevalent disorder (<xref ref-type="bibr" rid="B2">2</xref>, <xref ref-type="bibr" rid="B3">3</xref>). Chronic pain leads to significant losses of quality of life (adults: (<xref ref-type="bibr" rid="B4">4</xref>), adolescents: (<xref ref-type="bibr" rid="B5">5</xref>)). The disorder is associated with decreased levels of psycho-social functioning and productivity (<xref ref-type="bibr" rid="B6">6</xref>) and frequent utilization of healthcare services (<xref ref-type="bibr" rid="B7">7</xref>), causing a substantial economic burden (<xref ref-type="bibr" rid="B8">8</xref>&#x2013;<xref ref-type="bibr" rid="B10">10</xref>).</p>
<p>Despite the ever-rising prevalence of chronic pain (<xref ref-type="bibr" rid="B11">11</xref>, <xref ref-type="bibr" rid="B12">12</xref>), its underlying mechanisms remain poorly understood. Computational-level mathematical models (cmp. (<xref ref-type="bibr" rid="B13">13</xref>)) can help to elucidate the problem that the mind is trying to solve (e.g., detecting harm to the body) and where such solution attempts go wrong. Recent developments in cognitive computational neuroscience and machine learning may offer new opportunities for studying chronic pain from this statistical viewpoint.</p>
<p>A growing body of work suggests interoception, just like perception, follows probabilistic, Bayesian principles (<xref ref-type="bibr" rid="B14">14</xref>&#x2013;<xref ref-type="bibr" rid="B18">18</xref>). Perceiving both the external world and the internal milieu requires effectively dealing with noisy, ambiguous and incomplete sensory information, usually from a multitude of sources and modalities. For this, the mind appears to rely on prior knowledge in the shape of a generative model of the environment, which is integrated with the current sensory input (<xref ref-type="bibr" rid="B19">19</xref>&#x2013;<xref ref-type="bibr" rid="B21">21</xref>). Following this account, perception is not only shaped by the sensory information reaching the mind or body, but also by the predictions of the generative model this information is met with. This notion is summarized under the free-energy principle (<xref ref-type="bibr" rid="B22">22</xref>, <xref ref-type="bibr" rid="B23">23</xref>).</p>
<p>The modulatory effects of prior knowledge on pain perception have been widely demonstrated - from the level of neural processing in the brain stem (<xref ref-type="bibr" rid="B24">24</xref>) through to self-reported pain intensity (<xref ref-type="bibr" rid="B25">25</xref>&#x2013;<xref ref-type="bibr" rid="B28">28</xref>). For example, Tabor et al. (<xref ref-type="bibr" rid="B16">16</xref>) show that pain perception varies in dependence of an exteroceptive cue. When paired with a blue light (signalling safety), a noxious stimulus is perceived as less painful compared to when it is paired with a red light (signalling danger and heat). Further, placebo analgesia describes the phenomenon of experiencing pain relief after taking a placebo pill (<xref ref-type="bibr" rid="B15">15</xref>, <xref ref-type="bibr" rid="B29">29</xref>, <xref ref-type="bibr" rid="B30">30</xref>). In line with statistical accounts of pain, this further suggests an integration of expectations, or prior beliefs, with the physical sensory signal. In this context, Hechler and colleagues (2016) have instantiated Bayes&#x2019; theorem with pain and nociception-related variables to illustrate statistical computations underlying pain perception:<disp-formula id="disp-formula1"><label>(1)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM1"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mspace width="thinmathspace"/><mml:mo>&#x2217;</mml:mo><mml:mspace width="thinmathspace"/><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM4"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is the <bold>posterior</bold> probability. The posterior determines the individual&#x2019;s subjective experience of pain. It is proportional to the <bold>likelihood</bold> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM5"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> times <bold>prior</bold> probability of pain, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM6"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM7"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> describes the probability of being exposed to a certain sensation (i.e., nociceptive signals) when the body or a specific body part is in a state of pain. Following this account, chronic pain has been described in terms of aberrant Bayesian inference (<xref ref-type="bibr" rid="B14">14</xref>). For example, averse life events may lead to a heightened prior expectation of pain, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM8"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. This may underlie lowered pain thresholds and hyperalgesia frequently observed in chronic pain patients (<xref ref-type="bibr" rid="B31">31</xref>). Long-term learning experiences may alter the associations between sensory information and the inferred experience in patients. In other words, the likelihood term <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM9"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> may be subject to learned biases in patients so that a broader range of stimuli is associated with pain compared to healthy controls (<xref ref-type="bibr" rid="B14">14</xref>). This may underlie allodynia, where harmless stimuli are perceived as painful.</p>
<p>We build on these previous lines of work and propose a quantitative Bayesian framework, implementing the above-mentioned considerations within a hierarchical and sequential model. It is built on the assumptions that (i) pain perception emerges from Bayes-optimal combination of sensory input and prior beliefs, (ii) chronic pain is characterized by maladaptive learning processes on longer time-scales, (iii) these learning processes result in generalizing, heightened expectations of pain <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM10"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi mathvariant="normal">pain</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> as well as (iv) heightened <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM11"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, i.e. patients erroneously infer being in pain as the most likely cause of a wide range of sensations usually considered harmless.</p>
<p>Our formal implementation is based on the assumption that interoception is an inference problem that requires optimizing the generative models, i.e. the mental representation of causal relations between body state (i.e. a pain-free or pain state) and sensory (e.g. harmless or noxious) inputs. Using concepts from machine learning, such as variational inference, the system&#x2019;s state is approximated (<xref ref-type="bibr" rid="B32">32</xref>, <xref ref-type="bibr" rid="B20">20</xref>). Pain is represented in latent state variables and must be inferred from previous model states and sensory information, represented as observable variables. By representing perception and sensation on different computational levels, our model allows the formalization of non-veridical relationships between physical stimulus and pain perception. In other words, pain needs to be inferred using all available information, such as nociceptor firing rates (sensations), general pain expectations and specific pain expectations based on recent percepts. The model incorporates learning, which is implemented using message passing and the minimization of free energy over time (<xref ref-type="bibr" rid="B22">22</xref>, <xref ref-type="bibr" rid="B33">33</xref>). We first describe the model&#x2019;s architecture and features, before simulating the effects of the parameters. Here, we focus on the effects of a prior probability of pain, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM12"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, a likelihood model <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM13"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM14"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, which describes the probability of remaining in a state of pain between subsequent time steps <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM15"><mml:mi>t</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM16"><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>. To illustrate our results, chronic pain can emerge from a high and precise prior belief to be in pain <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM17"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, which is generally more likely than not being in pain (here over-lined, representing the complement of pain; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM18"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>); <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM19"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x003E;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, in patients. The prior is sharpened further with every time-step where pain is inferred as the most likely cause of a given sensation, leading to a stabilization of the system within this pathological state. The system then is resistant to correctly interpreting harmless sensory information, as represented in an ambiguous <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM20"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>.</p>
</sec>
<sec id="s3"><label>2.</label><title>Materials and methods</title>
<sec id="s3a"><label>2.1.</label><title>Model preliminaries</title>
<sec id="s3a1"><label>2.1.1.</label><title>Sequential data</title>
<p>Chronic pain develops over time, therefore an appropriate computational model has to be sequential in nature. Further, the state of the body a short moment ago is usually the best predictor for its current state. The brain seems to leverage this predictability to process sensory information more efficiently (<xref ref-type="bibr" rid="B22">22</xref>, <xref ref-type="bibr" rid="B32">32</xref>). It is hence plausible to introduce a dependency of states over time into the model. Since the mind is assumed to constantly predict future sensory input, the model needs to accommodate longer time-series. The more recent past is usually more relevant to the current situation than the distant past. For example, what you saw three seconds ago is more relevant to your current situation than things you saw 5 weeks ago. The future state of a Markov chain is conditionally independent of all previous states, given the current state (Markov property). A Markov chain model fulfills the requirements of accounting for time-series dependencies while maintaining computational tractability and hence biological plausibility (for more details, see <xref ref-type="app" rid="app1">Appendix A</xref>).</p>
</sec>
<sec id="s3a2"><label>2.2.2.</label><title>Perceptual hierarchies</title>
<p>While a given intensity of noxious stimulation induces intense pain in one subject, another one may barely experience discomfort (<xref ref-type="bibr" rid="B34">34</xref>). The non-veridical relationship between sensation and pain prompts the need for a differential representation of the two within a model. This large inter-individual heterogeneity in pain experience further suggests a crucial role of an individual&#x2019;s generative model of pain experience. We use a Hidden Markov model (<xref ref-type="bibr" rid="B35">35</xref>, <xref ref-type="bibr" rid="B36">36</xref>) with latent (hidden) variables, representing the pain state, or <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM21"><mml:mi>H</mml:mi></mml:math></inline-formula>, and observable variables, representing sensory input, or <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM22"><mml:mi>S</mml:mi></mml:math></inline-formula>; see <xref ref-type="fig" rid="F1">Figure&#x00A0;1</xref> for a graphical representation. Pain is represented in hidden state variables and functions as an indicator of the body&#x2019;s integrity. This information is not directly accessible to the individual and hence has to be inferred from the available sensory information and prior expectations. From a Bayesian perspective, the experience of pain is determined by performing inference on hidden states, given the individual&#x2019;s prior beliefs. Given the non-linear and inter-individually heterogeneous relationships between sensation (i.e. nociceptor firing after physical stimulation) and percept (pain experience), an appropriate model allows the distinction of the two. In our model, this is achieved via separate hierarchical layers (observable and hidden). Of note, this computational-level model (in the sense of (<xref ref-type="bibr" rid="B13">13</xref>)) does not translate directly to implementational, neuroanatomical levels relevant to pain perception (<xref ref-type="bibr" rid="B37">37</xref>).</p>
<fig id="F1" position="float"><label>Figure 1</label>
<caption><p>Bayesian network representation of pain perception. We model perception in time as a hidden Markov model, where unobservable (hidden) nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM23"><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> form a Markov chain. These hidden nodes represent the state of the body, that the perceiving agent has no direct access to. Each hidden node connects to an observable node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM24"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>, which represents a sensory (e.g. noxious) input. The connections indicate the direction of causation: body states cause noxious inputs in a healthy agent. Tables show exemplary settings of relevant probabilities for both healthy and chronic pain perception, where prior probabilities are chosen so that expectations are stable in time (see <xref ref-type="app" rid="app2">Appendix B</xref> for details).</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g001.tif"/>
</fig>
</sec>
<sec id="s3a3"><label>2.3.3.</label><title>Inference and learning</title>
<p>The perceiving individual does not have direct access to their surroundings. It has to rely on information from its sensory organs, such as signals from sensory receptors, to obtain a representation of its surroundings. The accuracy of this information is crucial to the individual&#x2019;s survival in a given environment. Since the information arriving at the sensory organs is noisy and ambiguous, the mind needs to infer the real environmental causes for the sensation. Both prior beliefs and sensory information appear to be represented in a probabilistic manner (<xref ref-type="bibr" rid="B38">38</xref>). Perception then is the Bayes-optimal combination of prior beliefs and said sensory information. Conscious experience is determined by the hidden state that is assigned the highest posterior probability. In the context of pain perception, inference means determining the extent to which an observation <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM25"><mml:mi>S</mml:mi><mml:mo>=</mml:mo><mml:mi>s</mml:mi></mml:math></inline-formula> allows the perceiver to update their knowledge about a hidden state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM26"><mml:mi>H</mml:mi><mml:mo>=</mml:mo><mml:mi>h</mml:mi></mml:math></inline-formula>. For this purpose, it is necessary to infer the marginal probabilities of the latent variables from observations. We use the sum-product algorithm, also known as belief-propagation, as an efficient means of deriving marginal probabilities in singly-connected graphical models (<xref ref-type="bibr" rid="B36">36</xref>, <xref ref-type="bibr" rid="B39">39</xref>). Inference and learning over time is governed by the imperative to minimize free energy, a principle which has been proposed as a general theory of brain function (<xref ref-type="bibr" rid="B22">22</xref>, <xref ref-type="bibr" rid="B20">20</xref>, <xref ref-type="bibr" rid="B40">40</xref>). Variational message passing, as proposed in this framework, has the advantage of both computational tractability and biological plausibility (<xref ref-type="bibr" rid="B22">22</xref>). The interested reader is referred to <xref ref-type="app" rid="app1">Appendix A</xref> for a detailed description of the approach to inference applied here. While inference in this framework refers to short-term conclusions about marginal probabilities of pain, learning refers to longer-term updates of model parameters. We here implemented batch-updates, where model parameters are updated once after the observation of a full time series. Psychologically, this would correspond to retrospective memory consolidation, for example during sleep.</p>
</sec>
</sec>
</sec>
<sec id="s4"><label>3.</label><title>Simulations</title>
<p>All models and simulations were implemented in Python (version 3.9.7), in particular the numpy package (version 1.21.2, (<xref ref-type="bibr" rid="B41">41</xref>)) and a package for variational inference with exponential family models by author D.E. (<xref ref-type="bibr" rid="B33">33</xref>). All plots were created using the matplotlib.pyplot library for Python (version 3.4.3, (<xref ref-type="bibr" rid="B42">42</xref>)).</p>
<p>A Hidden Markov Model with <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM27"><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mn>20</mml:mn></mml:math></inline-formula> time-steps and free-energy learning over time is implemented (see <xref ref-type="app" rid="app1">Appendix A</xref> for details). Its hidden state variables, or nodes, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM28"><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>, represent the system&#x2019;s internal state at time point <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM29"><mml:mi>t</mml:mi></mml:math></inline-formula>. There are two possible states in each state, (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM30"><mml:mi>H</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mrow><mml:mi mathvariant="normal">pain</mml:mi></mml:mrow><mml:mo>,</mml:mo><mml:mover><mml:mrow><mml:mi mathvariant="normal">pain</mml:mi></mml:mrow><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula>) which are inferred from the combination of previous nodes&#x2019; states and incoming sensory information. Sensory information is represented within the observable nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM31"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>. There are two possible observations: noxious or harmless sensory information, or <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM32"><mml:mi>S</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mrow><mml:mi mathvariant="normal">noxious</mml:mi><mml:mo>,</mml:mo><mml:mi mathvariant="normal">harmless</mml:mi></mml:mrow><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula>. Observable nodes can remain unobserved. The top-down factor contains the likelihood <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM33"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> which allows explicit modelling of the association between sensory information and internal model state. The hidden variable nodes form a Markov chain, connected through additional free-energy factor nodes necessary for message passing. Here, the transition probabilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM34"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> represent the implicit expectations of pain, or the pain prior, over time. In other words, transition probabilities translate into the individual&#x2019;s expectation regarding the persistence of pain. A fixed and precise transition probability can constrain the model to arrive at a steady state, where conflicting sensory information does not alter the model&#x2019;s inferred hidden state, which we will discuss in detail below. We systematically vary the above-mentioned parameters to capture healthy pain perception as well as one that is biased towards chronically inferring pain. While the underlying inference engine is the same in both cases, there are four starting points from which to manipulate the model&#x2019;s behavior, also summarized in <xref ref-type="fig" rid="F1">Figure&#x00A0;1</xref>.
<list>
<list-item><label>1.</label><p>The <bold>prior</bold> of the first hidden variable <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM35"><mml:msub><mml:mi>H</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math></inline-formula>;</p></list-item>
<list-item><label>2.</label><p>The <bold>likelihood function</bold>, or the top-down factor connecting observable and hidden nodes, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM36"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>;</p></list-item>
<list-item><label>3.</label><p>The <bold>transition probabilities</bold> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM37"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> between the hidden variables;</p></list-item>
<list-item><label>4.</label><p>The <bold>sensory information</bold> at observable nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM38"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>.</p></list-item>
</list>Parameters are characterized by both the probabilities for specific states or observations, represented by their sufficient statistics <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM39"><mml:mi>&#x03BB;</mml:mi></mml:math></inline-formula>; and their precision, formalized by their pseudocounts <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM40"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula> (see <xref ref-type="app" rid="app1">Appendix A</xref> for details). <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM41"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula> is referred to as the <italic>pseudocount</italic> as it keeps count of the number of observed data points.</p>
<p><italic>Healthy interoceptive inference</italic> (&#x201C;healthy observer,&#x201D; from here onward) is characterized by a low, but precise prior expectation of pain (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM42"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.2</mml:mn><mml:mo>,</mml:mo><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula>) and a precise likelihood function that allows accurate inferences based on incoming sensory information (e.g., <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM43"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>noxious</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.8</mml:mn><mml:mo>,</mml:mo></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM44"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula>). Transitions towards pain-free states are more likely than transitions to painful states, as implemented in the transition probabilities between hidden states (e.g. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM45"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>).</p>
<p>In contrast, <italic>chronic pain inference</italic> (&#x201C;chronic pain observer&#x201D;) is characterized by an increased prior expectation of pain (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM46"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.9</mml:mn><mml:mo>,</mml:mo><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula>). The likelihood functions are imprecise and do not allow accurate inference from incoming sensory data, with <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM47"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>noxious</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM48"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>20</mml:mn></mml:math></inline-formula>. Transitions between hidden states are biased so that transitions into painful states are more likely than transitions into pain-free states, e.g. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM49"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>.</p>
<sec id="s4a"><label>3.1.</label><title>Mixed observations</title>
<p>In a first simulation, both observers were exposed to the same sensory information: noxious input for time steps <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM50"><mml:msub><mml:mi>S</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mn>5</mml:mn></mml:msub></mml:math></inline-formula>, and harmless input st <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM51"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mn>18</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> for 40 learning trials. Results are shown in <xref ref-type="fig" rid="F2">Figure&#x00A0;2</xref>. In the healthy observer (<xref ref-type="fig" rid="F2">Figure&#x00A0;2</xref>A), the exposure to noxious stimuli between time steps 1 and 5 leads to an acute and highly precise inference to be in pain. In contrast, the presentation of harmless sensory information (time steps 12&#x2013;18) decreases the probability of inferring to be in pain drastically. In the chronic pain model, however, the response to acute noxious sensations follows a different pattern (<xref ref-type="fig" rid="F2">Figure&#x00A0;2</xref>B)&#x2014;the baseline probability of inferring pain is at ceiling levels already. In contrast, harmless information barely has any significant effects on the probability of inferring pain, which remains high.</p>
<fig id="F2" position="float"><label>Figure 2</label>
<caption><p>Healthy (<bold>A</bold>) and chronified pain inference (<bold>B</bold>). The <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM52"><mml:mi>X</mml:mi></mml:math></inline-formula>-axis shows the time-step, referring to e.g. times during a day. The learning trials on the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM53"><mml:mi>Z</mml:mi></mml:math></inline-formula>-axis refer to moments of parameter updates, e.g. consecutive days, or longer-term memory consolidation and learning. On the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM54"><mml:mi>Y</mml:mi></mml:math></inline-formula>-axis, the inferred marginal probabilities of pain are represented. Both models were exposed to <italic>noxious sensory information</italic> at nodes 1&#x2013;5, and to <italic>harmless sensory information</italic> at nodes 12&#x2013;18 for 40 learning trials. The inferred marginal probabilities develop dynamically in the case of healthy inference (left), whereas there are hardly any deviations from the prior beliefs in the case of chronified pain inference (right).</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g002.tif"/>
</fig>
</sec>
<sec id="s4b"><label>3.2.</label><title>Prolonged exposure</title>
<p>In another simulation, the two observers are exposed to only-noxious vs. only-harmless sensory observations for all 20 time steps and 40 learning trials. Results are shown in <xref ref-type="fig" rid="F3">Figure&#x00A0;3</xref>. In individuals with chronic pain (3A,B), the type of sensory information the system is exposed to does not result in significant changes of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM55"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. Especially compared to the healthy inference model (3C,D), exposure to harmless information does not lead to decreased probabilities of inferring pain. In healthy inference, prolonged exposure to noxious information leads to acute heightened probabilities of experiencing pain. In contrast, harmless stimulation results in very low probabilities of inferring pain. Of note, the inferred probabilities of pain at the first- and final time step differ from intermediate time steps. This is an artifact caused by the model architecture with a definite end node, and our batch-updates (learning) that are performed at the end of one time series (see figure caption for details).</p>
<fig id="F3" position="float"><label>Figure 3</label>
<caption><p>Effect of prolonged exposure to one type of information. The <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM56"><mml:mi>X</mml:mi></mml:math></inline-formula>-axis in all plots shows the time step (e.g., time steps during a day), and the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM57"><mml:mi>Y</mml:mi></mml:math></inline-formula>-axis shows learning trials (e.g., memory consolidation at the end of each day). (<bold>A</bold>) and (<bold>B</bold>) show the observer with chronic pain exposed to 20 time steps and 40 trials of noxious information (<bold>A</bold>), and 20 time steps and 40 trials of harmless observations (<bold>B</bold>). (<bold>C</bold>) and (<bold>D</bold>) illustrate the same observational scheme under healthy pain inference. Of note, the marginal probabilities inferred at the first- and last time step differ from the probabilities inferred in intermittent nodes for two reasons. First, due to our model architecture, the first- and final nodes only receive messages related to sensory inputs from one neighbour, while intermittent nodes receive richer sensory information from their two respective neighbours (e.g., past and future time steps), leading to increased certainty about the hidden state. Secondly, we perform batch updates at the end of one time series. In psychological terms, this corresponds to retrospective memory consolidation during sleep. With increased learning trials (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM58"><mml:mi>Y</mml:mi></mml:math></inline-formula>-axis), however, the marginal probability of inferring pain under noxious stimulation (panels <bold>A</bold> and <bold>C</bold>) increases as a consequence of learning.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g003.tif"/>
</fig>
</sec>
</sec>
<sec id="s5"><label>4.</label><title>Treatment-resistant chronic pain: a null space problem?</title>
<p>Some forms of chronic pain are remarkably resistant to psychotherapeutic (<xref ref-type="bibr" rid="B43">43</xref>) or pharmacological (<xref ref-type="bibr" rid="B44">44</xref>) intervention, which has led to calls for ever more radical treatments such as transcranial magnetic stimulation, or TMS, (<xref ref-type="bibr" rid="B45">45</xref>), electroconvulsive therapy (<xref ref-type="bibr" rid="B46">46</xref>) or ketamine infusions (<xref ref-type="bibr" rid="B47">47</xref>).</p>
<p>A computational model of chronic pain therefore needs to be able to demonstrate a rigid, &#x201C;treatment-resistant&#x201D; pain experience over time. In the present model, this translates to a high temporal stability of the inferred, latent pain-state, regardless of the varying sensory input at the observable nodes (i.e., psychotherapy) or modulation of the sensations (i.e. pharmacotherapy). To derive the parameter settings necessary to arrive in a fully stable model state, we borrow the concept of the null space from linear algebra (<xref ref-type="bibr" rid="B36">36</xref>).</p>
<p>The null space of a matrix <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM59"><mml:mrow><mml:mi mathvariant="bold-italic">A</mml:mi></mml:mrow></mml:math></inline-formula> contains all vectors <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM60"><mml:mrow><mml:mover><mml:mi>x</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo></mml:mover></mml:mrow></mml:math></inline-formula> such that <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM61"><mml:mrow><mml:mi mathvariant="bold-italic">A</mml:mi></mml:mrow><mml:mrow><mml:mover><mml:mi>x</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo></mml:mover></mml:mrow><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula> (<xref ref-type="bibr" rid="B48">48</xref>). We are interested in the null space of the matrix that constrains the transition probability distributions <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM62"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> to yield a <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM63"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mrow><mml:mi mathvariant="normal">pain</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> after waiting for a sufficiently long time, i.e. we are looking for the stable states of the Markov chain. In other words, we are interested in changes to the transition probabilities that have no effect in the long run &#x2013; i.e. a resistance of the system to treatment.</p>
<p>We demonstrate in <xref ref-type="app" rid="app2">Appendix B</xref> that for each marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM64"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, there are transition probabilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM65"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> that satisfy this condition. Resulting is a line, the one-dimensional null space, containing matching transition probabilities for a given marginal probability of pain. The combination of marginal and derived transition probabilities has a marked effect on the inferred state: it remains stable within the range of the preset marginal probability of pain, irrespective of the quality of observations made, or changes to the prior. In <xref ref-type="fig" rid="F4">Figure&#x00A0;4</xref>, we illustrate the example of the marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM66"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>. First, as above, a model with 20 time steps and 40 learning trials is created. The prior <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM67"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mn>0</mml:mn></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> on the first time step is chosen randomly. Crucially, we sample transition probabilities from the null space and then sample the inferred probability of pain at the last time step. It becomes evident that within a very short period, the inferred probability of pain stabilizes at the pre-defined marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM68"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM69"><mml:mn>0.7</mml:mn></mml:math></inline-formula>&#x2014;regardless of any changes to the prior information.</p>
<fig id="F4" position="float"><label>Figure 4</label>
<caption><p>Null space of deviations from final state: over time, the marginal probability of inferring pain approaches and stabilizes within ranges of the preset marginal, here, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM70"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>. Random changes to the prior expectation of pain (at time step 0, illustrated by the different-colour lines) are overridden within two time steps, and the marginal probability approaches the predefined value.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g004.tif"/>
</fig>
<sec id="s5a"><label>4.1.</label><title>Therapy for chronic pain</title>
<p>Exposure therapy is an effective treatment for chronic pain, where patients with high levels of fear-avoidance are gradually exposed to stimuli or movements that elicit fear (<xref ref-type="bibr" rid="B49">49</xref>). Relating the idea of exposure therapy to our model, its goal would translate to changing the interpretation of harmless sensory information so that the inference becomes less biased towards pain. As an example, a person with chronic low back pain may avoid lifting a basket with groceries out of their car. In graded exposure therapy, the therapist would guide the patient through lifting baskets with increasing weight, all while closely monitoring the patient&#x2019;s fearful assumptions and physical sensations. Over the course of the exposure treatment, we would expect a change in <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM71"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>lift-basket</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> in this patient. Altered likelihoods may lead to less guarding behavior in patients, an increase in corrective sensory information and, ultimately, remission. In the specific case of fixed transition probabilities, however, this approach may not be sufficient. This picture would be in line with treatment-resistant chronic pain. The maintenance of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM72"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> in these patients is mostly determined by the transition probabilities, while incoming sensory information is mostly disregarded. We simulated two observers who receive ten sessions of exposure therapy (presentation of harmless sensory information for one time step) over the course of fifty time steps. In both cases, the pre-set marginal probability of inferring pain is <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM73"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>, and the transition probabilities were sampled from the respective null space of a marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM74"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>. Results are illustrated in <xref ref-type="fig" rid="F5">Figure&#x00A0;5</xref>. In the case of the first patient, the likelihood functions, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM75"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> were ambiguous (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM76"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn><mml:mo>,</mml:mo></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM77"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.4</mml:mn><mml:mo>,</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo stretchy="false">&#x2223;</mml:mo></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM78"><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM79"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.4</mml:mn><mml:mo>,</mml:mo></mml:math></inline-formula> and imprecise (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM83"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>20</mml:mn></mml:math></inline-formula>). In the simulation, the marginal probability of inferring pain dips slightly, before moving towards the initial, high <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM84"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula> again. In a second case, the likelihood functions were intact, that is, accurate (<italic>p</italic>(noxious &#x007C; pain) = 0.8, <italic>p</italic> (harmless &#x007C; pain) = 0.2, <italic>p</italic>(noxious <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM85"><mml:mover><mml:mrow><mml:mi mathvariant="normal">pain</mml:mi></mml:mrow><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.1</mml:mn><mml:mo>,</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.9</mml:mn></mml:math></inline-formula>) and precise (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM86"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula>). Here, the inference moves away from the high prior faster and remains relatively pain-free for longer.</p>
<fig id="F5" position="float"><label>Figure 5</label>
<caption><p>Simulation of ten sessions of exposure therapy in the nullspace of marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM87"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>, with ambiguous and imprecise likelihoods (<bold>A</bold>) and precise and accurate likelihoods (<bold>B</bold>). On the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM88"><mml:mi>X</mml:mi></mml:math></inline-formula>-axis are the time steps, on the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM89"><mml:mi>Y</mml:mi></mml:math></inline-formula>-axis the marginal probabilities of pain, and on the <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM90"><mml:mi>Z</mml:mi></mml:math></inline-formula>-axis the learning trials per time step. The combination of null space derived transition probabilities and an imprecise and ambiguous likelihood model (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM91"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM92"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.4</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM93"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM94"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.4</mml:mn></mml:math></inline-formula>) renders the repeated presentation of innocuous sensory information rather inefficient: inferred probabilities of pain remain within the range of the predefined marginal. When the likelihood model is precise and unambiguous (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM95"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.8</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM96"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.2</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM97"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.1</mml:mn></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM98"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.9</mml:mn></mml:math></inline-formula>), however, the presentation of harmless sensory information is more efficient in reducing the inferred probability of pain.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g005.tif"/>
</fig>
</sec>
</sec>
<sec id="s6" sec-type="discussion"><label>5.</label><title>Discussion</title>
<sec id="s6a"><label>5.1.</label><title>A Bayesian model for chronic pain</title>
<p>The experience of pain seems to be the result of an inferential process, where prior expectations are integrated with current sensory information over time (<xref ref-type="bibr" rid="B14">14</xref>, <xref ref-type="bibr" rid="B16">16</xref>). This process is probabilistic and non-linear, with large inter-individual differences in the relationship between physical stimulation and pain experience. While statistical accounts of pain have gained momentum over the past decade, insights into computational underpinnings of the chronification of pain have been sparse (<xref ref-type="bibr" rid="B50">50</xref>, <xref ref-type="bibr" rid="B18">18</xref>, <xref ref-type="bibr" rid="B14">14</xref>).</p>
<p>We propose a Bayesian model for interoceptive perception and pain experience in chronic pain. Drawing on machine learning concepts such as belief-propagation and free-energy minimization, we demonstrate that the model captures well-studied phenomena such as decreased pain thresholds and allodynia (i.e. perceiving innocuous stimuli as painful) in individuals with chronic pain.</p>
<p>Our &#x201C;chronic pain&#x201D; observer is characterized by heightened prior expectations of pain, aberrant associations between sensory input and pain perception and heightened assumptions about the persistence of pain. In our model, this refers to overly precise priors for being in a state of pain, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM99"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>), ambiguous top-down or likelihood functions <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM100"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>sensation</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and more rigid transition probabilities between hidden states <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM101"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, respectively.</p>
</sec>
<sec id="s6b"><label>5.2.</label><title>Chronic pain as biased inference</title>
<p>It has been suggested that chronic pain is associated with a broad and heightened expectation of pain (<xref ref-type="bibr" rid="B51">51</xref>, <xref ref-type="bibr" rid="B14">14</xref>, <xref ref-type="bibr" rid="B24">24</xref>). Heightened expectations of pain may stem from early averse life events or injuries (<xref ref-type="bibr" rid="B52">52</xref>), or social factors such as an overprotective, fearful parental style (<xref ref-type="bibr" rid="B14">14</xref>). In the patient model, a heightened pain prediction led to inferring to be in pain more readily compared to the healthy model. Conflicting innocuous sensory stimulation hardly influenced the inferred state. In accordance with this, empirical studies suggest chronic pain patients may not attend to sensory information but rather rely on their prior expectations of pain (<xref ref-type="bibr" rid="B53">53</xref>, <xref ref-type="bibr" rid="B54">54</xref>). Our observations fit into a larger discussion about expectations as core features of a multitude of mental disorders (<xref ref-type="bibr" rid="B51">51</xref>, <xref ref-type="bibr" rid="B14">14</xref>). A further aspect of our model is that harmless sensory information is associated with pain over time. This phenomenon is commonly observed in patients and referred to as allodynia (<xref ref-type="bibr" rid="B55">55</xref>, <xref ref-type="bibr" rid="B14">14</xref>). The Bayesian Brain hypothesis postulates that the perceiving individual perpetually tries to infer the real-world causes for incoming sensory information (<xref ref-type="bibr" rid="B22">22</xref>). We here present a perspective on pain as an abstract percept of the body&#x2019;s state. Following this view, pain is an inference on a real-world &#x201C;cause&#x201D; (cmp. (<xref ref-type="bibr" rid="B56">56</xref>)), bodily damage, based on all available prior- and current sensory information. The chronic pain patient infers pain from a wide range of sensory stimuli that would not be perceived as painful by healthy controls. Further, the associations between sensory information and hidden state are noisy and unclear in the chronic pain observer. While a healthy observer maintains precise beliefs about the external causes of harmless sensory input, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM102"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, this probability is completely ambiguous in our patient model. Over time, the chronic pain patient is less and less likely to infer a pain-free state from any type of incoming sensation.</p>
<p>Contrarily, a flexible and highly dynamic pattern can be observed under healthy interoceptive inference. Here, noxious stimuli are associated with pain, and a pronounced and acute state of pain is inferred from incoming harmful stimuli. However, once the stream of harmful sensory input stops, the probability of pain normalizes very quickly. This dynamic pattern is in line with the function of pain as a warning signal&#x2014;motivating quick and adaptive behavior change following a threat to the individual&#x2019;s physical integrity (<xref ref-type="bibr" rid="B57">57</xref>).</p>
<p>Note that our results also hold under the active inference framework (<xref ref-type="bibr" rid="B23">23</xref>, <xref ref-type="bibr" rid="B58">58</xref>). Canonical versions of active inference contain a generative model that is very similar to ours (Hidden Markov model, HMM), with the difference of an additional layer that models an agent&#x2019;s actions, or choices (<xref ref-type="bibr" rid="B59">59</xref>). Actions then influence the transitions between states so that more likely, or expected states under the current observations, are reached. In other words, in active inference, an agent infers the hidden state from the combination of prior beliefs, sensory information and their own actions. As an example, consider the case of a child with chronic abdominal pain who involuntarily keeps picking at her abdomen (checking behavior, example from (<xref ref-type="bibr" rid="B14">14</xref>)), so that increased pain is experienced. Under active inference, this can be interpreted as causing observations (noxious sensations) that are expected under the most likely state (pain). Acting directly towards bringing about a certain observation changes the causal relationship between observation and hidden state (i.e., &#x201C;do-operations&#x201D;, (<xref ref-type="bibr" rid="B60">60</xref>)). However, it is impossible to act directly on one&#x2019;s nociceptors, so that pain still needs to be inferred from sensations, even when the noxious stimulation is self-induced.</p>
</sec>
<sec id="s6c"><label>5.3.</label><title>Exposure and the null space of psychotherapy</title>
<p>If the individual performs implicit Bayesian inference over hidden states, then we here demonstrate that this inference can be biased towards inferring pain. This has important implications for the treatment of chronic pain patients. Cognitive behavioral therapy (CBT) shows statistically significant, but small effects on certain aspects of chronic pain (pain, mood, disability, pain catastrophizing; (<xref ref-type="bibr" rid="B61">61</xref>)). Techniques usually include relaxation training, behavioral activation (i.e. engaging in behaviors that were previously avoided due to fear of pain), setting behavioral goals, problem-solving training and cognitive re-structuring (<xref ref-type="bibr" rid="B61">61</xref>, <xref ref-type="bibr" rid="B62">62</xref>). Further, exposure therapy has shown promising results in the treatment of chronic pain (<xref ref-type="bibr" rid="B63">63</xref>, <xref ref-type="bibr" rid="B64">64</xref>, <xref ref-type="bibr" rid="B49">49</xref>). Exposure therapy seems to reduce disability more effectively than CBT in patients with chronic lower back pain (<xref ref-type="bibr" rid="B49">49</xref>). It is designed to specifically target emotional responses to pain- or expectations of pain, which lead to excessive guarding behaviors. Guarding behaviors, in turn, may limit the opportunity for receiving corrective, conflicting sensory information, leading to the solidification of fearful pain beliefs (<xref ref-type="bibr" rid="B65">65</xref>, <xref ref-type="bibr" rid="B66">66</xref>). However, we show that inference can stabilize towards pain, even in light of conflicting sensory information. Specifically, when the transition probabilities between hidden states have a stable state with high marginal pain probability, a simulated patient model returns to the inference of being in pain while harmless sensory information is provided. In other words, the initial deviation from the high pain state, which might have been reached by therapeutic means, has vanished. This simulation could be a model for treatment-resistant chronic pain. We further show that these patients may be best served by targeting their beliefs about the associations between sensory stimuli and pain expectations (likelihoods). When likelihoods allow the precise and accurate association between sensory stimuli and pain, exposure to harmless sensory information leads to more promising results. However, when the patient does not have accurate likelihoods, exposure to certain sensory stimuli may not be a promising approach. It can hence be derived that patients may need to re-learn precise and accurate inferences about different types of sensory stimuli, ideally before being guided through exposure therapy.</p>
<p>One can speculate about the role of pharmacotherapy in the context of our model. Potentially, analgesics target the patient&#x2019;s likelihood model by inhibiting the transmission of noxious information from pain receptors (<xref ref-type="bibr" rid="B67">67</xref>). This assumption requires empirical testing with appropriate data of medicated chronic pain patients. In this case, combining pharmacological and psycho-therapeutical strategies may be promising. More data is necessary to assess which treatments target what model parameter. For an overview of expected self-report scores and hypothesized underlying model parameters, see <xref ref-type="fig" rid="F6">Figure&#x00A0;6</xref>.</p>
<fig id="F6" position="float"><label>Figure 6</label>
<caption><p>Tabular overview of patient experience and model predictions. <italic>Timepoints</italic>: T <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM103"><mml:mo>&#x00B1;</mml:mo></mml:math></inline-formula> month, <italic>Model parameters</italic>: expected pattern of change in the model parameters underlying chronic vs. acute pain experience; where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM104"><mml:mi>p</mml:mi></mml:math></inline-formula>, pain; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM105"><mml:mover><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi></mml:mrow><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover></mml:math></inline-formula>, pain-free state; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM106"><mml:mi>n</mml:mi></mml:math></inline-formula>, noxious input; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM107"><mml:mover><mml:mi>n</mml:mi><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover></mml:math></inline-formula>, harmless input. <italic>Self-report</italic>: Expected values, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM108"><mml:mo stretchy="false">&#x2191;</mml:mo></mml:math></inline-formula>, increased values compared to baseline; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM109"><mml:mo stretchy="false">&#x2193;</mml:mo></mml:math></inline-formula>, decreasing values; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM110"><mml:mo stretchy="false">&#x2192;</mml:mo></mml:math></inline-formula>, average values; *, there is some evidence that the generative model of patients with chronic pain is biased towards pain from childhood on, e.g. via traumatic experiences, abuse and earlier chronic pain experiences (<xref ref-type="bibr" rid="B68">68</xref>&#x2013;<xref ref-type="bibr" rid="B70">70</xref>).</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g006.tif"/>
</fig>
<p>In <xref ref-type="fig" rid="F6">Figure&#x00A0;6</xref>, we illustrate two hypothetical patients, one developing chronic pain after an acute injury, whereas the other patient does not develop chronic pain beyond acute injury.</p>
<p>In the chronic pain patient, pain persists for prolonged periods post injury. Empirically, we expect to see pain and disability scores that are persistently heightened in the months following injury. In experiments (i.e. fear conditioning, pain ratings following noxious stimuli), our model predicts increased levels of prior expectations of pain (note the shift in probabilities over time), as well as decreased precision of sensory information, corresponding to an ambiguous and imprecise likelihood model. Finally, for pain to persist in a very rigid way, transitions towards a pain-free state become more unlikely until therapy can reverse this effect. We can speculate that likelihood models may be treated via medication-supported exposure therapy, whereas prior beliefs could be best targeted using CBT.</p>
<p>In the acute pain patient, the injury is followed by a quick recovery and remission into a pain-free state. The noxious stimulus leads to a transient state of experiencing pain, including an elevated prior expectation of pain, which is a consequence of learning about an acutely noxious stimulus. Disability and pain experience as assessed with self-report measures are elevated in times of acute injury. With the injury healing and the noxious information subsiding, the pain-free state is re-established within 4 months. Note that the likelihood and transition models are assumed to stay relatively intact in this patient.</p>
</sec>
<sec id="s6d"><label>5.4.</label><title>Expectations as primary therapy target</title>
<p>Following our model, there are three factors that could be promising targets for patients with chronic pain. Firstly, therapy could target the heightened and generalized expectations of pain. This is in line with Rief et al. (<xref ref-type="bibr" rid="B51">51</xref>), Panitz et al. (<xref ref-type="bibr" rid="B71">71</xref>), who discuss the importance of psychological interventions that target expectations. Secondly, our model suggests a need for re-learning the associations between sensory information and the state of pain. This could be achieved via gradual, guided exposure to harmless stimuli in therapy. Lastly, a patient&#x2019;s belief about the persistence of pain over time could be targeted specifically with cognitive interventions (e.g. retrospective evaluations, re-learning to differentiate pain intensities), which may be especially relevant in cases of treatment-resistant chronic pain.</p>
</sec>
<sec id="s6e"><label>5.5.</label><title>Limitations</title>
<p>The presented model makes several simplifying assumptions and can be extended in numerous ways. Currently, latent- and observable variables could only take one of two possible states for the sake of simplicity. Continuous random variables could allow for more complex conclusions. It is generally feasible to perform inference on graphical models with continuous random variable nodes (<xref ref-type="bibr" rid="B72">72</xref>, <xref ref-type="bibr" rid="B73">73</xref>). However, this added complexity might detract from our main conclusions, which we expect to hold in the continuous case, too. In future work, we will extend our model to continuous sensory signals and internal states.</p>
<p>Further, future efforts to extend this framework should account for the vast heterogeneity that is found among chronic pain patients. Assuming a multitude of underlying etiological factors, they might need differential representation in quantitative terms as well.</p>
<p>With this model, we focus on computational and psychological aspects of pain inference. We have formalized a hypothesis about the generative process underlying chronic pain. While several salient features of acute and chronified pain perception are captured by our model, we do not have access to the true generative process and can hence not draw any definite conclusions about it.</p>
<p>Besides psychological factors (e.g. beliefs), a multitude of additional factors could be considered when extending this model. As Kiverstein and colleagues (2022) note, a model of chronic pain needs to take on a bio-psycho-social perspective, integrating a multitude of empirical findings under one approach. For example, pain experience seems to be alleviated by the presence of a close friend or partner, i.e. social support (<xref ref-type="bibr" rid="B74">74</xref>). In a recent review, Mohr and Fotopoulou (2018) argue that social support may serve as a security signal which renders prediction errors caused by threats to the physical integrity less precise. Future iterations of this model may be extended to incorporate these findings, e.g. by means of a context-layer. Here, however, we show that important characteristics of chronic pain experience can be simulated from a Bayesian perspective, by assuming that a) sensory information relates to pain experience through (biased) parameters of a generative model and b) the inferred states are marked by correlations, or stability, over time.</p>
</sec>
<sec id="s6f"><label>5.6.</label><title>Future directions</title>
<p>Rigid expectations that are not sufficiently constrained by incoming sensory information seem to lie at the heart of several mental disorders. Psychosis, for instance, can be regarded as resulting from an exaggerated and inflexible reliance on prior expectations (<xref ref-type="bibr" rid="B75">75</xref>, <xref ref-type="bibr" rid="B76">76</xref>). Computational accounts of mental disorders can provide novel mechanistic insights into their etiology, exacerbation and maintenance over time (<xref ref-type="bibr" rid="B77">77</xref>).</p>
<p>The computational model of chronic pain presented here can be applied to empirical data. Below, we outline possible avenues towards testing the model presented here.</p>
<p>Firstly, data from learning experiments in chronic pain patients and controls could shed light on the real-world significance of the model&#x2019;s parameters. In fear conditioning experiments, participants learn the association between a painful unconditioned stimulus (UCS, such as e.g. heat stimulation with a thermode or electric shocks) and unrelated, conditioned stimuli (CS, such as a visual cue). In a first step, significant group differences in model parameters would need to be investigated. Then, given their heightened pain prediction, our model would predict decreased pain thresholds in patients with chronic pain. Biased likelihood terms may lead to altered learning about the UCS-CS contingencies in chronic pain patients. Similarly, other assumptions that were used to furnish this model could be tested empirically (e.g. biased and imprecise likelihood models, heightened prior predictions of pain, or rigid and exaggerated beliefs about the persistence of pain in chronic pain patients).</p>
<p>A key challenge to empirical tests of our model in humans is the need to approximate observable sensations, i.e., nociceptor firing. Sensations can be approximated by measuring physical properties of experimental stimuli, such as the temperature of a heat thermode, the current used for electric stimuli, or the intensity of mechanical pressure. To empirically test our model, future studies could assess the relationship between physical sensations and the pain percept. For this, concurrent measurements of sensations (or proxies thereof, such as physical stimulus properties) and reports on the pain percept (e.g. pain scores) are necessary. Pain scores in response to stimuli with different intensities could be collected from healthy individuals and chronic pain patients. We would expect to find significant group differences with respect to prior, transition- and likelihood model parameters. Such data could further shed light on an important detail of patient experience: what model parameter is most relevant to the chronification of pain and the experience of patients? In all cases, the full Bayesian model would need to be compared to both simpler models, such as e.g. single-trial vs. sequential models, and more complex models, such as deeper predictive hierarchies, in a model comparison.</p>
</sec>
<sec id="s6g" sec-type="conclusions"><label>5.7.</label><title>Conclusion</title>
<p>A novel Bayesian model is able to reproduce common features of healthy- and chronified pain perception. Machine learning approaches, such as a hidden Markov model and variational inference, allowed the exploration of parameters that may underlie the development of chronic pain. We have demonstrated that this model is able to capture many phenomena observed in individuals with chronic pain, and during acute, healthy pain perception. Additionally, the model was able to simulate rigid, treatment resistant chronic pain. Further research could significantly advance our mechanistic understanding of chronic pain, which may help to inform treatment selection.</p>
</sec>
</sec>
</body>
<back>
<sec id="s9" sec-type="data-availability"><title>Data availability statement</title>
<p>All code used in this manuscript is publicly available here: <ext-link ext-link-type="uri" xlink:href="https://data.uni-marburg.de/handle/dataumr/163">https://data.uni-marburg.de/handle/dataumr/163</ext-link>.</p>
</sec>
<sec id="s10"><title>Ethics Statement</title>
<p>Ethical review and approval was not required for this study in accordance with the local legislation and institutional requirements.</p>
</sec>
<sec id="s11"><title>Author contributions</title>
<p>ALE and DE conducted the first set of analyses, further developed by KP, DE and ALE. ALE, KP and DE produced the final results presented here. ALE wrote a first draft of this manuscript. All authors reviewed and edited the manuscript and approved the submitted version.</p>
</sec>
<sec id="s12" sec-type="funding-information"><title>Funding</title>
<p>This work was supported by the cluster project &#x201C;The Adaptive Mind&#x201D;, funded by the Excellence Program of the Hessian Ministry for Science and the Arts and the DFG SFB-TRR289 &#x201C;Treatment Expectation&#x201D;, project number 422744262.</p>
</sec>
<sec id="s13" sec-type="COI-statement"><title>Conflict of interest statement</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec id="s14" sec-type="disclaimer"><title>Publisher&#x0027;s note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ref-list><title>References</title>
<ref id="B1"><label>1.</label><citation citation-type="other"><collab>IASP</collab>. <comment>Iasp terminology. (2017). Available at: <ext-link ext-link-type="uri" xlink:href="https://www.iasp-pain.org/Education/Content.aspx?ItemNumber=1698navItemNumb">https://www.iasp-pain.org/Education/Content.aspx?ItemNumber=1698navItemNumb</ext-link></comment></citation></ref>
<ref id="B2"><label>2.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>King</surname><given-names>S</given-names></name><name><surname>Chambers</surname><given-names>CT</given-names></name><name><surname>Huguet</surname><given-names>A</given-names></name><name><surname>MacNevin</surname><given-names>RC</given-names></name><name><surname>McGrath</surname><given-names>PJ</given-names></name><name><surname>Parker</surname><given-names>L</given-names></name></person-group>, et al. <article-title>The epidemiology of chronic pain in children, adolescents revisited: a systematic review</article-title>. <source>Pain</source> (<year>2011</year>) <volume>152</volume>:<fpage>2729</fpage>&#x2013;<lpage>38</lpage>. <pub-id pub-id-type="doi">10.1016/j.pain.2011.07.016</pub-id><pub-id pub-id-type="pmid">22078064</pub-id></citation></ref>
<ref id="B3"><label>3.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Breivik</surname><given-names>H</given-names></name><name><surname>Collett</surname><given-names>B</given-names></name><name><surname>Ventafridda</surname><given-names>V</given-names></name><name><surname>Cohen</surname><given-names>R</given-names></name><name><surname>Gallacher</surname><given-names>D</given-names></name></person-group>. <article-title>Survey of chronic pain in Europe: prevalence, impact on daily life, and treatment</article-title>. <source>Eur J Pain</source> (<year>2006</year>) <volume>10</volume>:<fpage>287</fpage>&#x2013;<lpage>333</lpage>. <pub-id pub-id-type="doi">10.1016/j.ejpain.2005.06.009</pub-id><pub-id pub-id-type="pmid">16095934</pub-id></citation></ref>
<ref id="B4"><label>4.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lam&#x00E9;</surname><given-names>IE</given-names></name><name><surname>Peters</surname><given-names>ML</given-names></name><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>Kleef</surname><given-names>MV</given-names></name><name><surname>Patijn</surname><given-names>J</given-names></name></person-group>. <article-title>Quality of life in chronic pain is more associated with beliefs about pain, than with pain intensity</article-title>. <source>Eur J Pain</source> (<year>2005</year>) <volume>9</volume>:<fpage>15</fpage>&#x2013;<lpage>24</lpage>. <pub-id pub-id-type="doi">10.1016/j.ejpain.2004.02.006</pub-id></citation></ref>
<ref id="B5"><label>5.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hunfeld</surname><given-names>JA</given-names></name><name><surname>Perquin</surname><given-names>CW</given-names></name><name><surname>Duivenvoorden</surname><given-names>HJ</given-names></name><name><surname>Hazebroek-Kampschreur</surname><given-names>AA</given-names></name><name><surname>Passchier</surname><given-names>J</given-names></name><name><surname>van Suijlekom-Smit</surname><given-names>LW</given-names></name></person-group>, et al. <article-title>Chronic pain and its impact on quality of life in adolescents and their families</article-title>. <source>J Pediatr Psychol</source> (<year>2001</year>) <volume>26</volume>:<fpage>145</fpage>&#x2013;<lpage>53</lpage>. <pub-id pub-id-type="doi">10.1093/jpepsy/26.3.145</pub-id><pub-id pub-id-type="pmid">11259516</pub-id></citation></ref>
<ref id="B6"><label>6.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tunks</surname><given-names>ER</given-names></name><name><surname>Crook</surname><given-names>J</given-names></name><name><surname>Weir</surname><given-names>R</given-names></name></person-group>. <article-title>Epidemiology of chronic pain with psychological comorbidity: prevalence, risk, course, and prognosis</article-title>. <source>Can J Psychiatry</source> (<year>2008</year>) <volume>53</volume>:<fpage>224</fpage>&#x2013;<lpage>34</lpage>. <pub-id pub-id-type="doi">10.1177/070674370805300403</pub-id><pub-id pub-id-type="pmid">18478825</pub-id></citation></ref>
<ref id="B7"><label>7.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>M&#x00E4;ntyselk&#x00E4;</surname><given-names>P</given-names></name><name><surname>Kumpusalo</surname><given-names>E</given-names></name><name><surname>Ahonen</surname><given-names>R</given-names></name><name><surname>Kumpusalo</surname><given-names>A</given-names></name><name><surname>Kauhanen</surname><given-names>J</given-names></name><name><surname>Viinam&#x00E4;ki</surname><given-names>H</given-names></name></person-group>, et al. <article-title>Pain as a reason to visit the doctor: a study in Finnish primary health care</article-title>. <source>Pain</source> (<year>2001</year>) <volume>89</volume>:<fpage>175</fpage>&#x2013;<lpage>80</lpage>. <pub-id pub-id-type="doi">10.1016/s0304-3959(00)00361-4</pub-id></citation></ref>
<ref id="B8"><label>8.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gaskin</surname><given-names>DJ</given-names></name><name><surname>Richard</surname><given-names>P</given-names></name></person-group>. <article-title>The economic costs of pain in the united states</article-title>. <source>J Pain</source> (<year>2012</year>) <volume>13</volume>:<fpage>715</fpage>&#x2013;<lpage>24</lpage>. <pub-id pub-id-type="doi">10.1016/j.jpain.2012.03.009</pub-id><pub-id pub-id-type="pmid">22607834</pub-id></citation></ref>
<ref id="B9"><label>9.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Phillips</surname><given-names>CJ</given-names></name></person-group>. <article-title>Economic burden of chronic pain</article-title>. <source>Expert Rev Pharmacoecon Outcomes Res</source> (<year>2006</year>) <volume>6</volume>:<fpage>591</fpage>&#x2013;<lpage>601</lpage>. <pub-id pub-id-type="doi">10.1586/14737167.6.5.591</pub-id><pub-id pub-id-type="pmid">20528505</pub-id></citation></ref>
<ref id="B10"><label>10.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Groenewald</surname><given-names>CB</given-names></name><name><surname>Essner</surname><given-names>BS</given-names></name><name><surname>Wright</surname><given-names>D</given-names></name><name><surname>Fesinmeyer</surname><given-names>MD</given-names></name><name><surname>Palermo</surname><given-names>TM</given-names></name></person-group>. <article-title>The economic costs of chronic pain among a cohort of treatment-seeking adolescents in the United States</article-title>. <source>J Pain</source> (<year>2014</year>) <volume>15</volume>:<fpage>925</fpage>&#x2013;<lpage>33</lpage>. <pub-id pub-id-type="doi">10.1016/j.jpain.2014.06.002</pub-id><pub-id pub-id-type="pmid">24953887</pub-id></citation></ref>
<ref id="B11"><label>11.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Freburger</surname><given-names>JK</given-names></name><name><surname>Holmes</surname><given-names>GM</given-names></name><name><surname>Agans</surname><given-names>RP</given-names></name><name><surname>Jackman</surname><given-names>AM</given-names></name><name><surname>Darter</surname><given-names>JD</given-names></name><name><surname>Wallace</surname><given-names>AS</given-names></name></person-group>, et al. <article-title>The rising prevalence of chronic low back pain</article-title>. <source>Arch Intern Med</source> (<year>2009</year>) <volume>169</volume>:<fpage>251</fpage>&#x2013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1001/archinternmed.2008.543</pub-id><pub-id pub-id-type="pmid">19204216</pub-id></citation></ref>
<ref id="B12"><label>12.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>S&#x00E1;</surname><given-names>KN</given-names></name><name><surname>Moreira</surname><given-names>L</given-names></name><name><surname>Baptista</surname><given-names>AF</given-names></name><name><surname>Yeng</surname><given-names>LT</given-names></name><name><surname>Teixeira</surname><given-names>MJ</given-names></name><name><surname>Galhardoni</surname><given-names>R</given-names></name></person-group>, et al. <article-title>Prevalence of chronic pain in developing countries: systematic review and meta-analysis</article-title>. <source>Pain Rep</source> (<year>2019</year>) <volume>4</volume>:<fpage>1</fpage>&#x2013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1097/PR9.0000000000000779</pub-id></citation></ref>
<ref id="B13"><label>13.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Marr</surname><given-names>D</given-names></name></person-group>, <source>Vision: a computational investigation into the human representation and processing of visual information</source>, <publisher-name>Cambridge, Mass.</publisher-name>: <publisher-loc>MIT Press</publisher-loc> (<year>1982</year>).</citation></ref>
<ref id="B14"><label>14.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hechler</surname><given-names>T</given-names></name><name><surname>Endres</surname><given-names>DM</given-names></name><name><surname>Thorwart</surname><given-names>A</given-names></name></person-group>. <article-title>Why harmless sensations might hurt in individuals with chronic pain: about heightened prediction and perception of pain in the mind</article-title>. <source>Front Psychol</source> (<year>2016</year>) <volume>7</volume>:<fpage>1638</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2016.01638</pub-id><pub-id pub-id-type="pmid">27826271</pub-id></citation></ref>
<ref id="B15"><label>15.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>B&#x00FC;chel</surname><given-names>C</given-names></name><name><surname>Geuter</surname><given-names>S</given-names></name><name><surname>Sprenger</surname><given-names>C</given-names></name><name><surname>Eippert</surname><given-names>F</given-names></name></person-group>. <article-title>Placebo analgesia: a predictive coding perspective</article-title>. <source>Neuron</source> (<year>2014</year>) <volume>81</volume>:<fpage>1223</fpage>&#x2013;<lpage>39</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuron.2014.02.042</pub-id></citation></ref>
<ref id="B16"><label>16.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tabor</surname><given-names>A</given-names></name><name><surname>Thacker</surname><given-names>MA</given-names></name><name><surname>Moseley</surname><given-names>GL</given-names></name><name><surname>K&#x00F6;rding</surname><given-names>KP</given-names></name></person-group>. <article-title>Pain: a statistical account</article-title>. <source>PLoS Comput Biol</source> (<year>2017</year>) <volume>13</volume>:<fpage>e1005142</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pcbi.1005142</pub-id><pub-id pub-id-type="pmid">28081134</pub-id></citation></ref>
<ref id="B17"><label>17.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wall</surname><given-names>P</given-names></name><name><surname>McMahon</surname><given-names>S</given-names></name></person-group>. <article-title>The relationship of perceived pain to afferent nerve impulses</article-title>. <source>Trends Neurosci</source> (<year>1986</year>) <volume>9</volume>(<issue>6</issue>):<fpage>254</fpage>&#x2013;<lpage>5</lpage>. <pub-id pub-id-type="doi">10.1016/0166-2236(86)90070-6</pub-id></citation></ref>
<ref id="B18"><label>18.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kiverstein</surname><given-names>J</given-names></name><name><surname>Kirchhoff</surname><given-names>MD</given-names></name><name><surname>Thacker</surname><given-names>M</given-names></name></person-group>. <article-title>An embodied predictive processing theory of pain experience</article-title>. <source>Rev Philos Psychol</source> (<year>2022</year>):<fpage>11</fpage>&#x2013;<lpage>26</lpage>. <pub-id pub-id-type="doi">10.1007/s13164-022-00616-2</pub-id></citation></ref>
<ref id="B19"><label>19.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Dayan</surname><given-names>P</given-names></name><name><surname>Hinton</surname><given-names>GE</given-names></name><name><surname>Neal</surname><given-names>RM</given-names></name><name><surname>Zemel</surname><given-names>RS</given-names></name></person-group>. <article-title>The helmholtz machine</article-title>. <source>Neural Comput</source> (<year>1995</year>) <volume>7</volume>:<fpage>889</fpage>&#x2013;<lpage>904</lpage>. <pub-id pub-id-type="doi">10.1162/neco.1995.7.5.889</pub-id><pub-id pub-id-type="pmid">7584891</pub-id></citation></ref>
<ref id="B20"><label>20.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>Kiebel</surname><given-names>S</given-names></name></person-group>. <article-title>Predictive coding under the free-energy principle</article-title>. <source>Philos Trans R Soc Lond B Biol Sci</source> (<year>2009</year>) <volume>364</volume>:<fpage>1211</fpage>&#x2013;<lpage>21</lpage>. <pub-id pub-id-type="doi">10.1098/rstb.2008.0300</pub-id><pub-id pub-id-type="pmid">19528002</pub-id></citation></ref>
<ref id="B21"><label>21.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Barrett</surname><given-names>LF</given-names></name><name><surname>Simmons</surname><given-names>WK</given-names></name></person-group>. <article-title>Interoceptive predictions in the brain</article-title>. <source>Nat Rev Neurosci</source> (<year>2015</year>) <volume>16</volume>:<fpage>419</fpage>&#x2013;<lpage>29</lpage>. <pub-id pub-id-type="doi">10.1038/nrn3950</pub-id><pub-id pub-id-type="pmid">26016744</pub-id></citation></ref>
<ref id="B22"><label>22.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name></person-group>. <article-title>The free-energy principle: a unified brain theory?</article-title>. <source>Nat Rev Neurosci</source> (<year>2010</year>) <volume>11</volume>:<fpage>127</fpage>&#x2013;<lpage>38</lpage>. <pub-id pub-id-type="doi">10.1038/nrn2787</pub-id><pub-id pub-id-type="pmid">20068583</pub-id></citation></ref>
<ref id="B23"><label>23.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>Daunizeau</surname><given-names>J</given-names></name><name><surname>Kiebel</surname><given-names>S</given-names></name></person-group>. <article-title>Active inference or reinforcement learning</article-title>. <source>PLoS ONE</source> (<year>2009</year>) <volume>4</volume>:<fpage>e6421</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0006421</pub-id><pub-id pub-id-type="pmid">19641614</pub-id></citation></ref>
<ref id="B24"><label>24.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fairhurst</surname><given-names>M</given-names></name><name><surname>Wiech</surname><given-names>K</given-names></name><name><surname>Dunckley</surname><given-names>P</given-names></name><name><surname>Tracey</surname><given-names>I</given-names></name></person-group>. <article-title>Anticipatory brainstem activity predicts neural processing of pain in humans</article-title>. <source>Pain</source> (<year>2007</year>) <volume>128</volume>:<fpage>101</fpage>&#x2013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.1016/j.pain.2006.09.001</pub-id><pub-id pub-id-type="pmid">17070996</pub-id></citation></ref>
<ref id="B25"><label>25.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wiech</surname><given-names>K</given-names></name><name><surname>Vandekerckhove</surname><given-names>J</given-names></name><name><surname>Zaman</surname><given-names>J</given-names></name><name><surname>Tuerlinckx</surname><given-names>F</given-names></name><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>Tracey</surname><given-names>I</given-names></name></person-group>. <article-title>Influence of prior information on pain involves biased perceptual decision-making</article-title>. <source>Curr Biol</source> (<year>2014</year>) <volume>24</volume>:<fpage>R679</fpage>&#x2013;<lpage>81</lpage>. <pub-id pub-id-type="doi">10.1016/j.cub.2014.06.022</pub-id><pub-id pub-id-type="pmid">25093555</pub-id></citation></ref>
<ref id="B26"><label>26.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wiech</surname><given-names>K</given-names></name></person-group>. <article-title>Deconstructing the sensation of pain: the influence of cognitive processes on pain perception</article-title>. <source>Science</source> (<year>2016</year>) <volume>354</volume>:<fpage>584</fpage>&#x2013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1126/science.aaf8934</pub-id><pub-id pub-id-type="pmid">27811269</pub-id></citation></ref>
<ref id="B27"><label>27.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hird</surname><given-names>EJ</given-names></name><name><surname>Jones</surname><given-names>A</given-names></name><name><surname>Talmi</surname><given-names>D</given-names></name><name><surname>El-Deredy</surname><given-names>W</given-names></name></person-group>. <article-title>A comparison between the neural correlates of laser, electric pain stimulation, their modulation by expectation</article-title>. <source>J Neurosci Methods</source> (<year>2018</year>) <volume>293</volume>:<fpage>117</fpage>&#x2013;<lpage>27</lpage>. <pub-id pub-id-type="doi">10.1016/j.jneumeth.2017.09.011</pub-id><pub-id pub-id-type="pmid">28935423</pub-id></citation></ref>
<ref id="B28"><label>28.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Gligorov</surname><given-names>N</given-names></name></person-group>. <article-title>Telling the truth about pain: informed consent and the role of expectation in pain intensity</article-title>. <source>AJOB Neurosci</source> (<year>2018</year>) <volume>9</volume>:<fpage>173</fpage>&#x2013;<lpage>82</lpage>. <pub-id pub-id-type="doi">10.1080/21507740.2018.1496163</pub-id></citation></ref>
<ref id="B29"><label>29.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Anchisi</surname><given-names>D</given-names></name><name><surname>Zanon</surname><given-names>M</given-names></name></person-group>. <article-title>A Bayesian perspective on sensory and cognitive integration in pain perception and placebo analgesia</article-title>. <source>PLoS ONE</source> (<year>2015</year>) <volume>10</volume>:<fpage>e0117270</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0117270</pub-id><pub-id pub-id-type="pmid">25664586</pub-id></citation></ref>
<ref id="B30"><label>30.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ongaro</surname><given-names>G</given-names></name><name><surname>Kaptchuk</surname><given-names>TJ</given-names></name></person-group>. <article-title>Symptom perception, placebo effects, and the Bayesian brain</article-title>. <source>Pain</source> (<year>2019</year>) <volume>160</volume>:<fpage>1</fpage>. <pub-id pub-id-type="doi">10.1097/j.pain.0000000000001367</pub-id><pub-id pub-id-type="pmid">30086114</pub-id></citation></ref>
<ref id="B31"><label>31.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Imamura</surname><given-names>M</given-names></name><name><surname>Chen</surname><given-names>J</given-names></name><name><surname>Matsubayashi</surname><given-names>SR</given-names></name><name><surname>Targino</surname><given-names>RA</given-names></name><name><surname>Alfieri</surname><given-names>FM</given-names></name><name><surname>Bueno</surname><given-names>DK</given-names></name></person-group>, et al. <article-title>Changes in pressure pain threshold in patients with chronic nonspecific low back pain</article-title>. <source>Spine</source> (<year>2013</year>) <volume>38</volume>:<fpage>2098</fpage>&#x2013;<lpage>107</lpage>. <pub-id pub-id-type="doi">10.1097/01.brs.0000435027.50317.d7</pub-id><pub-id pub-id-type="pmid">24026153</pub-id></citation></ref>
<ref id="B32"><label>32.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>Kilner</surname><given-names>J</given-names></name><name><surname>Harrison</surname><given-names>L</given-names></name></person-group>. <article-title>A free energy principle for the brain</article-title>. <source>J Physiol Paris</source> (<year>2006</year>) <volume>100</volume>:<fpage>70</fpage>&#x2013;<lpage>87</lpage>. <pub-id pub-id-type="doi">10.1016/j.jphysparis.2006.10.001</pub-id><pub-id pub-id-type="pmid">17097864</pub-id></citation></ref>
<ref id="B33"><label>33.</label><citation citation-type="other"><person-group person-group-type="author"><name><surname>Endres</surname><given-names>DM</given-names></name><name><surname>Pabst</surname> <given-names>K</given-names></name><name><surname>Eckert</surname><given-names>A-L</given-names></name><name><surname>Schween</surname><given-names>R</given-names></name></person-group>. <comment>A collection of identities for variational inference with exponential-family models</comment>. (<year>2022</year>). <pub-id pub-id-type="doi">10.17192/fdr/112</pub-id></citation></ref>
<ref id="B34"><label>34.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fillingim</surname><given-names>RB</given-names></name></person-group>. <article-title>Individual differences in pain: understanding the mosaic that makes pain personal</article-title>. <source>Pain</source> (<year>2017</year>) <volume>158</volume>:<fpage>S11</fpage>. <pub-id pub-id-type="doi">10.1097/j.pain.0000000000000775</pub-id><pub-id pub-id-type="pmid">27902569</pub-id></citation></ref>
<ref id="B35"><label>35.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fine</surname><given-names>S</given-names></name><name><surname>Singer</surname><given-names>Y</given-names></name><name><surname>Tishby</surname><given-names>N</given-names></name></person-group>. <article-title>The hierarchical hidden Markov model: analysis, applications</article-title>. <source>Mach Learn</source> (<year>1998</year>) <volume>32</volume>:<fpage>41</fpage>&#x2013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1023/A:1007469218079</pub-id></citation></ref>
<ref id="B36"><label>36.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Bishop</surname><given-names>CM</given-names></name></person-group>, <source>Pattern recognition, machine learning</source>, <publisher-loc>New York</publisher-loc>: <publisher-name>Springer</publisher-name> (<year>2006</year>).</citation></ref>
<ref id="B37"><label>37.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Schnitzler</surname><given-names>A</given-names></name><name><surname>Ploner</surname><given-names>M</given-names></name></person-group>. <article-title>Neurophysiology and functional neuroanatomy of pain perception</article-title>. <source>J Clin Neurophysiol</source> (<year>2000</year>) <volume>17</volume>:<fpage>592</fpage>&#x2013;<lpage>603</lpage>. <pub-id pub-id-type="doi">10.1097/00004691-200011000-00005</pub-id><pub-id pub-id-type="pmid">11151977</pub-id></citation></ref>
<ref id="B38"><label>38.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Hohwy</surname><given-names>J</given-names></name></person-group>, <source>The predictive mind</source>. <publisher-name>Oxford: Oxford University Press</publisher-name> (<year>2013</year>).</citation></ref>
<ref id="B39"><label>39.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Pearl</surname><given-names>J</given-names></name></person-group>, <source>Reverend Bayes on inference engines: a distributed hierarchical approach</source>. <publisher-name>Menlo Park: Cognitive Systems Laboratory, School of Engineering, Applied Science</publisher-name> (<year>1982</year>).</citation></ref>
<ref id="B40"><label>40.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name></person-group>. <article-title>A theory of cortical responses</article-title>. <source>Philos Trans R Soc Lond B Biol Sci</source> (<year>2005</year>) <volume>360</volume>:<fpage>815</fpage>&#x2013;<lpage>36</lpage>. <pub-id pub-id-type="doi">10.1098/rstb.2005.1622</pub-id><pub-id pub-id-type="pmid">15937014</pub-id></citation></ref>
<ref id="B41"><label>41.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Harris</surname><given-names>CR</given-names></name><name><surname>Millman</surname><given-names>KJ</given-names></name><name><surname>van der Walt</surname><given-names>SJ</given-names></name><name><surname>Gommers</surname><given-names>R</given-names></name><name><surname>Virtanen</surname><given-names>P</given-names></name><name><surname>Cournapeau</surname><given-names>D</given-names></name></person-group>, et al. <article-title>Array programming with NumPy</article-title>. <source>Nature</source> (<year>2020</year>) <volume>585</volume>:<fpage>357</fpage>&#x2013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1038/s41586-020-2649-2</pub-id><pub-id pub-id-type="pmid">32939066</pub-id></citation></ref>
<ref id="B42"><label>42.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hunter</surname><given-names>JD</given-names></name></person-group>. <article-title>Matplotlib: a 2D graphics environment</article-title>. <source>Comput Sci Eng</source> (<year>2007</year>) <volume>9</volume>:<fpage>90</fpage>&#x2013;<lpage>5</lpage>. <pub-id pub-id-type="doi">10.1109/MCSE.2007.55</pub-id></citation></ref>
<ref id="B43"><label>43.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Borsook</surname><given-names>D</given-names></name><name><surname>Youssef</surname><given-names>AM</given-names></name><name><surname>Simons</surname><given-names>L</given-names></name><name><surname>Elman</surname><given-names>I</given-names></name><name><surname>Eccleston</surname><given-names>C</given-names></name></person-group>. <article-title>When pain gets stuck: the evolution of pain chronification, treatment resistance</article-title>. <source>Pain</source> (<year>2018</year>) <volume>159</volume>:<fpage>2421</fpage>. <pub-id pub-id-type="doi">10.1097/j.pain.0000000000001401</pub-id><pub-id pub-id-type="pmid">30234696</pub-id></citation></ref>
<ref id="B44"><label>44.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Morrone</surname><given-names>L</given-names></name><name><surname>Scuteri</surname><given-names>D</given-names></name><name><surname>Rombola</surname><given-names>L</given-names></name><name><surname>Mizoguchi</surname><given-names>H</given-names></name><name><surname>Bagetta</surname><given-names>G</given-names></name></person-group>. <article-title>Opioids resistance in chronic pain management</article-title>. <source>Curr Neuropharmacol</source> (<year>2017</year>) <volume>15</volume>:<fpage>444</fpage>&#x2013;<lpage>56</lpage>. <pub-id pub-id-type="doi">10.2174/1570159X14666161101092822</pub-id><pub-id pub-id-type="pmid">28503117</pub-id></citation></ref>
<ref id="B45"><label>45.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lefaucheur</surname><given-names>J-P</given-names></name><name><surname>Nguyen</surname><given-names>J-P</given-names></name></person-group>. <article-title>A practical algorithm for using rTMS to treat patients with chronic pain</article-title>. <source>Neurophysiol Clin</source> (<year>2019</year>) <volume>49</volume>:<fpage>301</fpage>&#x2013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1016/j.neucli.2019.07.014</pub-id><pub-id pub-id-type="pmid">31375381</pub-id></citation></ref>
<ref id="B46"><label>46.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Suda</surname><given-names>S</given-names></name><name><surname>Takagai</surname><given-names>S</given-names></name><name><surname>Inoshima-Takahashi</surname><given-names>K</given-names></name><name><surname>Sugihara</surname><given-names>G</given-names></name><name><surname>Mori</surname><given-names>N</given-names></name><name><surname>Takei</surname><given-names>N</given-names></name></person-group>. <article-title>Electroconvulsive therapy for burning mouth syndrome</article-title>. <source>Acta Psychiatr Scand</source> (<year>2008</year>) <volume>118</volume>:<fpage>503</fpage>&#x2013;<lpage>4</lpage>. <pub-id pub-id-type="doi">10.1111/j.1600-0447.2008.01261.x</pub-id><pub-id pub-id-type="pmid">18764838</pub-id></citation></ref>
<ref id="B47"><label>47.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pickering</surname><given-names>G</given-names></name><name><surname>Pereira</surname><given-names>B</given-names></name><name><surname>Morel</surname><given-names>V</given-names></name><name><surname>Corriger</surname><given-names>A</given-names></name><name><surname>Giron</surname><given-names>F</given-names></name><name><surname>Marcaillou</surname><given-names>F</given-names></name></person-group>, et al. <article-title>Ketamine and magnesium for refractory neuropathic pain: a randomized, double-blind, crossover trial</article-title>. <source>Anesthesiology</source> (<year>2020</year>) <volume>133</volume>:<fpage>154</fpage>&#x2013;<lpage>64</lpage>. <pub-id pub-id-type="doi">10.1097/ALN.0000000000003345</pub-id><pub-id pub-id-type="pmid">32384291</pub-id></citation></ref>
<ref id="B48"><label>48.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Greub</surname><given-names>WH</given-names></name></person-group>, <source>Linear algebra</source>. Vol. <volume>23</volume>. <publisher-name>New York: Springer Science / Business Media</publisher-name> (<year>2012</year>).</citation></ref>
<ref id="B49"><label>49.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Glombiewski</surname><given-names>JA</given-names></name><name><surname>Holzapfel</surname><given-names>S</given-names></name><name><surname>Riecke</surname><given-names>J</given-names></name><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>de Jong</surname><given-names>J</given-names></name><name><surname>Lemmer</surname><given-names>G</given-names></name></person-group>, et al. <article-title>Exposure and CBT for chronic back pain: an RCT on differential efficacy and optimal length of treatment</article-title>. <source>J Consult Clin Psychol</source> (<year>2018</year>) <volume>86</volume>:<fpage>533</fpage>. <pub-id pub-id-type="doi">10.1037/ccp0000298</pub-id><pub-id pub-id-type="pmid">29781651</pub-id></citation></ref>
<ref id="B50"><label>50.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tabor</surname><given-names>A</given-names></name><name><surname>Burr</surname><given-names>C</given-names></name></person-group>. <article-title>Bayesian learning models of pain: a call to action</article-title>. <source>Curr Opin Behav Sci</source> (<year>2019</year>) <volume>26</volume>:<fpage>54</fpage>&#x2013;<lpage>61</lpage>. <pub-id pub-id-type="doi">10.1016/j.cobeha.2018.10.006</pub-id></citation></ref>
<ref id="B51"><label>51.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Rief</surname><given-names>W</given-names></name><name><surname>Glombiewski</surname><given-names>JA</given-names></name><name><surname>Gollwitzer</surname><given-names>M</given-names></name><name><surname>Schub&#x00F6;</surname><given-names>A</given-names></name><name><surname>Schwarting</surname><given-names>R</given-names></name><name><surname>Thorwart</surname><given-names>A</given-names></name></person-group>. <article-title>Expectancies as core features of mental disorders</article-title>. <source>Curr Opin Psychiatry</source> (<year>2015</year>) <volume>28</volume>:<fpage>378</fpage>&#x2013;<lpage>85</lpage>. <pub-id pub-id-type="doi">10.1097/YCO.0000000000000184</pub-id><pub-id pub-id-type="pmid">26164612</pub-id></citation></ref>
<ref id="B52"><label>52.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Victoria</surname><given-names>NC</given-names></name><name><surname>Murphy</surname><given-names>AZ</given-names></name></person-group>. <article-title>Exposure to early life pain: long term consequences and contributing mechanisms</article-title>. <source>Curr Opin Behav Sci</source> (<year>2016</year>) <volume>7</volume>:<fpage>61</fpage>&#x2013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.1016/j.cobeha.2015.11.015</pub-id><pub-id pub-id-type="pmid">27525299</pub-id></citation></ref>
<ref id="B53"><label>53.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Tracey</surname><given-names>I</given-names></name></person-group>. <article-title>Getting the pain you expect: mechanisms of placebo, nocebo and reappraisal effects in humans</article-title>. <source>Nat Med</source> (<year>2010</year>) <volume>16</volume>:<fpage>1277</fpage>&#x2013;<lpage>83</lpage>. <pub-id pub-id-type="doi">10.1038/nm.2229</pub-id><pub-id pub-id-type="pmid">20948533</pub-id></citation></ref>
<ref id="B54"><label>54.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Edwards</surname><given-names>MJ</given-names></name><name><surname>Adams</surname><given-names>RA</given-names></name><name><surname>Brown</surname><given-names>H</given-names></name><name><surname>Parees</surname><given-names>I</given-names></name><name><surname>Friston</surname><given-names>KJ</given-names></name></person-group>. <article-title>A Bayesian account of &#x201C;hysteria&#x201D;</article-title>. <source>Brain</source> (<year>2012</year>) <volume>135</volume>:<fpage>3495</fpage>&#x2013;<lpage>512</lpage>. <pub-id pub-id-type="doi">10.1093/brain/aws129</pub-id><pub-id pub-id-type="pmid">22641838</pub-id></citation></ref>
<ref id="B55"><label>55.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Flor</surname><given-names>H</given-names></name><name><surname>Turk</surname><given-names>D</given-names></name></person-group>. <source>Chronic pain: an integrated biobehavioral approach</source>. Washington: IASP Press (<year>2015</year>).</citation></ref>
<ref id="B56"><label>56.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Hohwy</surname><given-names>J</given-names></name></person-group>, <source>The predictive mind</source>. <publisher-name>Aalphen an de Rijn: Oxford University Press</publisher-name> (<year>2013</year>).</citation></ref>
<ref id="B57"><label>57.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Wiech</surname><given-names>K</given-names></name><name><surname>Tracey</surname><given-names>I</given-names></name></person-group>. <article-title>Pain, decisions, and actions: a motivational perspective</article-title>. <source>Front Neurosci</source> (<year>2013</year>) <volume>7</volume>:<fpage>46</fpage>. <pub-id pub-id-type="doi">10.3389/fnins.2013.00046</pub-id><pub-id pub-id-type="pmid">23565073</pub-id></citation></ref>
<ref id="B58"><label>58.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>FitzGerald</surname><given-names>T</given-names></name><name><surname>Rigoli</surname><given-names>F</given-names></name><name><surname>Schwartenbeck</surname><given-names>P</given-names></name><name><surname>Pezzulo</surname><given-names>G</given-names></name></person-group>. <article-title>Active inference: a process theory</article-title>. <source>Neural Comput</source> (<year>2017</year>) <volume>29</volume>:<fpage>1</fpage>&#x2013;<lpage>49</lpage>. <pub-id pub-id-type="doi">10.1162/NECO_a_00912</pub-id><pub-id pub-id-type="pmid">27870614</pub-id></citation></ref>
<ref id="B59"><label>59.</label><citation citation-type="other"><person-group person-group-type="author"><name><surname>Heins</surname><given-names>C</given-names></name><name><surname>Millidge</surname><given-names>B</given-names></name><name><surname>Demekas</surname><given-names>D</given-names></name><name><surname>Klein</surname><given-names>B</given-names></name><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>Couzin</surname><given-names>I</given-names></name></person-group>, et al. <comment>pymdp: a Python library for active inference in discrete state spaces. [Preprint] (2022). Available at: arXiv:2201.03904</comment>.</citation></ref>
<ref id="B60"><label>60.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Pearl</surname><given-names>J</given-names></name></person-group>. <source>Causality</source>. <publisher-name>Cambridge: Cambridge University Press</publisher-name> (<year>2009</year>).</citation></ref>
<ref id="B61"><label>61.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ehde</surname><given-names>DM</given-names></name><name><surname>Dillworth</surname><given-names>TM</given-names></name><name><surname>Turner</surname><given-names>JA</given-names></name></person-group>. <article-title>Cognitive-behavioral therapy for individuals with chronic pain: efficacy, innovations, and directions for research</article-title>. <source>Am Psychol</source> (<year>2014</year>) <volume>69</volume>:<fpage>153</fpage>. <pub-id pub-id-type="doi">10.1037/a0035747</pub-id><pub-id pub-id-type="pmid">24547801</pub-id></citation></ref>
<ref id="B62"><label>62.</label><citation citation-type="other"><person-group person-group-type="author"><name><surname>Beehler</surname><given-names>G</given-names></name><name><surname>Murphy</surname><given-names>JL</given-names></name><name><surname>King</surname><given-names>PR</given-names></name><name><surname>Dollar</surname><given-names>KM</given-names></name></person-group>. <comment>Brief cognitive behavioral therapy for chronic pain: therapist manual. US Department of Veterans Affairs</comment>.</citation></ref>
<ref id="B63"><label>63.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>Morley</surname><given-names>S</given-names></name><name><surname>Linton</surname><given-names>SJ</given-names></name><name><surname>Boersma</surname><given-names>K</given-names></name><name><surname>de Jong</surname><given-names>J</given-names></name></person-group>, <source>Pain-related fear: exposure-based treatment of chronic pain</source>. <publisher-name>Washington, D.C.: IASP Press</publisher-name> (<year>2012</year>).</citation></ref>
<ref id="B64"><label>64.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Trost</surname><given-names>Z</given-names></name><name><surname>Parsons</surname><given-names>TD</given-names></name></person-group>. <article-title>Beyond distraction: virtual reality graded exposure therapy as treatment for pain-related fear and disability in chronic pain</article-title>. <source>J Appl Biobehav Res</source> (<year>2014</year>) <volume>19</volume>:<fpage>106</fpage>&#x2013;<lpage>26</lpage>. <pub-id pub-id-type="doi">10.1111/jabr.12021</pub-id></citation></ref>
<ref id="B65"><label>65.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>Kole-Snijders</surname><given-names>AM</given-names></name><name><surname>Rotteveel</surname><given-names>AM</given-names></name><name><surname>Ruesink</surname><given-names>R</given-names></name><name><surname>Heuts</surname><given-names>PH</given-names></name></person-group>. <article-title>The role of fear of movement/(re) injury in pain disability</article-title>. <source>J Occup Rehabil</source> (<year>1995</year>) <volume>5</volume>:<fpage>235</fpage>&#x2013;<lpage>52</lpage>. <pub-id pub-id-type="doi">10.1007/BF02109988</pub-id><pub-id pub-id-type="pmid">24234727</pub-id></citation></ref>
<ref id="B66"><label>66.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Vlaeyen</surname><given-names>JW</given-names></name><name><surname>Crombez</surname><given-names>G</given-names></name><name><surname>Linton</surname><given-names>SJ</given-names></name></person-group>. <article-title>The fear-avoidance model of pain</article-title>. <source>Pain</source> (<year>2016</year>) <volume>157</volume>:<fpage>1588</fpage>&#x2013;<lpage>89</lpage>. <pub-id pub-id-type="doi">10.1097/j.pain.0000000000000574</pub-id><pub-id pub-id-type="pmid">27428892</pub-id></citation></ref>
<ref id="B67"><label>67.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Finnerup</surname><given-names>NB</given-names></name><name><surname>Sindrup</surname><given-names>SH</given-names></name><name><surname>Jensen</surname><given-names>TS</given-names></name></person-group>. <article-title>Chronic neuropathic pain: mechanisms, drug targets and measurement</article-title>. <source>Fundam Clin Pharmacol</source> (<year>2007</year>) <volume>21</volume>:<fpage>129</fpage>&#x2013;<lpage>36</lpage>. <pub-id pub-id-type="doi">10.1111/j.1472-8206.2007.00474.x</pub-id><pub-id pub-id-type="pmid">17391285</pub-id></citation></ref>
<ref id="B68"><label>68.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Davis</surname><given-names>DA</given-names></name><name><surname>Luecken</surname><given-names>LJ</given-names></name><name><surname>Zautra</surname><given-names>AJ</given-names></name></person-group>. <article-title>Are reports of childhood abuse related to the experience of chronic pain in adulthood?: a meta-analytic review of the literature</article-title>. <source>Clin J Pain</source> (<year>2005</year>) <volume>21</volume>:<fpage>398</fpage>&#x2013;<lpage>405</lpage>. <pub-id pub-id-type="doi">10.1097/01.ajp.0000149795.08746.31</pub-id><pub-id pub-id-type="pmid">16093745</pub-id></citation></ref>
<ref id="B69"><label>69.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lampe</surname><given-names>A</given-names></name><name><surname>Doering</surname><given-names>S</given-names></name><name><surname>Rumpold</surname><given-names>G</given-names></name><name><surname>S&#x00F6;lder</surname><given-names>E</given-names></name><name><surname>Krismer</surname><given-names>M</given-names></name><name><surname>Kantner-Rumplmair</surname><given-names>W</given-names></name></person-group>, et al. <article-title>Chronic pain syndromes and their relation to childhood abuse and stressful life events</article-title>. <source>J Psychosom Res</source> (<year>2003</year>) <volume>54</volume>:<fpage>361</fpage>&#x2013;<lpage>7</lpage>. _ <pub-id pub-id-type="doi">10.1016/S0022-3999(02)00399-9</pub-id><pub-id pub-id-type="pmid">12670615</pub-id></citation></ref>
<ref id="B70"><label>70.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Goldberg</surname><given-names>RT</given-names></name><name><surname>Pachasoe</surname><given-names>WN</given-names></name><name><surname>Keith</surname><given-names>D</given-names></name></person-group>. <article-title>Relationship between traumatic events in childhood and chronic pain</article-title>. <source>Disabil Rehabil</source> (<year>1999</year>) <volume>21</volume>:<fpage>23</fpage>&#x2013;<lpage>30</lpage>. <pub-id pub-id-type="doi">10.1080/096382899298061</pub-id><pub-id pub-id-type="pmid">10070600</pub-id></citation></ref>
<ref id="B71"><label>71.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Panitz</surname><given-names>C</given-names></name><name><surname>Endres</surname><given-names>DM</given-names></name><name><surname>Buchholz</surname><given-names>M</given-names></name><name><surname>Khosrowtaj</surname><given-names>Z</given-names></name><name><surname>Sperl</surname><given-names>MF</given-names></name><name><surname>Mueller</surname><given-names>EM</given-names></name></person-group>, et al. <article-title>A revised framework for the investigation of expectation update versus maintenance in the context of expectation violations: the violex 2.0 model</article-title>. <source>Front Psychol</source> (<year>2021</year>) <volume>13</volume>(<issue>11</issue>):<fpage>5237</fpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2021.726432</pub-id></citation></ref>
<ref id="B72"><label>72.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Noorshams</surname><given-names>N</given-names></name><name><surname>Wainwright</surname><given-names>MJ</given-names></name></person-group>. <article-title>Belief propagation for continuous state spaces: stochastic message-passing with quantitative guarantees</article-title>. <source>J Mach Learn Res</source> (<year>2013</year>) <volume>14</volume>:<fpage>2799</fpage>&#x2013;<lpage>835</lpage>.</citation></ref>
<ref id="B73"><label>73.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>KJ</given-names></name><name><surname>Parr</surname><given-names>T</given-names></name><name><surname>de Vries</surname><given-names>B</given-names></name></person-group>. <article-title>The graphical brain: belief propagation and active inference</article-title>. <source>Netw Neurosci</source> (<year>2017</year>) <volume>1</volume>:<fpage>381</fpage>&#x2013;<lpage>414</lpage>. <pub-id pub-id-type="doi">10.1162/NETN_a_00018</pub-id><pub-id pub-id-type="pmid">29417960</pub-id></citation></ref>
<ref id="B74"><label>74.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>von Mohr</surname><given-names>M</given-names></name><name><surname>Krah&#x00E9;</surname><given-names>C</given-names></name><name><surname>Beck</surname><given-names>B</given-names></name><name><surname>Fotopoulou</surname><given-names>A</given-names></name></person-group>. <article-title>The social buffering of pain by affective touch: a laser-evoked potential study in romantic couples</article-title>. <source>Soc Cogn Affect Neurosci</source> (<year>2018</year>) <volume>13</volume>:<fpage>1121</fpage>&#x2013;<lpage>30</lpage>. <pub-id pub-id-type="doi">10.1093/scan/nsy085</pub-id><pub-id pub-id-type="pmid">30247679</pub-id></citation></ref>
<ref id="B75"><label>75.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Fletcher</surname><given-names>PC</given-names></name><name><surname>Frith</surname><given-names>CD</given-names></name></person-group>. <article-title>Perceiving is believing: a Bayesian approach to explaining the positive symptoms of schizophrenia</article-title>. <source>Nat Rev Neurosci</source> (<year>2009</year>) <volume>10</volume>:<fpage>48</fpage>&#x2013;<lpage>58</lpage>. <pub-id pub-id-type="doi">10.1038/nrn2536</pub-id><pub-id pub-id-type="pmid">19050712</pub-id></citation></ref>
<ref id="B76"><label>76.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sterzer</surname><given-names>P</given-names></name><name><surname>Adams</surname><given-names>RA</given-names></name><name><surname>Fletcher</surname><given-names>P</given-names></name><name><surname>Frith</surname><given-names>C</given-names></name><name><surname>Lawrie</surname><given-names>SM</given-names></name><name><surname>Muckli</surname><given-names>L</given-names></name></person-group>, et al. <article-title>The predictive coding account of psychosis</article-title>. <source>Biol Psychiatry</source> (<year>2018</year>) <volume>84</volume>:<fpage>634</fpage>&#x2013;<lpage>43</lpage>. <pub-id pub-id-type="doi">10.1016/j.biopsych.2018.05.015</pub-id><pub-id pub-id-type="pmid">30007575</pub-id></citation></ref>
<ref id="B77"><label>77.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Huys</surname><given-names>QJ</given-names></name><name><surname>Browning</surname><given-names>M</given-names></name><name><surname>Paulus</surname><given-names>MP</given-names></name><name><surname>Frank</surname><given-names>MJ</given-names></name></person-group>. <article-title>Advances in the computational understanding of mental illness</article-title>. <source>Neuropsychopharmacology</source> (<year>2021</year>) <volume>46</volume>:<fpage>3</fpage>&#x2013;<lpage>19</lpage>. <pub-id pub-id-type="doi">10.1038/s41386-020-0746-4</pub-id><pub-id pub-id-type="pmid">32620005</pub-id></citation></ref>
<ref id="B78"><label>78.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Friston</surname><given-names>K</given-names></name><name><surname>Mattout</surname><given-names>J</given-names></name><name><surname>Trujillo-Barreto</surname><given-names>N</given-names></name><name><surname>Ashburner</surname><given-names>J</given-names></name><name><surname>Penny</surname><given-names>W</given-names></name></person-group>. <article-title>Variational free energy and the laplace approximation</article-title>. <source>Neuroimage</source> (<year>2007</year>) <volume>34</volume>:<fpage>220</fpage>&#x2013;<lpage>34</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2006.08.035</pub-id><pub-id pub-id-type="pmid">17055746</pub-id></citation></ref>
<ref id="B79"><label>79.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Parr</surname><given-names>T</given-names></name><name><surname>Friston</surname><given-names>KJ</given-names></name></person-group>. <article-title>Generalised free energy and active inference</article-title>. <source>Biol Cybern</source> (<year>2019</year>) <volume>113</volume>:<fpage>495</fpage>&#x2013;<lpage>513</lpage>. <pub-id pub-id-type="doi">10.1007/s00422-019-00805-w</pub-id><pub-id pub-id-type="pmid">31562544</pub-id></citation></ref>
<ref id="B80"><label>80.</label><citation citation-type="other"><person-group person-group-type="author"><name><surname>Yang</surname><given-names>X</given-names></name></person-group>. <comment>Understanding the variational lower bound. In: <italic>variational lower bound, ELBO, hard attention</italic>. (2017) p. 1&#x2013;4</comment>.</citation></ref>
<ref id="B81"><label>81.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kullback</surname><given-names>S</given-names></name><name><surname>Leibler</surname><given-names>RA</given-names></name></person-group>. <article-title>On information and sufficiency</article-title>. <source>Ann Math Stat</source> (<year>1951</year>) <volume>22</volume>:<fpage>79</fpage>&#x2013;<lpage>86</lpage>. <pub-id pub-id-type="doi">10.1214/aoms/1177729694</pub-id></citation></ref>
<ref id="B82"><label>82.</label><citation citation-type="journal"><person-group person-group-type="author"><name><surname>Jensen</surname><given-names>JLWV</given-names></name></person-group>. <article-title>Sur les fonctions convexes et les in&#x00E9;galit&#x00E9;s entre les valeurs moyennes</article-title>. <source>Acta Math</source> (<year>1906</year>) <volume>30</volume>:<fpage>175</fpage>&#x2013;<lpage>93</lpage>. <pub-id pub-id-type="doi">10.1007/BF02418571</pub-id></citation></ref>
<ref id="B83"><label>83.</label><citation citation-type="book"><person-group person-group-type="author"><name><surname>Kolmogorov</surname><given-names>AN</given-names></name><name><surname>Bharucha-Reid</surname><given-names>AT</given-names></name></person-group>, <source>Foundations of the theory of probability</source>. <edition>2nd English ed</edition>. <publisher-name>Mineola, N.Y.: Courier Dover Publications</publisher-name> (<year>2018</year>).</citation></ref></ref-list>
<app-group><app id="app1"><title>Appendix A. Variational inference</title>
<p>Graphical models are a useful tool when representing and manipulating joint probability distributions. In the present context, we are interested in characterizing the mind&#x2019;s probability computations of being in pain, given a series of noxious or harmless observations, i.e. <italic>inference</italic> (<xref ref-type="bibr" rid="B36">36</xref>). For this purpose, we use a modified state-space model architecture, i.e. a Hidden Markov Model (<xref ref-type="bibr" rid="B36">36</xref>). It features a chain of unobservable nodes, representing the inferred pain state at different time points. Each unobservable node is connected to one observable node representing sensations at the same time point (see <xref ref-type="fig" rid="F7">Figure&#x00A0;A1</xref>).</p>
<fig id="F7" position="float"><label>Figure A1</label>
<caption><p>Graphical model (fragment). <bold>Graphical model</bold> depicting the inter-dependencies between hidden <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM111"><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> and observable <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM112"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> variables. Connected to each variable node, factor nodes are necessary to perform variational inference message passing. This approach is computationally efficient because it allows message passing without re-instantiating the model at each time-step. The observable nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM113"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> represent incoming sensory information, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM114"><mml:mi>S</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mtext>noxious</mml:mtext><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy="false">)</mml:mo><mml:mo>,</mml:mo><mml:mtext>harmless</mml:mtext><mml:mo stretchy="false">(</mml:mo><mml:mn>0</mml:mn><mml:mo stretchy="false">)</mml:mo><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula>. The hidden states, representing the system&#x2019;s internal model, can in turn take on the values 0 (no pain) or 1 (pain), or <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM115"><mml:mi>H</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>,</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula>. The likelihood <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM116"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is the top-down factor in this model and allows quantifying the association between the model&#x2019;s state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM117"><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> and the observed sensory input at <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM118"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>. The hidden variable nodes form a Markov chain and contain transition probabilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM119"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, which can be interpreted as the development of the pain prior over time. In individuals with chronic pain, we here assume that <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM120"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x003E;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>.</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g007.tif"/>
</fig>
<p>Assuming that hidden state inference evolves in an auto-correlated manner over time seems plausible because pain states tend to persist at least for some time. For this reason, a Markov chain of hidden states was selected as the basis of the model. The Markov property states that inference at a given future node, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM121"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula>, is conditionally independent (&#x22A5;) from all but the previous states <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM122"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>:<disp-formula id="disp-formula2"><label>(A1)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM2"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x22A5;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></disp-formula>In other words, the future state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM123"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> does not depend on the distribution of the past state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM124"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> given the current state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM125"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula>.</p>
<p>It follows that the conditional distribution for the current state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM126"><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> given <italic>all</italic> previous <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM127"><mml:mi>S</mml:mi></mml:math></inline-formula> is given by only its predecessor&#x2019;s state <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM128"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula>:<disp-formula id="disp-formula3"><label>(A2)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM3"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>,</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>The main inference goal is the estimation of the hidden random variables&#x2019; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM129"><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub></mml:math></inline-formula> state, given a series of observations. This is expressed in the random variable&#x2019;s respective marginal probabilities. Computing marginal probabilities is challenging, because it usually entails computing the sums of integrals over high-dimensional spaces. There are a number of widely used algorithms for efficient inference on graphical models. However, to perform inference with message-passing algorithms, we first need to transform the Bayesian network to a factor graph. A graphical representation of the factor graph is illustrated in <xref ref-type="fig" rid="F7">Figure&#x00A0;A1</xref> (<xref ref-type="bibr" rid="B36">36</xref>). A factor graph represents the factorization of a function, or here, the probability distributions of interest. In other words, it provides a graphical representation of how probabilities of certain events depend on adjacent events in the observable and hidden layers. A factor graph consists of variable- and factor nodes, denoted by circles and squares, respectively. In our model, variable nodes can take on a binary set of values (here; <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM130"><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo>,</mml:mo><mml:mover><mml:mrow><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:mi>n</mml:mi></mml:mrow><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover></mml:math></inline-formula> for the hidden nodes, and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM131"><mml:mi>n</mml:mi><mml:mi>o</mml:mi><mml:mi>x</mml:mi><mml:mi>i</mml:mi><mml:mi>o</mml:mi><mml:mi>u</mml:mi><mml:mi>s</mml:mi><mml:mo>,</mml:mo><mml:mi>h</mml:mi><mml:mi>a</mml:mi><mml:mi>r</mml:mi><mml:mi>m</mml:mi><mml:mi>l</mml:mi><mml:mi>e</mml:mi><mml:mi>s</mml:mi><mml:mi>s</mml:mi></mml:math></inline-formula> input in the observable nodes). They can be observed or unobserved. Factor nodes contain exponential family distributions, parameterized by their natural parameters. These parameters represent probabilities for either transitioning between different hidden states or for observing certain types of sensory information, given a specific latent model state. We here further introduce hypernodes, denoted by thick squares, that govern the learning of factor-inherent natural parameters over time. All nodes are connected via edges to their respective neighbour nodes.</p>
<p>Once transformed, it is now appropriate to apply inference algorithms to the factorized graphical model. Sum-product message passing, also known as belief propagation, is a well-known algorithm that allows the efficient computation of marginal probability distributions on graphical networks without loops (<xref ref-type="bibr" rid="B39">39</xref>). In sum-product message passing, all nodes compute sums and integrals locally, before passing the results on towards the set of their neighbours in the shape of messages (<xref ref-type="bibr" rid="B72">72</xref>). In the present use-case, all variables are binary. In sum-product message-passing, the marginal probabilities for each state of a variable node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM132"><mml:mi>V</mml:mi></mml:math></inline-formula> are given by the product of all incoming messages from factor nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM133"><mml:mi>F</mml:mi></mml:math></inline-formula>:<disp-formula id="disp-formula4"><label>(A3)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM4"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:munder><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM134"><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> are the are the factor nodes&#x2019; neighbouring the variable node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM135"><mml:mi>V</mml:mi></mml:math></inline-formula>, and conversely for factor nodes. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM136"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is the message received from factor node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM137"><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub></mml:math></inline-formula>. Variable nodes can only have factor nodes as neighbours. Messages from a factor node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM138"><mml:mi>F</mml:mi></mml:math></inline-formula> to a variable node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM139"><mml:mi>V</mml:mi></mml:math></inline-formula> are given by<disp-formula id="disp-formula5"><label>(A4)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM5"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mi>F</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow></mml:munder><mml:mo>&#x22EF;</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mi>M</mml:mi></mml:msub></mml:mrow></mml:munder><mml:mi>F</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo>,</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>,</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>M</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:munder><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>m</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi mathvariant="normal">&#x2216;</mml:mi><mml:mi>V</mml:mi></mml:mrow></mml:munder><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>F</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>Here, the product of the factor and all messages the factor node received from its neighbouring variables nodes except <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM140"><mml:mi>V</mml:mi></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM141"><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi mathvariant="normal">&#x2216;</mml:mi><mml:mi>V</mml:mi></mml:math></inline-formula> (a set of variable nodes without the recipient node), <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM142"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>F</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is computed and summed over all variables except <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM143"><mml:mi>V</mml:mi></mml:math></inline-formula>. When <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM144"><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mrow><mml:mi>N</mml:mi><mml:mi>o</mml:mi><mml:mi>n</mml:mi><mml:mi>e</mml:mi></mml:mrow></mml:math></inline-formula>, that is, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM145"><mml:mi>F</mml:mi></mml:math></inline-formula>&#x2019;s only neighbour is <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM146"><mml:mi>V</mml:mi></mml:math></inline-formula>, the factor node sends its factor information:<disp-formula id="disp-formula6"><label>(A5)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM6"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mi>F</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>.</mml:mo></mml:math></disp-formula>Messages from variable node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM147"><mml:mi>V</mml:mi></mml:math></inline-formula> to a neighbouring factor node are given by<disp-formula id="disp-formula7"><label>(A6)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM7"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mi>V</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>F</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:munder><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi mathvariant="normal">&#x2216;</mml:mi><mml:mi>F</mml:mi></mml:mrow></mml:munder><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub></mml:math></disp-formula>that is, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM148"><mml:mi>V</mml:mi></mml:math></inline-formula> computes the product of all received messages from neighbouring factor nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM149"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> except <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM150"><mml:mi>F</mml:mi></mml:math></inline-formula>. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM151"><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi mathvariant="normal">&#x2216;</mml:mi><mml:mi>F</mml:mi></mml:math></inline-formula> is empty when <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM152"><mml:mi>F</mml:mi></mml:math></inline-formula> is the only neighbour of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM153"><mml:mi>V</mml:mi></mml:math></inline-formula>, in which case the message reduces to<disp-formula id="disp-formula8"><label>(A7)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM8"><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:mi>V</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>F</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></disp-formula>where the value 1 is sent from unobserved nodes. Otherwise, the message value corresponds to the observation made at <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM154"><mml:mi>V</mml:mi></mml:math></inline-formula>. Note that variable nodes can also remain unobserved.</p>
<p>We here adapt this scheme and transform messages into log-space to avoid over- or underflow of the usual floating point computer arithmetic. With this, the marginal of a variable <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM155"><mml:mi>V</mml:mi></mml:math></inline-formula> is now computed via<disp-formula id="disp-formula9"><label>(A8)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM9"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>l</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:munder><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>)</mml:mo></mml:mrow><mml:mo>,</mml:mo></mml:math></disp-formula>i.e., the marginal of a variable node is given by the sum of all incoming, log-transformed messages (<inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM156"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>V</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:mi>V</mml:mi></mml:math></inline-formula>)) from factor nodes <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM157"><mml:msub><mml:mi>F</mml:mi><mml:mi>l</mml:mi></mml:msub></mml:math></inline-formula>.</p>
<p>Exact inference can become computationally inefficient or intractable in case of continuous variables. This is the case in our model due to the natural parameters of the probability distributions. To avoid this, we use variational inference and the lower bound approximation as implemented in the free-energy framework (<xref ref-type="bibr" rid="B78">78</xref>, <xref ref-type="bibr" rid="B79">79</xref>, <xref ref-type="bibr" rid="B36">36</xref>). Instead of estimating the (potentially intractable) posterior <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM158"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, with <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM159"><mml:mi>X</mml:mi></mml:math></inline-formula> representing model parameters and data <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM160"><mml:mi>D</mml:mi></mml:math></inline-formula>, an approximate distribution <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM161"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is estimated. This further sidesteps the increasingly costly computation of sums over all variables and their values by introducing more simple update rules in order to compute the posterior distribution. The optimal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM162"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is estimated by maximizing the lower bound on the marginal log-likelihood of the data <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM163"><mml:mi>D</mml:mi></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM164"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, also referred to as the evidence lower bound (ELBO, (<xref ref-type="bibr" rid="B80">80</xref>)). In the present case, the marginal log-likelihood of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM165"><mml:mi>D</mml:mi></mml:math></inline-formula> is given by<disp-formula id="disp-formula10"><label>(A9)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM10"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>,</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>Meaning that the marginal log-likelihood of the data is given by the sum over all joint probability distributions <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM166"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>,</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, or the sum of an approximate distribution of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM167"><mml:mi>X</mml:mi></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM168"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, and conditional probabilities of the data given <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM169"><mml:mi>X</mml:mi></mml:math></inline-formula>, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM170"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and a normalization factor <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM171"><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mo>/</mml:mo></mml:mrow><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:math></inline-formula>. The last term of <xref ref-type="disp-formula" rid="disp-formula21">Equation A9</xref> represents a factorization of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM172"><mml:mrow><mml:mi>l</mml:mi><mml:mi>o</mml:mi><mml:mi>g</mml:mi></mml:mrow><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>.</p>
<p>When trying to approximate <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM173"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, there are two essential ingredients: Jensen&#x2019;s inequality and the Kullback-Leibler divergence (<xref ref-type="bibr" rid="B81">81</xref>, <xref ref-type="bibr" rid="B82">82</xref>, <xref ref-type="bibr" rid="B36">36</xref>). Applying Jensen&#x2019;s inequality for convex functions, it follows that<disp-formula id="disp-formula11"><label>(A10)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM11"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x2265;</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow><mml:mo>.</mml:mo></mml:math></disp-formula>The log sum of the factorized marginal probability distribution of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM174"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is larger or equal to the sum of all approximate distributions <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM175"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> times the log-normalized conditional probability distribution <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM176"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>.</p>
<p>In a second step, we decompose the right-hand side of <xref ref-type="disp-formula" rid="disp-formula11">Equation A10</xref>, using <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM177"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>x</mml:mi><mml:mspace width="thinmathspace"/><mml:mo>&#x2217;</mml:mo><mml:mspace width="thinmathspace"/><mml:mi>y</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> into the expected likelihood and the Kullback-Leibler (KL) divergence between the approximate posterior and prior. The KL divergence can be interpreted as the dissimilarity of two functions. In this context, we are interested in the dissimilarity between our prior and the approximated marginal probability <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM178"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>.<disp-formula id="disp-formula12"><label>(A11)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM12"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mi>log</mml:mi></mml:mtd><mml:mtd><mml:mrow><mml:mo>(</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mi>x</mml:mi></mml:munder><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mspace width="thinmathspace" /><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>)</mml:mo></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mo>&#x2265;</mml:mo><mml:mo fence="false" stretchy="false">&#x27E8;</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:msub><mml:mo fence="false" stretchy="false">&#x27E9;</mml:mo><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:msub><mml:mo>&#x2212;</mml:mo><mml:mi>K</mml:mi><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo fence="false" stretchy="false">&#x2016;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM179"><mml:mi>K</mml:mi><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo fence="false" stretchy="false">&#x2016;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is the non-negative KL divergence (<xref ref-type="bibr" rid="B81">81</xref>, <xref ref-type="bibr" rid="B36">36</xref>) between prior <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM180"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and approximate distribution <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM181"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. It follows that a lower-bound <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM182"><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>,</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> on the marginal log-likelihood <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM183"><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is given by<disp-formula id="disp-formula13"><label>(A12)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM13"><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>,</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mo fence="false" stretchy="false">&#x27E8;</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:msub><mml:mo fence="false" stretchy="false">&#x27E9;</mml:mo><mml:mrow><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:msub><mml:mo>&#x2212;</mml:mo><mml:mi>K</mml:mi><mml:mi>L</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo fence="false" stretchy="false">&#x2016;</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2264;</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>D</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>.</mml:mo></mml:math></disp-formula>Maximising the lower bound <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM184"><mml:mi>L</mml:mi></mml:math></inline-formula> w.r.t. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM185"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is equivalent to optimising the approximation between <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM186"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM187"><mml:mi>q</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. When these conditions are met, updated model parameters will improve the approximation of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM188"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> over time.</p>
<p>We now turn to describing the parameter updates in more detail. Variational inference schemes with identities vastly facilitate parameter updates by re-parameterizing the messages that are sent within the graphical model (<xref ref-type="bibr" rid="B36">36</xref>, <xref ref-type="bibr" rid="B33">33</xref>). We specifically base parameter updates on exponential family distributions. The main reason for this choice is the mathematically convenient property of conjugate priors. A conjugate prior ensures that the prior probability distribution and the posterior probability distribution come from the same distribution family (<xref ref-type="bibr" rid="B36">36</xref>), which vastly facilitates parameter updates.</p>
<p>In the present model, we use the multinomial distribution, which in its standard form is given by (see (<xref ref-type="bibr" rid="B33">33</xref>))<disp-formula id="disp-formula14"><label>(A13)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM14"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>x</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>q</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:munderover><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:msubsup><mml:mi>q</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>k</mml:mi></mml:msub></mml:mrow></mml:msubsup><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:msub><mml:mi>q</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:msub><mml:mi>x</mml:mi><mml:mi>K</mml:mi></mml:msub></mml:mrow></mml:msup><mml:mo>.</mml:mo></mml:math></disp-formula>The multinomial distribution is a generalization of the Bernoulli distribution to <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM189"><mml:mi>K</mml:mi></mml:math></inline-formula> possible outcomes (<xref ref-type="bibr" rid="B36">36</xref>). In it, multinomial random variables <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM190"><mml:mi>x</mml:mi></mml:math></inline-formula> are represented by vectors with <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM191"><mml:mi>K</mml:mi></mml:math></inline-formula> components. Each component is either 0 or 1, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM192"><mml:msub><mml:mi>x</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula> and all components sum up to 1, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM193"><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:msub><mml:mi>x</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM194"><mml:mi>q</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>,</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mi>K</mml:mi></mml:msub></mml:math></inline-formula> are the respective probabilities that <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM195"><mml:msub><mml:mi>x</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>, i.e. we use a 1-out-of-K or one-hot representation.</p>
<p>The conjugate prior on the multinomial distribution is given by the Dirichlet distribution. The density of the Dirichlet distribution is given by Endres (<xref ref-type="bibr" rid="B33">33</xref>):<disp-formula id="disp-formula15"><label>(A14)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM15"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo>&#x2223;</mml:mo><mml:mi>&#x03B1;</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>M</mml:mi><mml:munderover><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:msubsup><mml:mi>q</mml:mi><mml:mi>k</mml:mi><mml:mrow><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msubsup><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:msub><mml:mi>q</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mrow></mml:mrow></mml:msup><mml:mrow><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mi>K</mml:mi></mml:msub><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mo>.</mml:mo></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM196"><mml:mi>M</mml:mi></mml:math></inline-formula> is<disp-formula id="disp-formula16"><label>(A15)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM16"><mml:mi>M</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi mathvariant="normal">&#x0393;</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi></mml:mrow></mml:munderover><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:munderover><mml:mo>&#x220F;</mml:mo><mml:mrow><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>K</mml:mi></mml:mrow></mml:munderover><mml:mi mathvariant="normal">&#x0393;</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mi>k</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>,</mml:mo></mml:math></disp-formula>and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM197"><mml:mi mathvariant="normal">&#x0393;</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>x</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> is the gamma function.</p>
<p>Variational inference allows for learning over time and avoids computationally costly new instantiations. For this, we need to introduce free energy nodes with natural parameters <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM198"><mml:mi>&#x03B7;</mml:mi></mml:math></inline-formula> (given by sufficient statistics <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM199"><mml:mi>&#x03BB;</mml:mi></mml:math></inline-formula> and pseudocounts <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM200"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula>) into the graphical model. Free energy factors (in the following: factors/ factor nodes) are described by conjugate prior and exponential family distributions introduced above. Parameter updates in this context lead to improved model predictions so that here, we can interpret the updating process as learning over time.</p>
<p>Further efficiency is introduced into the model by incorporating hypernodes, e.g. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM201"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM202"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> in <xref ref-type="fig" rid="F7">Figure&#x00A0;A1</xref>. Hypernodes ensure equal prior and posterior distributions of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM203"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM204"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> at each time step <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM205"><mml:mi>t</mml:mi></mml:math></inline-formula>. They are connected to the more local, time point specific factor nodes implicitly via the lower bound, so that no loops occur and the application of belief-propagation algorithms remains feasible. In this design, each factor node&#x2019;s parameters <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM206"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM207"><mml:mi>&#x03BB;</mml:mi></mml:math></inline-formula> are set to the specific parameter values of their associated hypernode. Hypernodes are not involved in sum-product message passing. Also, local factor parameters are not updated individually, but rather, the hypernodes&#x2019; parameters are updated after message-passing and then passed on to the local factor nodes. It can be shown that updates of hypernode parameters are done via:<disp-formula id="disp-formula17"><label>(A16)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM17"><mml:mrow><mml:mover><mml:mi>&#x03BD;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow><mml:mspace width="thinmathspace" /><mml:mo>&#x003A;</mml:mo><mml:mspace width="thinmathspace" /><mml:mo>=</mml:mo><mml:mi>&#x03BD;</mml:mi><mml:mo>+</mml:mo><mml:mi>r</mml:mi></mml:math></disp-formula><disp-formula id="disp-formula18"><label>(A17)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM18"><mml:mrow><mml:mover><mml:mi>&#x03BB;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow><mml:mspace width="thinmathspace" /><mml:mo>&#x003A;</mml:mo><mml:mspace width="thinmathspace" /><mml:mo>=</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>&#x03BD;</mml:mi><mml:mi>&#x03BB;</mml:mi><mml:mo>+</mml:mo><mml:mi>r</mml:mi></mml:mrow><mml:mrow><mml:mover><mml:mi>&#x03BD;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow></mml:mfrac></mml:mrow></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM208"><mml:mrow><mml:mover><mml:mi>&#x03BD;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow></mml:math></inline-formula> are the updated <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM209"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula>, or pseudocounts and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM210"><mml:mrow><mml:mover><mml:mi>&#x03BB;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow></mml:math></inline-formula> denotes the updated <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM211"><mml:mi>&#x03BB;</mml:mi></mml:math></inline-formula>, or natural parameters. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM212"><mml:mi>&#x03B2;</mml:mi></mml:math></inline-formula> is an inverse temperature parameter. In case of Bayes-optimal updates, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM213"><mml:mi>&#x03B2;</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula>. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM214"><mml:mi>&#x03BD;</mml:mi></mml:math></inline-formula> is referred to as the <italic>pseudocount</italic> as it keeps count of the number of observed data points. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM215"><mml:mrow><mml:mover><mml:mi>&#x03BB;</mml:mi><mml:mo stretchy="false">&#x007E;</mml:mo></mml:mover></mml:mrow></mml:math></inline-formula> can be interpreted as a weighted mean of the prior <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM216"><mml:mi>&#x03BB;</mml:mi></mml:math></inline-formula> and the observed data. The responsibilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM217"><mml:mi>r</mml:mi></mml:math></inline-formula>, i.e. the accumulated posterior probabilities of observations, can be computed from all messages a factor node <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM218"><mml:mi>F</mml:mi></mml:math></inline-formula> received:<disp-formula id="disp-formula19"><label>(A18)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM19"><mml:mi>r</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>m</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>M</mml:mi></mml:mrow></mml:munderover><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>v</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">&#x2192;</mml:mo><mml:mi>F</mml:mi></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>m</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>M</mml:mi></mml:mrow></mml:munderover><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>X</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>m</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>F</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>M</mml:mi></mml:mrow></mml:munderover><mml:mi>log</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>&#x03BC;</mml:mi><mml:mrow><mml:msub><mml:mi>V</mml:mi><mml:mrow><mml:msub><mml:mi>m</mml:mi><mml:mo stretchy="false">&#x2192;</mml:mo></mml:msub><mml:mi>F</mml:mi></mml:mrow></mml:msub></mml:mrow></mml:msub><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>V</mml:mi><mml:mi>m</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow></mml:math></disp-formula>Hypernodes collect responsibilities from all neighbouring factor nodes:<disp-formula id="disp-formula20"><label>(A19)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM20"><mml:msub><mml:mi>r</mml:mi><mml:mi>H</mml:mi></mml:msub><mml:mspace width="thinmathspace" /><mml:mo>&#x003A;</mml:mo><mml:mspace width="thinmathspace" /><mml:mo>=</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>n</mml:mi><mml:mi>e</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>H</mml:mi><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover><mml:msub><mml:mi>r</mml:mi><mml:mi>n</mml:mi></mml:msub></mml:math></disp-formula>This way, all observations made at the observable variable nodes in each time step are taken into account. Before message-passing can be instantiated anew, the posterior parameters of all local factor nodes are set to those of their neighbouring hypernode. The exponential family parameterization used in our simulations of healthy vs. chronic pain inference are summarized in <xref ref-type="table" rid="T1">Table A1</xref>.</p>
<table-wrap id="T1" position="float"><label>Table A1.</label>
<caption><p>Detailed overview of exponential family parameter settings for the simulation of healthy- and chronic pain inference.</p></caption>
<table frame="hsides" rules="groups">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th valign="top" align="left">Parameter</th>
<th valign="top" align="center">Healthy inference</th>
<th valign="top" align="center">Pathological inference</th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Prior on 1st node</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM219"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.222</mml:mn></mml:math></inline-formula>,</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM220"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.777</mml:mn></mml:math></inline-formula>,</td>
</tr>
<tr>
<td valign="top" align="left"/>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM221"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula></td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM222"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula></td>
</tr>
<tr>
<td valign="top" align="left">Top-down prior</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM223"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>noxious</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.8</mml:mn></mml:math></inline-formula>,</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM224"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>noxious</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.6</mml:mn></mml:math></inline-formula>,</td>
</tr>
<tr>
<td valign="top" align="left"/>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM225"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>harmless</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.9</mml:mn></mml:math></inline-formula>,</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM226"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>harmless</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.4</mml:mn></mml:math></inline-formula>,</td>
</tr>
<tr>
<td valign="top" align="left"/>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM227"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula></td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM228"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>20</mml:mn></mml:math></inline-formula></td>
</tr>
<tr>
<td valign="top" align="left">Transition prior</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM229"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.2</mml:mn></mml:math></inline-formula>,</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM230"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>,</td>
</tr>
<tr>
<td valign="top" align="left"/>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM231"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.7</mml:mn></mml:math></inline-formula>,</td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM232"><mml:mi>&#x03BB;</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mi>t</mml:mi></mml:msub><mml:mo>=</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mi>H</mml:mi><mml:mrow><mml:mi>t</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mtext>pain</mml:mtext><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>0.2</mml:mn></mml:math></inline-formula>,</td>
</tr>
<tr>
<td valign="top" align="left"/>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM233"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula></td>
<td valign="top" align="left"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM234"><mml:mi>&#x03BD;</mml:mi><mml:mo>=</mml:mo><mml:mn>100</mml:mn></mml:math></inline-formula></td>
</tr>
<tr>
<td valign="top" align="left">Observations</td>
<td valign="top" align="left" colspan="2"><inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM235"><mml:msub><mml:mi>S</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mn>5</mml:mn></mml:msub></mml:math></inline-formula>= noxious input, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM236"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mn>12</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mn>18</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> = harmless input for both cases</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<fn id="table-fn1"><p>Further, the first observation scheme to produce <xref ref-type="fig" rid="F2">Figure&#x00A0;2</xref> is shown below. Probabilities and their respective counter-probabilities sum up to 1 (<xref ref-type="bibr" rid="B83">83</xref>).</p></fn>
</table-wrap-foot>
</table-wrap></app>
<app id="app2"><title>Appendix B. Derivation of null space and stable pain states</title>
<p>Chronic pain is marked by very tenacious pain perception in the absence of acute tissue damage. This translates to stable pain states in our model. To achieve this stability of hidden states, we need to derive transition probabilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM237"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM238"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> such that the inferred probability of pain in some next time step <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM239"><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula> is equal to the probability of pain in the current time step <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM240"><mml:mi>t</mml:mi></mml:math></inline-formula>, which in turn is equal to some stable marginal probability of pain, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM241"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>:<disp-formula id="disp-formula21"><label>(B1)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM21"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></disp-formula>The probability of pain in a future time step <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM242"><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:math></inline-formula> is given by the sum over the possible previous states <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM243"><mml:mo>&#x2208;</mml:mo><mml:mo fence="false" stretchy="false">{</mml:mo><mml:mtext>pain</mml:mtext><mml:mo>,</mml:mo><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo fence="false" stretchy="false">}</mml:mo></mml:math></inline-formula> and the transition probabilities:<disp-formula id="disp-formula22"><label>(B2)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM22"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mspace width="thickmathspace" /><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mo>+</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>Thus, for a stable pain percept after a long time, we require:<disp-formula id="disp-formula23"><label>(B3)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM23"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mspace width="thickmathspace" /><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mo>+</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>Conversely, the condition for a stable no-pain percept is:<disp-formula id="disp-formula24"><label>(B4)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM24"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mspace width="thickmathspace" /><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mo>+</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>Solving <xref ref-type="disp-formula" rid="disp-formula23">equation B3</xref> for the transition probabilities, we find<disp-formula id="disp-formula25"><label>(B5)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM25"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mspace width="1em" /><mml:mo>=</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn><mml:mo>=</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>Similarly, for <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM244"><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> (<xref ref-type="disp-formula" rid="disp-formula24">Equation B4</xref>):<disp-formula id="disp-formula26"><label>(B6)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM26"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd /><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mspace width="negativethinmathspace" /><mml:mo>&#x22C5;</mml:mo><mml:mspace width="negativethinmathspace" /><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x22C5;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mspace width="1em" /><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula>In matrix-vector form, the last two equations can be written as<disp-formula id="disp-formula27"><label>(B7)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM27"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>To solve for the transition probabilities, we decompose the transition probability vector into the sum of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM245"><mml:mi>q</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM246"><mml:mi>r</mml:mi></mml:math></inline-formula>:<disp-formula id="disp-formula28"><label>(B8)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM28"><mml:mo stretchy="false">(</mml:mo><mml:mi>q</mml:mi><mml:mo>+</mml:mo><mml:mi>r</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>where <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM247"><mml:mi>q</mml:mi></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM248"><mml:mi>r</mml:mi></mml:math></inline-formula> are constrained such that<disp-formula id="disp-formula29"><label>(B9)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM29"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x22C5;</mml:mo><mml:mi>q</mml:mi><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>in other words, <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM249"><mml:mi>q</mml:mi></mml:math></inline-formula> is that summand of the transition probability decomposition which produces the desired stable marginal probability of pain.</p>
<p>In contrast, for <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM250"><mml:mi>r</mml:mi></mml:math></inline-formula> we require that<disp-formula id="disp-formula30"><label>(B10)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM30"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x22C5;</mml:mo><mml:mi>r</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:math></disp-formula>i.e. <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM251"><mml:mi>r</mml:mi></mml:math></inline-formula> is a vector in the null space of the matrix in <xref ref-type="disp-formula" rid="disp-formula27">Equation B7</xref>.</p>
<p>Solving for the components of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM252"><mml:mi>q</mml:mi><mml:mo>=</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:msup><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mi>T</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM253"><mml:mi>r</mml:mi><mml:mo>=</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>,</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msub><mml:msup><mml:mo stretchy="false">)</mml:mo><mml:mrow><mml:mi>T</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> yields<disp-formula id="disp-formula31"><label>(B11)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM31"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mi>p</mml:mi><mml:mi>a</mml:mi><mml:mi>i</mml:mi><mml:msub><mml:mi>n</mml:mi><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>q</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>.</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>i.e.<disp-formula id="disp-formula32"><label>(B12)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM32"><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>=</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn><mml:mo>=</mml:mo><mml:mo>&#x2212;</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math></disp-formula>Similarly, for <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM254"><mml:mi>r</mml:mi></mml:math></inline-formula>,<disp-formula id="disp-formula33"><label>(B13)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM33"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd><mml:mtd><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:msub><mml:mi>r</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:msub><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>0</mml:mn></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>we find<disp-formula id="disp-formula34"><label>(B14)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM34"><mml:mrow><mml:mfrac><mml:mrow><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:msub><mml:mi>r</mml:mi><mml:mn>1</mml:mn></mml:msub><mml:mo>=</mml:mo><mml:msub><mml:mi>r</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math></disp-formula>Thus, the transition probabilities for a given <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM255"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> can be written as<disp-formula id="disp-formula35"><label>(B15)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM35"><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mi>&#x03B2;</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mtable rowspacing="4pt" columnspacing="1em"><mml:mtr><mml:mtd><mml:mn>1</mml:mn></mml:mtd></mml:mtr><mml:mtr><mml:mtd><mml:mrow><mml:mfrac><mml:mrow><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow></mml:mtd></mml:mtr></mml:mtable><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula>for any <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM256"><mml:mi>&#x03B2;</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mrow><mml:mrow><mml:mi mathvariant="double-struck">R</mml:mi></mml:mrow></mml:mrow></mml:math></inline-formula> such that <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM257"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>,</mml:mo></mml:math></inline-formula> <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM258"><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2208;</mml:mo><mml:mo stretchy="false">[</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy="false">]</mml:mo></mml:math></inline-formula>. This implies a one-dimensional null space, where pain percepts remain stable even if the transition probabilities change within this null space (further illustrated in <xref ref-type="fig" rid="F8">Figure B1</xref>). Alternatively, we can eliminate <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM259"><mml:mi>&#x03B2;</mml:mi></mml:math></inline-formula> by solving the first component equation for <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM260"><mml:mi>&#x03B2;</mml:mi></mml:math></inline-formula> and substituting the result into the second component equation:<disp-formula id="disp-formula36"><label>(B16)</label><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="DM36"><mml:mtable columnalign="right left" rowspacing=".5em" columnspacing="thickmathspace" displaystyle="true"><mml:mtr><mml:mtd><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mover><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo>&#x2223;</mml:mo><mml:mover><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mo stretchy="false">)</mml:mo><mml:mo>=</mml:mo></mml:mtd><mml:mtd><mml:mrow><mml:mfrac><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mn>2</mml:mn><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow></mml:mtd></mml:mtr><mml:mtr><mml:mtd /><mml:mtd><mml:mo>+</mml:mo><mml:mrow><mml:mfrac><mml:mrow><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow><mml:mrow><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mrow></mml:mfrac></mml:mrow><mml:mo>&#x22C5;</mml:mo><mml:mi>P</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:mtd></mml:mtr></mml:mtable></mml:math></disp-formula></p>
<fig id="F8" position="float"><label>Figure B1</label>
<caption><p>Illustration of null space of transition probability deviations for different marginal <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM261"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. We show that when the transition probabilities <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM262"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> and <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM263"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mrow><mml:mi>t</mml:mi><mml:mo>+</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2223;</mml:mo><mml:msub><mml:mover><mml:mtext>pain</mml:mtext><mml:mo accent="false">&#x00AF;</mml:mo></mml:mover><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> are sampled from the one-dimensional null space of a given marginal probability of pain <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM264"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>, the marginal probability <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM265"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi>t</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula> will approach and stabilize within the range of <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM266"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo></mml:math></inline-formula>. Here, we illustrate the transition probabilities for <inline-formula><mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="IM267"><mml:mi>p</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:msub><mml:mtext>pain</mml:mtext><mml:mi mathvariant="normal">&#x221E;</mml:mi></mml:msub><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2208;</mml:mo><mml:mo stretchy="false">[</mml:mo><mml:mn>0.1</mml:mn><mml:mo>,</mml:mo><mml:mn>0.9</mml:mn><mml:mo stretchy="false">]</mml:mo></mml:math></inline-formula> (legend).</p></caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="fpain-03-966034-g008.tif"/>
</fig></app>
</app-group>
</back>
</article>