Traditionally, statistical goals can be divided into two categories: explanation-oriented models, in which researchers are interested in testing an a priori hypothesized relationship between two or more variables, and prediction-oriented models, in which researchers are interested in finding an algorithm capable of recognizing which set of variables yields the best predictions about new observations (1, 2). For many years, psychotherapy research focused mainly on models aimed at understanding and explaining associations between variables (3). Many hundreds of studies testing predictors, moderators, and mediators have contributed greatly to our understanding of the process and outcome of psychotherapy (2). For instance, research has shown that the therapeutic alliance is strongly related to improved treatment outcomes, with a more pronounced effect observed in clients dealing with interpersonal problems (4).

Contemporary psychotherapy research is interested not only in driving understanding, but also in making accurate predictions about who may be a good therapist (to direct therapist selection and training efforts), and about treatment prognosis (to select the most efficient treatment for individual clients). Despite the statistical and mathematical similarities, the choice between explanatory models and predictive models can have a significant impact on researchers’ decisions regarding data collection, data preparation, and the statistical models they employ (1). For example, while explanatory models tend to emphasize goodness of fit based on a specific sample and the testing of statistical significance, predictive models assess how accurately the model can apply information about one sample to make correct predictions or decisions when applied to a new dataset (5). Researchers, therefore, must make a deliberate choice to identify the best models for achieving their aims (1, 6).

When following a hypothesis-driven approach, researchers rarely use tools to verify that the models they propose are capable of predicting the outcomes they are modeling (7). Instead, researchers focus largely on the statistical significance of various factors that may be related to the outcome (3). The statistical significance approach does not guarantee predictive accuracy when models are applied to new (i.e., out-of-sample) data. Furthermore, in the past years, there is an increasing concern that many findings in the field cannot be reliably reproduced in subsequent studies, raising questions about the credibility and robustness of psychological research findings (i.e., the replication crisis) (8–11). There is increasing consensus that this widespread replication failure is due largely to “p-hacking” and other questionable research practices that have been historically prevalent in the field (12). At the same time, the increased capacity to collect and analyze massive amounts of data, gathered through new technologies or the internet, has contributed to the adoption of a computational approach to analysis (13). While a hypothesis-driven approach is rooted in, and therefore constrained by, substantive theory, a computational approach prioritizes prediction, thus allowing model complexity to increase as long as it continues to enhance predictive performance (7). Inspired by the computational approach, researchers started to use predictability models by applying data-driven approaches and principles from the machine learning field to increase the predictive power of the models (7). Machine learning is a subset of artificial intelligence that focuses on developing algorithms and models capable of learning from data and making predictions or decisions without being explicitly programmed. It involves the use of statistical techniques to enable computers to recognize patterns, draw insights, and improve their performance based on experience (5). In this regard, the objectives of machine learning closely parallel those of psychotherapists. Both aim to accumulate knowledge from prior data, such as client histories, and apply that knowledge to new cases, even those that may be unique (14). Machine learning encompasses various approaches, including supervised learning (where models are trained on labeled data), unsupervised learning (for discovering patterns within data), and reinforcement learning (teaching machines to make sequential decisions). These algorithms have applications across a wide range of fields including healthcare, revolutionizing the way we analyze data and solve complex problems (15).

Some promising evidence for the potential utility of implementations of data-driven and machine learning approaches in psychotherapy research has emerged in recent years, answering questions of treatment personalization (16–18) while providing critical solutions to p-hacking (19) and replicability issues (10). Moreover, data-driven approaches that use principles from the machine learning field may be better able to capture the richness and complexity of the therapeutic encounter than explanatory models. No single, independent predictor, or even three or four, can be as informative as a set of interrelated variables that jointly increase predictive power to explain variance in the process and outcome of treatment (16). Some of the explanatory approaches, which test each predictor as a separate hypothesis, can lead to erroneous conclusions because of multiple comparisons (inflated type I errors), model misspecification, and multicollinearity (20). Findings may also be affected by publication bias, as statistically significant predictors have a better chance of being reported in the literature (21). Machine learning approaches provide a robust solution to these problems by looking at data insights, developing a predictive model, and finally, applying self-validation procedures (22, 23). However, a significant drawback of some (but not all) machine learning approaches is their tendency to treat models mostly as a “black box,” resulting in a lack of interpretability of the findings (3, 7).

We sought to integrate the benefits of the interpretability and predictability of statistical models. As such, the present study was designed as a proof of concept of the ability of models to predict variability in treatment process and outcome, based on pre-treatment client and therapist characteristics. Our study focused on changes in psychological dysfunction from pre- to post-treatment as the treatment outcome. We chose the therapeutic alliance as an example of a process variable because it is one of the most researched constructs in psychotherapy research, and one of the most consistent predictors of treatment outcome (24). Two important aspects of the alliance are of interest when predicting clients’ and therapists’ reported alliance: (a) the trait-like tendency of the client and therapist to form a strong alliance across various phases of treatment (also known as the between-clients alliance), and (b) the state-like changes occurring during treatment in client and therapist reports of the alliance (the within-client alliance). The sample was chosen to avoid restriction of range, a substantial risk when testing therapist characteristics because in many trials only a narrow range of therapist characteristics is allowed. As the therapist effect may have been skewed in previous studies owing to restriction of range, to increase variance, we selected therapists prospectively, based on their level of social skills and performance on a simulated task. For the same reason, we also selected both trained and untrained therapists. Using prediction models in a sample that is less restricted in range, we expect the baseline characteristics of clients and therapists to predict a large portion of treatment outcome and within- and between-clients alliance.

For client characteristics that may affect the alliance and the outcome of treatment, we focused on those that have received extensive attention in the literature. Specifically, we focused on clients’ demographics (gender, age, and income), symptom severity, and functioning. We also focused on clients’ interpersonal skills and problems [for a review, see (25, 26)]. Whereas previous studies showed little evidence that demographic variables significantly predict psychotherapy outcome or alliance, more complex results were found concerning symptom and interpersonal problem severity (25). Severity of symptoms and functional impairment predicted worse outcome and lower trait-like alliance [e.g., (27–30)]. At the same time, severity of symptoms also predicted greater improvement of both symptoms and state-like alliance [e.g., (31–33)]. The same pattern of results was found for interpersonal skills and problems [e.g., (34–36)].

With regard to therapists, in addition to including therapist training and clinical orientation as potential predictors of treatment process and outcome, we focused on therapist characteristics that are typically easily available in clinical practice, including demographics (age, gender, and income), and on those receiving theoretical and empirical support, such as previously being in treatment themselves (37), and Facilitative Interpersonal Skills [FIS; (38)]. Previous studies examining the association between such therapists’ characteristics and alliance or outcome have yielded mixed results. Specifically, whereas previous studies mostly found that training and experience were not related to outcome and alliance [e.g., (39–41)], other studies found them to be related to poorer outcome and lower alliance (42–44). Manne et al. (45) reported that therapists’ experience negatively predicted mean alliance when alliance was rated by the client, but positively predicted alliance when alliance was rated by the therapist. These findings may suggest that a complex predictive model is needed to better explain the mixed results.

In the same vein, most studies examining the relationship between the therapist’s demographic variables and alliance or treatment outcome have found that these variables are not significantly related to therapy outcome or alliance [Outcome: (37, 46, 47); Alliance: (42, 45)]. At the same time, other studies suggested that therapists being older or a woman may be related to outcome and alliance, presumably because of their relation to superior interpersonal and social skills (48). For instance, results of a large naturalistic study suggested that gender may have an indirect role on alliance and outcome such that female therapists intervene more empathically, whereas male therapists tend to use more confrontational techniques (49). Similarly, Anderson et al. (38) found that older therapists produced superior outcomes. However, when therapists’ social skills and FIS were examined, age no longer predicted outcome. These results may point to the importance of including both demographic characteristics and interpersonal and social skills in the predictive model for better prediction of alliance and outcome. One way to measure therapists’ interpersonal and social skills is using the FIS. FIS includes common relational skills, such as the ability to respond empathetically to clients, express the appropriate amount and type of emotion when responding to clients, and efficiently respond to ruptures that may arise in the alliance with clients. FIS is measured using a performance task in which therapists record audio responses to a set of stimulus clips simulating various difficult interpersonal situations derived from real therapy sessions. Each therapist’s responses are then rated by trained observers for their coverage of the eight FIS items: verbal fluency, emotional expression, persuasiveness, warmth/acceptance/understanding, hope/positive expectations, empathy, alliance bond capacity, and alliance-rupture-repair responsiveness (38, 50). Therapist FIS has been found to predict treatment outcome across settings (38, 51, 52). Anderson et al. (28) found that therapist FIS predicted outcomes at a university counseling center. In a randomized clinical trial, Anderson et al. (51) found that helpers with significantly higher FIS showed better outcomes and developed stronger client-related alliances when counseling undergraduate students with heightened levels of clinical distress. In a naturalistic study, Anderson et al. (42) found that therapist FIS predicted the outcomes of clients treated by graduate students in clinical psychology over 1 year later.

In addition to FIS’s prediction of outcome in several studies, multiple metanalyses have revealed several other predictors of outcome [e.g., (2)]. For example, a previous review by Beutler (53) identified multiple therapist characteristics, such as their training and experience, which give promise to predicting outcome. Nevertheless, at that point, these characteristics were impossible to consider jointly. It is reasonable to include these client and therapist characteristics in a model which allows them to combine with FIS. Furthermore, the notion of therapist responsiveness (54) recognizes that there are a very large number of possible variables within the “emergent context” (55) that might be accounted for to explain what works in any particular therapeutic situation.

Taken together, in the present study, we aim to benefit from the advantages of the two potentially complementary approaches, the interpretability and predictability approaches. We will do so by applying a data-driven approach and principles from the multilevel machine learning framework. Our approach allows: (a) identifying the models with the best fit to the data (given a set of potential predictors), considering the linear and quadratic shapes of the relationships as well as pairwise interactions, and taking into account their nested data structure; (b) providing an exact interpretation of all model effects; (c) cross-validating the proposed model; and (d) estimating the contribution of each model effect, as defined below.

Results of the current study showed that a large portion of variance, both in alliance and outcome, can be explained based only on the pre-treatment characteristics of clients and therapists. Baseline characteristics (clients’ psychological dysfunction, interpersonal dysfunction, and social skills, and therapists’ general treatment orientation) were able to predict 52.8% of the variance in outcome and 24.1–52.8% of the variance in alliance for clients whose data were not used for building the models on which the predictions were based, to avoid overfitting. Thus, the present study demonstrates the potential utility of integrating explanatory and predictive models for psychotherapy science. Such integration may allow for deriving theoretical insights through selecting the variables that have the most relevance in terms of explanation (based on the theoretical framework) and including them in a model of prediction of treatment processes and outcomes. Whereas the most prominent explanatory studies in psychotherapy today can explain about 5–7% variance (71), the present study increased this range several fold.

Results of the current study showed that changes in symptoms from pre- to post-treatment were predicted mostly by clients’ baseline symptoms as well as psychological and interpersonal functioning. Previous studies using machine learning to predict treatment response found mixed results. For example, whereas Yin et al. (72) reported that greater baseline symptoms severity were among the most important predictors of treatment response, Ziobrowski et al. (73) did not find baseline symptom severity to be among the important predictors. Future studies should address this discrepancy in the literature. If replicated, the finding that baseline symptoms predict treatment response may suggest that in tailoring treatment to the individual client, considerable attention should be given to the clients’ symptomatic complaints (25, 26). The process of tailoring treatment to the individual client can be further enhanced using network analysis to recognize the key symptoms and their dynamics when identifying treatment targets (74, 75).

Concerning alliance prediction models, using both within- and between-clients prediction enabled us to recognize predictors of both state-like and trait-like components of the alliance. At the trait-like component, we found that therapist characteristics were better predictors of trait-like alliance than were client characteristics. Specifically, we found that therapist age moderated the effect of training on alliance, with training being associated with better alliance only for young therapists. In addition, we found that therapists’ gender moderated the association between therapists’ income and alliance, with higher income being associated with poorer alliance only in male therapists. Gender also moderated the association between FIS and alliance, with poorer FIS being associated with poorer alliance only in male therapists. Considering this proof-of-concept study, we will not individually elaborate on each outcome. However, in a broader sense, the observed interaction effects potentially indicate that cultural factors might influence how therapists’ characteristics impact the therapeutic alliance. For instance, income could affect males differently, implying that financial status might correlate with the therapeutic relationship in gender-specific ways. Moreover, interpersonal skills, often more culturally encouraged in women who tend to exhibit greater supportive communication, potentially leading to a reduction in the variance explained by FIS among women (76). In our sample, 66% of the females had high FIS scores, whereas only 37.5% of the men had high FIS scores. This is in accordance with a recent study indicating that female students demonstrate superior interpersonal skills compared to males in the context of medical consultations (77). Importantly, the centrality of therapists’ characteristics in predicting alliance is consistent with a recent meta-analysis by Del Re et al. (78), who found support for therapist effects on alliance, indicating that some therapists are better at forming strong alliances with their clients than others. Still, previous studies focused mainly on a given therapist’s characteristics at a certain point in time. For example, one study found that therapist age was positively associated with trait-like alliance but found no effect of gender or therapist years of experience on alliance (42). Using the proposed framework, we were able to identify a more nuanced understanding that has the potential to elucidate previously inconsistent findings in psychotherapy research.

The current findings, together with those reported in the literature (79–82), highlight the importance of therapist characteristics in determining alliance and treatment outcome. Such characteristics, when supported by theory and clinical observations, can be used to identify candidates for clinical training who are expected to form strong alliances with their clients. Furthermore, characteristics that are amenable to change and will be found to be causally related to outcome should be the focus of empirically guided training programs for clinicians (83).

The findings suggest that whereas trait-like alliance was predicted mostly by therapist characteristics, state-like changes in alliance over the course of treatment were predicted by both client and therapist characteristics, and by the interaction between them. Here again, although previous studies reported no effect of client age on changes in alliance (28, 84), the findings of the current study suggest that client age may have a more complex effect on alliance. Specifically, older clients were found to benefit more from therapist FIS and were more negatively affected by their own interpersonal skills. Such nuanced understanding may facilitate progress toward personalization in both client treatment and therapist training.

Taken together, the explainable nature (a glass box vs. a black box) of the proposed framework of integrating explanation and prediction in psychotherapy may be instrumental in reaching a more nuanced understanding of the richness of clinical practice, where each predictor is not isolated from the others, but rather, interacts with them in complex ways in predicting the process and treatment outcome. Given the small sample size, the findings should be interpreted with caution, and serve mainly as a proof of concept demonstrating the potential utility of integrating explanation and prediction in computational psychotherapy science. Still, the results of these predictive models have the potential to assist in drawing a map of the factors contributing to the psychotherapy process and outcome and the complex interconnections between them. If replicated in future studies with large samples, the current findings offer instructive insights that expand the available literature on the predictors of psychotherapy process and outcome.

The most important limitation of the present study is the flip side of one of its main merits: its unique sample. Because of ethical considerations, maximizing variance in therapists results in minimizing variance in clients. To avoid restriction of range in therapist characteristics, we recruited a diverse set of treatment providers, who then provided counseling to a sample of individuals who had not actively sought treatment. For the same reasons, the effect of baseline characteristics was quite likely inflated. In the case of therapists who are more skilled in repairing alliance ruptures and who received appropriate alliance-focused training, it may be possible to rise above the deterministic view that the strengthening of alliance can be predicted based on the predispositions of the clients and bring about a real therapeutic change in the clients’ pre-treatment potential to form a strong alliance. In addition, since therapists represent both ends of highly skilled and unskilled communicators, this sample might have inflated effect sizes. Importantly, given the unique characteristics of the sample, it is questionable whether the results can be consistently generalized to psychotherapy settings. Another important limitation is that the small sample size restricted our ability to calculate therapist effects. Given the unique characteristics of the sample that were required, specifically, the richness of the variance and availability of baseline predictors (including therapist performance, as coded before the treatment, based on a standard evaluative task), we were limited by the available sample size. Future work on larger samples is critical. Thus, the current findings should be regarded as a proof of concept of our suggested framework of integrating explanation and prediction, rather than serving to inform clinical practice. In this work, we used a quasi-R² estimate of Nakagawa and Shielzeth (67) because of the relatively simple data architecture and our interest in just the marginal explained variability. The approach suggested by Rights and Sterba (85) provides estimates with a better variance composition, taking into account various scenarios of the data and model structures. One should consider using these estimates.

Although the current proof of concept focused on client and therapist pretreatment characteristics, the proposed framework for integrating explanation and prediction in psychotherapy science can also be implemented on in-treatment data. Personalized treatment does not end with pretreatment clinical decision-making. Rather, ongoing tailoring is needed as well (86, 87). The literature on therapist responsiveness highlights the importance of ongoing tailoring of the treatment to the client. Effective therapists are responsive to client behaviors within the emerging context of the treatment (55). For example, observations of the many changing characteristics and behaviors of the client may prompt a therapist to use different interventions in the course of therapy (88). Likewise, the response of the client to a particular strategy may prompt the therapist’s next move, whether to stay the course or try something else—e.g., the therapist should not push an interpretation if the client responds to it defensively (55, 89). Therapeutic interactions exist in a constant loop of feedback and mutual influence (89). Using the proposed framework, ongoing data collection can be fed into the proposed models to support in-treatment decision-making processes (86). While the current proof of concept focused on pretreatment client and therapist characteristics, it is essential to note that the available data limited the construction of individualized models for each participant due to most variables having only one assessment per participant. Nevertheless, embracing idiographic approaches holds promise in potentially enhancing model predictability and optimizing treatment outcomes. Future studies adopting idiographic approaches are recommended to explore and leverage the benefits of constructing personalized models.

Explanatory models have contributed greatly to developments in the field of psychotherapy research, and today we know much more than we did 50 years ago about what drives therapeutic change and about the factors influencing it. This study demonstrates the great potential of the proposed approach to produce robust predictions of the process and outcome of treatment, offering a potential solution to the p-hacking and replicability problem. Such an approach is needed to answer questions in which future predictions are important, such as therapist selection, client prognosis to benefit from treatment, and so on. The present findings show how measures that have been used before in psychotherapy research can predict a large portion of the process (in this case, alliance) and outcome of psychotherapy, shedding new light on previously inconsistent findings.

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

The studies involving humans were approved by Institutional Review Board at Ohio University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

HF: Conceptualization, Writing – original draft, Writing – review & editing. SS: Conceptualization, Writing – original draft, Writing – review & editing. SZ-M: Conceptualization, Writing – original draft, Writing – review & editing. PG: Formal analysis, Conceptualization, Writing – review & editing. TA: Conceptualization, Resources, Supervision, Writing – review & editing.