Abstract
Precarious manhood theory (PMT) explains men's compensatory responses to masculinity threats in domains such as affect, aggression, risk-taking, dominance, and social attitudes. While recent meta-analytic evidence suggests that masculinity threat has a small-to-medium average effect, recent preregistered replication work has failed to reproduce some influential masculinity-threat effects in a large representative sample, raising questions about which parts of the published record provide robust evidence, and which may reflect selection for significance. We conducted a field-level credibility analysis of the experimental PMT literature using z-curve, a selection model that estimates discovery and replication indicators from the distribution of reported test statistics. Across 188 independent samples contributing 452 hypothesis tests, clustered z-curve estimates indicated substantial selection for significance at the conventional 5% threshold and moderate expected replicability. Subgroup analyses suggested more favorable credibility indicators for main effects than interaction tests, for preregistered compared with non-preregistered studies, and for post-2015 work, although evidence of selective reporting remained. Affective, behavioral, and cognitive outcomes showed broadly similar selection patterns, with behavioral outcomes having the weakest evidential profile. Our analysis of the credibility of the published significant-results record suggests that interpreting past findings at a more stringent threshold of α = 0.01 reduces the estimated false-discovery risk below 5%, although such retrospective threshold analyses should not be read as a proposal that changing α alone would control false discoveries prospectively. Future PMT research will be most informative when theoretically specified threat–outcome hypotheses are tested in preregistered studies, supported by explicit design justification, transparent reporting, and direct or multilab replications.
Preregistration:
https://osf.io/pcr2s/overview.
Materials, data, and R code:
Introduction
Ideas about what it means to be a “real man” (or a “real woman”) are widespread, yet contemporary scholarship emphasizes that masculinities are not a single, fixed trait. Rather, masculinities are plural, culturally and historically contingent, and analyzable across levels ranging from self-ascribed identities to macro-level social structures ().
Within this broader landscape, precarious manhood theory (PMT; ; ) has become a particularly influential account of situational masculinity dynamics. It proposes that, in many cultural contexts, manhood is a hard-won, socially conferred status that must be continually earned and can be lost when one's masculinity is questioned (), such that masculinity threats elicit anxiety and motivate compensatory responses aimed at restoring status in the eyes of relevant audiences. This premise has guided a large experimental literature on masculinity threat and men's downstream judgments, attitudes, and behavior.
The empirical record is now large enough to support quantitative synthesis, but it is also heterogeneous enough to warrant careful interpretation. A recent meta-analysis of masculinity-threat experiments in heterosexual men found an overall small-to-medium effect, d = 0.45, 95% CI [0.38, 0.51], while also showing that effects vary across components of the threat process, including trigger type, consequence type, publicity, target of compensatory reaction, and outcome order (). In that synthesis, internalized responses such as negative emotions and reduced self-perceived masculinity were stronger than externalized compensatory reactions, and self-induced prototypicality threats—situations in which men themselves infer that they do not meet masculine ideals—were especially consequential. These findings suggest that masculinity threat is not a single uniform manipulation effect, but a family of related experimental procedures and outcomes whose effects may differ in size, proximity to the theorized threat process, and likely replicability.
At the same time, recent replication evidence shows why effect-size synthesis alone cannot settle questions about credibility of the experimental PMT literature. ) conducted a preregistered replication and extension of ) highly cited masculine-overcompensation experiments using a large, nationally representative probability-based U.S. sample. Whereas, the original studies reported that masculinity threat increased men's support for war, homophobic attitudes, and dominance-related political attitudes, the replication found no consistent evidence that masculinity threat altered political attitudes. This failure need not imply that PMT is false as a theory. It may instead reflect a false-positive original result, changes in historical context, weaker threat salience in an online survey, differences between student and population-based samples, or theoretically important boundary conditions. Such ambiguity motivates a field-level credibility analysis: the same literature may contain genuine effects, boundary-conditioned effects, inflated estimates, and false-positive findings.
More generally, the last decade has sharpened concern about the credibility of published psychological findings (; ). Large-scale replication efforts have reported substantially lower reproducibility than suggested by the original literature, motivating renewed scrutiny of how evidential claims accumulate in psychology (). More broadly, methodological work has highlighted that low power, undisclosed researcher analytic flexibility, and selection for statistical significance can lead to inflated estimates and an excess of apparently “successful” findings in the published record (see ).
These concerns are especially relevant for heterogeneous experimental traditions such as PMT. The issue is not merely that studies use different manipulations and outcomes. Some of these differences are substantively meaningful: a threat to physical toughness may be more proximal to body image, pain tolerance, aggression, or strength display, whereas a sexuality-related threat may be more proximal to heterosexual self-presentation, anti-gay behavior, or discomfort with gay men. Similarly, affective responses such as anxiety, anger, or public discomfort may be temporally and theoretically closer to the threat episode than more distal behavioral or political outcomes. Although PMT specifies a general threat-and-compensation process, it often leaves open which specific threat inductions should affect which outcomes, under which audience conditions, and for which men. Thus, heterogeneity in the PMT literature is both substantively meaningful and theoretically diagnostic: it reflects researchers' attempts to apply a broad theory across diverse operationalizations while also revealing where the theory requires greater specification.
Accordingly, cumulative synthesis in PMT benefits from separating two questions. First, how large are masculinity-threat effects, on average and across moderators? Second, how credible is the published significant-results record from which those effect-size estimates are derived? Conventional meta-analyses primarily address the first question by pooling effect sizes and examining moderators (). The recent meta-analysis by ) therefore contributes by estimating average masculinity-threat effects and identifying theoretically relevant moderators such as trigger type, consequence type, publicity, and outcome order.
The second question is less straightforward. Although meta-analyses can in principle evaluate publication bias and selective reporting, audits of more than 1,000 meta-analyses suggest that publication-bias assessment and correction are applied unevenly, heterogeneity and dependence among effect sizes are common, and conclusions often rely on diagnostics with limited sensitivity or difficult interpretation (). Contemporary syntheses increasingly combine tools such as PET-PEESE, trim-and-fill, and selection models, but simulation studies indicate that no single correction performs optimally across all data-generating conditions (; ). In highly heterogeneous literatures, some approaches may under-correct, over-correct, or become difficult to interpret (; ; ).
These challenges are particularly relevant for PMT, where studies differ substantially in threat inductions, outcomes, sample sizes, and analytic specifications. ) assessed small-study effects and applied trim-and-fill procedures, but these analyses were necessarily constrained by the structure of the available evidence and cannot directly estimate field-level credibility indicators. The present study therefore addresses a complementary question. Using z-curve (; ; ), we estimate the observed discovery rate (ODR), expected discovery rate (EDR), expected replication rate (ERR), and false-discovery risk (FDR), which are defined in greater detail in the z-curve overview below. Rather than estimating average effect sizes, we estimate field-level credibility indicators of the published significant-results record.
Over the past two decades, researchers have developed a diverse set of experimental procedures to induce masculinity threat. A recent systematic review documented substantial methodological diversity in how threats are operationalized (e.g., target of threat, content, delivery, and context), underscoring the need for synthesis approaches that can characterize the evidential profile of this literature (). Building on this descriptive evidence, the present analysis asks whether field-level credibility indicators vary across recurring design features of PMT experiments.
To our knowledge, this is the first z-curve analysis of the experimental PMT literature. Replication failures (e.g., ) highlight why evaluating the credibility of the broader record is important, but z-curve cannot determine why any individual finding replicated or failed to replicate. Instead, it estimates whether the published literature as a whole has the statistical profile expected of a record containing many readily replicable findings or one shaped more strongly by selection for statistical significance.
A brief overview of the experimental literature on precarious manhood
According to PMT, threats to manhood are not merely technical manipulations but socially meaningful events in which men perceive their masculine status as questioned or at risk. Men who experience such threats are expected to feel negative emotions—most notably anxiety, anger, and shame—which in turn motivate compensatory statements and behaviors aimed at restoring manhood in the eyes of relevant audiences (; ; ; ). These compensatory responses are theorized to both emphasize masculinity (e.g., toughness, dominance, risk-taking) and signal distance from femininity. Consistent with this, the empirical literature spans a broad range of outcomes, including negative attitudes toward women and sexual and gender minorities; cognitions, attitudes, and behaviors related to dominance and aggression; and engagement in risky behaviors across life domains such as health, finances, and politics (; ).
Empirically, masculinity threats have been induced through a diverse set of procedures. ) identify several broad families, including feminized activities or products, imagined “unmanly” scenarios, feedback suggesting inadequate masculinity, performance failures on masculinity-typed tasks, and messages highlighting threats to men's broader social status. These threat manipulations also vary in theoretically relevant ways, including whether the threat targets the individual or men as a group, whether it is local or global, whether it implies insufficient masculinity or excessive femininity, and whether the threat is private or publicly observable.
This diversity creates both theoretical opportunities and methodological challenges. PMT has been applied across a wide range of manipulations and outcomes that differ in how directly they test the theory's proposed threat-and-compensation process. For example, an effect on immediate anxiety following masculinity-threatening feedback is conceptually closer to the theorized threat response than an effect on downstream political attitudes, which depends on additional assumptions linking threatened manhood to political judgment. Similarly, some manipulations target masculinity directly at the individual level, whereas others focus on broader concerns about men's social status. As a result, not all PMT experiments should necessarily be expected to yield effects of similar magnitude, consistency, or replicability.
At the same time, many studies report findings consistent with PMT. Across diverse contexts, masculinity threats have been associated with greater endorsement of traditional masculine norms, distancing from femininity, increased aggression, risk-taking, and gender-traditional attitudes (e.g., ; ; ; ). These findings have contributed to the use of PMT as a framework for understanding men's responses to social change, gender equality initiatives, and public-health messaging (for an overview, see ).
For the present study, this diversity is important for two reasons. First, it means that the PMT literature contains substantial variation in manipulations, outcomes, and study designs, making it a suitable candidate for a credibility analysis that does not assume a single common effect size. Second, it motivates our subgroup analyses, which examine whether field-level credibility indicators differ across recurring classes of manipulations, outcomes, and study characteristics. The goal is not to determine whether PMT is “true” or “false,” but to characterize the credibility profile of the published experimental record.
Z-curve analysis
Z-curve is a selection-model–based approach for characterizing the evidentiary value of a published record under the premise that statistically significant results have a higher probability of entering and remaining in the visible literature than statistically non-significant results (; ; ). The method does not assume any particular magnitude of this difference. Rather, it assumes only that statistical significance influences selection into the published record, while allowing the strength of that selection to remain unspecified.
Importantly, z-curve does not estimate average effect sizes or researchers' planned statistical power. Instead, it estimates credibility-related properties of a literature based on the observed distribution of test statistics. Z-curve produces several credibility indicators (; ; ). Observed Discovery Rate (ODR) is the observed proportion of extracted tests that are statistically significant at a chosen significance threshold, usually α = 0.05. Because published literatures often contain a disproportionately high share of significant results (e.g., ; ), ODR is not by itself a trustworthy estimate of the probability that all conducted studies would produce significant findings.
The Expected Discovery Rate (EDR) is the model-based estimate of the proportion of significant results expected across the underlying set of conducted studies under the fitted selection model. Thus, EDR estimates how often studies in the literature would be expected to yield significant results before selection for significance is taken into account.
The Expected Replication Rate (ERR) is the estimated average probability of significance among the subset of studies that produced significant results. It corresponds to the expected success rate of close replications using the same design, sample size, significance threshold, and direction of effect. Because ERR is conditional on studies already having produced significant results, it is usually higher than EDR. ERR can therefore be interpreted as an optimistic benchmark for replication success among published significant findings. Validation studies comparing z-curve estimates with large replication projects support this interpretation (; ), although actual replication success may be lower when replications differ from original studies in samples, procedures, or contexts (; ).
A central use of z-curve is to compare ODR and EDR. When the observed share of significant results substantially exceeds the model-implied expected discovery rate, the discrepancy is consistent with selection for statistical significance under the assumptions of the model (; ). However, ODR–EDR discrepancies should be interpreted as model-based indicators rather than direct measurements of all unpublished studies or researcher decisions.
Z-curve estimates can also be used to compute false-discovery risk (FDR) using Sorić's logic (; ). Here, FDR does not denote the observed proportion of false positives among the extracted studies. Rather, it is an upper-bound estimate of the proportion of statistically significant findings that could be false discoveries, given the estimated discovery rate and the assumptions of the model. Sorić's logic is following: If only a small proportion of all conducted studies are expected to produce statistically significant results before publication selection (i.e., the EDR is low), then even a relatively small number of false-positive results can constitute a substantial share of the published significant findings. Conversely, when truly significant findings are expected to occur more frequently (higher EDR), false positives necessarily make up a smaller proportion of the significant record. For example, suppose the model estimates that only about 10% of all conducted studies would have produced statistically significant results before selection. Under Sorić's logic, this low discovery rate implies that false discoveries could comprise a substantial fraction of the published significant findings. By contrast, if the estimated discovery rate were much higher, the maximum plausible proportion of false discoveries would necessarily be lower. Accordingly, in z-curve applications the EDR serves as the selection-adjusted estimate of the underlying discovery rate from which this upper-bound FDR is derived. FDR should therefore not be interpreted as the exact proportion of false positives in the literature, nor does it identify which individual findings are false. Rather, it quantifies the maximum false-discovery risk that remains compatible with the estimated discovery rate under the assumptions of the model (for more information, see ; ).
Z-curve is particularly informative for heterogeneous literatures because it does not require all included studies to test the same effect or to have the same probability of significance. This is important for PMT, where studies vary in threat induction, outcome domain, sample size, audience context, and analytic specification. Compared with p-curve, z-curve explicitly models heterogeneity in the probability of significance and is therefore often preferred for heterogeneous literatures ().
At the same time, it is important to consider what z-curve does and does not. First, it does not estimate the average substantive effect size of masculinity threat. Second, it cannot determine whether any specific significant finding is true, false positive, inflated, or context dependent. Third, its estimates depend on assumptions about the selection process. The standard model assumes that statistically significant findings are more likely to be reported or published than non-significant findings. If selection also occurs among already significant results—for example because some findings are larger, more attractive, or easier to publish—estimates may be biased.
Recent work has raised questions about the interpretation and performance of z-curve outputs under complex selection processes (). In particular, EDR estimates and ODR–EDR comparisons may be affected when selection is graded within significant findings or when dependence among extracted tests is ignored. These concerns reinforce the importance of clustered analyses and sensitivity checks. However, ) emphasize that this limitation is not unique to z-curve. All selection models rely on assumptions about how results enter the published record, and if the true selection mechanism differs from the model, estimates may be biased. Moreover, evidence for selection based on statistical significance is much stronger in psychology than evidence that authors or editors systematically suppress smaller-but-still-significant effects because they are too small (). Under the more established scenario of threshold-based selection for significance, z-curve remains informative for estimating selection bias, expected replication, and false-discovery risk.
Accordingly, ODR, EDR, ERR, and FDR should be interpreted as credibility indicators rather than definitive measurements of the unseen research process. A low EDR or ERR does not imply that PMT effects are absent, only that the published significant-results record is less informative under the fitted model. Likewise, subgroup differences may reflect variation in effect sizes, measurement precision, sample sizes, theoretical specificity, or selective reporting, rather than intrinsic differences in theoretical validity.
Finally, z-curve has been applied in several domains to evaluate the credibility of published findings, including experimental system-justification research (), moderation-effect literatures (; ), and emotion science (). ) situated z-curve among other methods for evaluating the trustworthiness of published findings, emphasizing its role in estimating discovery and replication rates.
As an external benchmark for the credibility concerns that motivate such methods, the ) reproducibility project remains a widely cited large-scale replication effort illustrating that published significance does not guarantee successful replication (see also ). These reviews illustrate the core value for the z-curve, which provides a quantitative characterization of expected replicability and selection bias in a heterogeneous experimental literature. These applications illustrate the broader use of z-curve as a tool for quantifying expected replicability, discovery rates, and potential selection bias in heterogeneous research literatures
Present study
The review was pre-registered on the Open Science Framework (OSF: https://osf.io/pcr2s/). Deviations from the preregistration are reported in the text where they occurred.
Our primary pre-registered research question was: Among experimentally manipulated tests of precarious manhood theory (2008–2025), what are the field-level Observed Discovery Rate (ODR), Expected Discovery Rate (EDR), Expected Replication Rate (ERR), and False-Discovery Risk (FDR) as estimated with z-curve?
We further pre-registered four secondary research questions:
Effect type. Does ERR differ between main-effect tests and interaction-effect tests?
Manipulation family. Do ODR, EDR, ERR, and FDR vary across manipulation families?
Temporal change. Have replicability metrics improved in the post-2015 open-science era compared with 2008–2014?
Publication status. Do peer-reviewed articles show different replicability estimates from unpublished literature (e.g., dissertations)?
Thus, our primary goals were four-fold. First, we estimated field-level ODR, EDR, ERR, and FDR for the full corpus and, following ), evaluated the robustness of conclusions across several significance thresholds (α = 0.05, 0.01, 0.005, 0.001). Second, we examined whether credibility indicators differ for main effects vs. interaction effects, given that interaction tests typically require substantially larger samples to achieve adequate power (). Third, we inspected whether credibility estimates differ between studies published in or before 2015 vs. after 2015, treating 2015 as a marker of intensified attention to reproducibility and open science norms in psychology (see ). Fourth, because preregistration has become a central reform intended to improve the credibility and transparency of empirical research, we compared credibility indicators for preregistered vs. non-preregistered studies.
We also preregistered subgroup analyses by types of masculinity-threat manipulations. Following ), we operationalized subgroups as the type of manipulation (mode of threat delivery), because it provides a transparent, reliably codable grouping that also captures several other threat features that are often empirically linked with delivery mode (e.g., whether the target is the individual vs. men-as-a-group, and whether the threat is more local vs. global). Because z-curve evaluates properties of the published significant-results record rather than underlying effect magnitudes, these subgroup analyses should be interpreted as comparisons of evidential profiles rather than tests of the theoretical validity of particular manipulation families.
Two preregistered comparisons proved non-applicable or less informative in practice. Although we considered examining between- vs. within-subject designs, all eligible studies employed between-subject designs. We also preregistered comparisons based on publication status (journal articles vs. theses/dissertations). However, because our primary interest was in reform-relevant indicators of research credibility, we prioritized preregistration and publication-era comparisons.
Overall, the aim of this study is not to evaluate whether PMT is correct or to estimate the average size of masculinity-threat effects. Rather, it is to assess the credibility characteristics of the published significant-results record and to identify conditions under which future PMT research may yield more informative and reproducible evidence.
Method
Literature search
Potential sources for inclusion were identified via title-, keyword-, and abstract-level searches of five bibliographic databases: APA PsycINFO, ProQuest (including Dissertations & Theses Global), Web of Science Core Collection, Scopus, and Directory of Open Access Journals (DOAJ). The search targeted the experimental precarious manhood or masculinity threat literature and was completed in the first week of July 2025. Our preregistered Boolean query string was (“precarious manhood” OR “manhood threat” OR “threatened manhood” OR “masculin* threat” OR “masculine identity threat” OR “gender status threat” OR “fragile masculin*”) AND (men OR male OR masculin*).
During execution, we omitted the population filter to increase sensitivity and applied the “adult men” requirement at screening. All records were imported into Rayyan for deduplication and screening. The database searches produced 1,131 records; after duplicate removal, 641 unique records underwent title/abstract screening by two reviewers. Of these, 140 proceeded to full-text screening. We then excluded 40 records (e.g., no threat manipulation, no men-only statistics, qualitative-only, or duplicate reporting). Citation tracing, including of ), yielded 24 additional records, 23 of which were eligible and were added to extraction. Although the preregistered window was 2008–2025, we retained 10 clearly relevant pre-2008 masculinity threat experiments as a sensitivity extension.
The final coded corpus comprised 188 unique samples contributing 452 focal tests, drawn from 99 peer-reviewed journal articles, 13 doctoral dissertations, and 11 master's theses. Full list of references is provided on the OSF. A PRISMA flow diagram is presented in Figure 1.
Figure 1
Inclusion and exclusion criteria
Inclusion criteria followed our preregistered PICOS framework specification (). We included experiments that (a) recruited self-identified adult men or reported separable men-only statistics; (b) used a researcher-controlled manipulation intended to threaten, question, or highlight the precarious status of manhood; (c) included at least one comparison condition (e.g., neutral control or masculine-affirming condition); and (d) reported at least one hypothesis test explicitly derived from PMT that yielded an exact or convertible two-tailed p-value. We considered peer-reviewed journal articles and doctoral or master's theses within the 2008–2025 window, while allowing a small number of clearly on-topic pre-2008 experiments identified through citation tracing. We excluded non-experimental designs, studies without a masculinity-threat manipulation, reports lacking men-only statistics or convertible inferential tests, and records without retrievable full text or unique empirical content. Duplicate reporting was handled by retaining the most complete empirical report and excluding redundant versions as duplicates. These criteria were applied during title/abstract and full-text screening by two reviewers, with disagreements resolved by discussion.
Information retrieved from the studies
Data extraction was conducted manually by two human coders using a preregistered codebook. For each included report, extractors recorded bibliographic and design descriptors (authors, year, publication type, doi, country, recruitment setting, design, manipulation, number of conditions, cell sizes, age, and gender composition where applicable). Extractors then identified every statistical test that evaluated a prediction derived from precarious-manhood theory within the male sample and extracted (a) the statistical result and (b) the surrounding verbatim text needed for transparent auditing (e.g., the paper's statement of the prediction and the reported inferential result).
Coding of masculinity threat manipulations
Guided by the taxonomy proposed by ), we coded each eligible study's masculinity-threat induction into one mutually exclusive manipulation type. The categories are listed in Table 1. When manipulations combined multiple elements, we coded them by the primary threat component. In addition to Rogalin and Addison's taxonomy, we introduced one category—Interpersonal threat—to capture manipulations in which masculinity is challenged within an actual or simulated social interaction (e.g., devaluing feedback, exclusion, or public evaluation as insufficiently masculine).
Table 1
| Manipulation category | Brief description |
|---|---|
| False gender knowledge test feedback | Bogus feedback indicating poor performance on a gender/masculinity knowledge test (implying low masculinity). |
| False personality feedback—BSRI | Bogus feedback based on the Bem Sex Role Inventory suggesting low masculinity/high femininity (or otherwise masculinity-threatening profile). |
| False personality feedback—CMNI | Bogus feedback based on the Conformity to Masculine Norms Inventory indicating low conformity to masculine norms (masculinity-threatening). |
| False personality feedback—other personality test | Bogus feedback from another personality measure implying low masculinity/inadequate masculine traits. |
| Other false feedback | Bogus masculinity-relevant feedback not covered above (e.g., other “diagnostic” feedback implying insufficient manhood). |
| Writing task | Self-reflective writing prompt about a masculinity-challenging event, personal inadequacy, or threatened manhood (threat induced via reflection). |
| Failed at masculine task | Performance failure or poor evaluation on a masculinity-typed task (e.g., strength, competition, spatial task framed as masculine). |
| Interpersonal threat | Masculinity challenged by another person/audience in an interaction (real or simulated), such as rejection, ridicule, exclusion, or public evaluation as “not masculine enough.” |
| General threat to men—men are becoming feminized | Societal-level information suggesting men are becoming feminized/masculinity norms are declining (group-based status threat framed as feminization). |
| General threat to men—other | Societal-level threat to men's status or masculinity that is not specifically framed as feminization (diffuse “threat to men” messages). |
| Imagined threat | “Vividly imagining” a masculinity challenge or threat scenario (threat induced via guided imagination). |
| Feminine activity | Enacting or being assigned a femininity-coded behavior/task/activity that threatens masculine identity. |
| Feminine product | Choosing, endorsing, or being associated with a femininity-coded product/preference as the threat induction. |
| Other | Manipulations that do not fit any category above or for which no primary threat component can be determined. |
Masculinity threat manipulation categories.
Coding of outcome types
To capture heterogeneity in what PMT experiments aim to change, we coded each focal dependent variable into one of three mutually exclusive outcome domains—affective, behavioral, or cognitive. Affective outcomes comprised emotions, affective states, arousal, or emotion-relevant physiological indicators; behavioral outcomes comprised observed behavior, choice/allocations, task performance, or behavioral intentions; and cognitive outcomes comprised attitudes, beliefs, judgments, self-evaluations/self-concepts, stereotypes, or related cognitive appraisals. When multiple outcomes were reported, we coded the outcome that corresponded most directly to the focal hypothesis test extracted for z-curve; when an outcome plausibly spanned domains, we classified it based on its operationalization (e.g., intention/choice coded as behavioral; appraisal/judgment coded as cognitive).
Coding tests
We extracted all eligible tests per study. For each eligible test, we extracted both the reported test statistic and the p-value as presented in the paper. Specifically, we recorded the test family (e.g., t, F), the associated degrees of freedom (when applicable), and the exact p-value when reported. When studies reported only threshold values (e.g., p < 0.05, p < 0.001) or when a p-value was not explicitly reported but a convertible statistic was, the exact p-value was computed from the test statistic and df and stored alongside the originally reported information. All extracted tests were subjected to plausibility screening (e.g., impossible combinations of df and test values, mismatches between stated significance and the implied p-value).
Analytical strategy
Analyses followed our preregistered plan to estimate field-level credibility and replicability in the experimental precarious-manhood literature using z-curve. All computations were conducted in R using the z-curve package (), with conceptual interpretation guided by the z-curve framework described by ) and subsequent extensions (e.g., ).
Preparation of inputs
Each eligible result was represented as a male-sample, two-tailed p-value, either extracted directly from the report or recomputed from an extracted test statistic and degrees of freedom. Two-tailed p-values were converted to two-tailed z-scores, which are the standard input for z-curve. We retained exact p-values whenever available. When p-values were reported as inequalities (e.g., p < 0.001) or were rounded, we recomputed the exact two-tailed p-value from the underlying test statistic and df and used that exact value for z-curve.
Z-curve model
Z-curve is a selection model that infers the properties of the full set of conducted studies from the distribution of statistically significant results. It models selection for significance by assuming that the probability a study enters the literature is proportional to its power. Under this assumption, the distribution of observed significant z-scores is a biased subset of the underlying distribution, and z-curve estimates the degree of this distortion.
Z-curve leverages the distribution that, for a set of studies with a given power, z-statistics are approximately normally distributed around a mean corresponding to that power. Because real literatures are heterogeneous (different effects, measures, and sample sizes), z-curve models the observed significant z-scores as a mixture of multiple components, implemented as truncated folded-normal distributions. Mixture weights are estimated via an expectation–maximization (EM) algorithm to best reproduce the observed distribution of significant z-scores, after which the model extrapolates to the implied full distribution (including non-significant results).
From each fitted model we extracted the preregistered indicators: (1) ODR: proportion of extracted tests that were significant at α = 0.05; (2) EDR: selection-adjusted estimate of the proportion of significant results before selection for significance (i.e., the discovery rate expected across all conducted studies); (3) ERR: mean power after selection for significance, interpreted as the predicted proportion of significant results that would again be significant in exact replications with the same designs and sample sizes. ERR typically exceeds EDR because selection for significance preferentially retains higher-powered study realizations; (4) FDR: an upper-bound estimate of the proportion of significant findings that could be false positives at α = 0.05, computed as a transformation of the EDR using Sorić's formula.
Because multiple eligible tests could be extracted from the same study, we addressed within-article dependence using a cluster bootstrap (10,000 iterations). In each bootstrap iteration, we sampled one test per article for model fitting. Furthermore, for conversion accuracy, tests based on very small samples (N < 30) were not used in z-curve modeling because conversion to z-scores would be unreliable with such small samples.
Several implementation choices in the present review directly address the main threats to validity raised by ). First, we manually coded focal hypothesis tests rather than automatically scraped statistics, not coding manipulation checks and auxiliary analyses. Second, we structured extraction at the sample level and used clustered estimation to account for residual within-study dependence, rather than treating multiple tests from the same article as independent draws. Third, we examined the robustness of our conclusions across different significance thresholds and supplemented the main z-curve analyses with diagnostic checks on the shape of the z-value distribution reported in Supplementary materials.
To address preregistered secondary questions, we fit separate z-curve models for subsets defined by: effect type (main vs. interaction), manipulation family, publication period ( ≤ 2014 vs. 2015–2025), and preregistration status (yes/no). Subgroup z-curve models were fit only when the subset contained at least 10 significant results. Data and R codes are available at OSF, https://doi.org/10.17605/OSF.IO/8QNHP.
Results
Description of sample characteristics
Across all included studies, publication years ranged from 1979 to 2025. Sample sizes of men only varied from N = 24 to N = 867 (Figure 2), with a median of N = 126 and a mean of N = 167.2. Data were collected in 21 countries (Table 2). Most samples came from the USA, followed by the UK, Canada, Poland, Australia, Turkey, and Italy. The remaining samples were distributed across a range of European, Latin American, and Asian countries, as well as a small number of multi-country samples.
Figure 2
Table 2
| Country | Samples | Effects |
|---|---|---|
| USA | 135 | 331 |
| UK | 8 | 14 |
| Canada | 6 | 20 |
| Poland | 5 | 10 |
| Australia | 4 | 18 |
| Turkey | 4 | 15 |
| Italy | 4 | 9 |
| Switzerland | 3 | 4 |
| China | 2 | 6 |
| Germany | 2 | 6 |
| Sweden | 2 | 3 |
| UK/US | 2 | 3 |
| Brazil | 2 | 2 |
| Ecuador | 2 | 2 |
| Norway | 1 | 3 |
| Spain | 1 | 2 |
| Switzerland & France | 1 | 2 |
| Kosovo | 1 | 1 |
| Taiwan | 1 | 1 |
| UK/USA/Canada/EU | 1 | 1 |
| USA/Canada | 1 | 1 |
Number of samples and effects by country.
Figure 3 plots the distribution of sample sizes by publication year. Early work relied predominantly on relatively small samples, whereas more recent studies tend to employ larger samples, with limited studies after 2015 exceeding N = 500. To formally test the visual trend in Figure 4, we regressed log10(sample size) on year of publication (centered). This test was not pre-registered. Sample sizes increased significantly over time, b = 0.024, SE = 0.003, 95% CI [0.018, 0.031], t(186) = 7.73, p < 0.001, R2 = 0.24. On the original scale this corresponds to approximately a 1.7-fold increase in average sample size per decade.
Figure 3
Figure 4
Regarding experimental operationalizations of precarious manhood, we coded 14 distinct manipulation types (Figure 4). The most common categories involved false feedback manipulations: false gender-knowledge test feedback, false personality feedback based on the Bem Sex-Role Inventory (BSRI), other false personality-feedback tests, and false feedback based on the Conformity to Masculinity Norms Inventory (CMNI). Other frequently used manipulations were writing tasks, interpersonal threat, general threat-to-men messages focusing on men's alleged feminization or other forms of general threat, and “failed at masculine task” procedures. Less common were feminine activities, imagined threats, other false-feedback variants, feminine products, and “other” manipulations.
Only a small proportion of samples was formally preregistered: 16 of 188 coded samples (8.5%). As shown in Figure 5, preregistered studies tended to use larger samples (M = 305.6, SD = 197.2, n = 16) than non-preregistered studies (M = 154.4, SD = 137.9, n = 172). An exploratory, non-preregistered Welch t-test on log10-transformed sample sizes indicated that this difference was statistically significant, t(19.3) = −4.84, p < 0.001, Cohen's d = −1.27.
Figure 5
At the effect level, we distinguished between main effects (259 effects) and interaction effects (193 effects). As depicted in Figure 6, sample sizes used to test main effects (M = 178.2, SD = 177.5, n = 259) and interactions (M = 177.4, SD = 152.5, n = 193) were similar. An exploratory, non-preregistered Welch t-test on log10(N) confirmed that there was no significant difference in sample size as a function of whether the test reflected a main effect or an interaction, t(443.1) = −1.21, p = 0.226, Cohen's d = −0.12. This similarity is unsurprising, as many samples contributed both a main effect and an interaction effect, such that the same total N was used to evaluate multiple types of tests within a given study.
Figure 6
Finally, Figure 7 displays the distribution of all focal z-values. Three features are worth noting. First, there is a clear visual drop in the frequency of tests just below the two-sided significance threshold (z ≈ 1.96) and a visible spike in the adjacent “just-significant” band (orange region, 1.96–2.6), indicating an over-representation of results that barely exceed the conventional threshold. Second, relatively few tests fall just above this band, and the distribution then tapers off gradually in the higher-z region highlighted in blue (z ≥ 2.6, roughly corresponding to α ≈ 0.01), suggesting that tests based on large samples sized are rare. Third, there is a long but thin right tail of very large z-values, consistent with a mixture of a few strong effects and many low-powered tests. Overall, the shape of the histogram is consistent with a literature characterized by substantial selection for statistically significant results and limited average power.
Figure 7
Z-curve estimates
Clustered z-curve estimates for all tests are summarized in Table 3. Across the full set of p-values, the ODR was substantially higher than the EDR at α = 0.05, and their confidence intervals did not overlap. Across the full corpus of extracted tests, the clustered z-curve estimates indicated a marked gap between the observed discovery rate and the expected discovery rate at α = 0.05 (ODR ≈ 0.51 vs. EDR ≈ 0.10), pointing to substantial selection for statistical significance.
Table 3
| Z-curve subset | ksig | ktotal | ODR [95% CI] | EDR [95% CI] | ERR [95% CI] | Soric FDR [95% CI] |
|---|---|---|---|---|---|---|
| All p-values | ||||||
| α = 0.05 | 229 | 452 | 0.507 [0.460, 0.554] | 0.103 [0.050, 0.189] | 0.383 [0.300, 0.467] | 0.458 [0.225, 1.000] |
| α = 0.01 | 110 | 452 | 0.243 [0.186, 0.265] | 0.268 [0.071, 0.451] | 0.433 [0.330, 0.549] | 0.028 [0.012, 0.133] |
| α = 0.005 | 85 | 452 | 0.188 [0.154, 0.228] | 0.017 [0.005, 0.152] | 0.355 [0.240, 0.475] | 0.282 [0.028, 1.000] |
| α = 0.001 | 50 | 452 | 0.111 [0.084, 0.144] | 0.002 [0.001, 0.055] | 0.472 [0.331, 0.623] | 0.418 [0.017, 1.000] |
| Manipulation type | ||||||
| False gender knowledge test feedback | 59 | 108 | 0.546 [0.448, 0.641] | 0.112 [0.050, 0.295] | 0.366 [0.243, 0.495] | 0.418 [0.126, 1.000] |
| False personality feedback—BSRI | 42 | 93 | 0.452 [0.349, 0.558] | 0.120 [0.050, 0.517] | 0.396 [0.260, 0.554] | 0.385 [0.049, 1.000] |
| False personality feedback—CMNI | 21 | 40 | 0.525 [0.363, 0.682] | 0.140 [0.050, 0.666] | 0.370 [0.081, 0.715] | 0.324 [0.026, 1.000] |
| False personality feedback—other personality test | 35 | 81 | 0.432 [0.324, 0.547] | 0.190 [0.050, 0.589] | 0.328 [0.096, 0.589] | 0.224 [0.037, 1.000] |
| General threat to men | 15 | 24 | 0.625 [0.408, 0.804] | 0.241 [0.089, 0.618] | 0.648 [0.509, 0.759] | 0.165 [0.033, 0.542] |
| Interpersonal threat | 11 | 18 | 0.611 [0.361, 0.817] | 0.293 [0.182, 0.624] | 0.617 [0.478, 0.742] | 0.127 [0.032, 0.236] |
| Writing task | 14 | 30 | 0.467 [0.288, 0.654] | 0.061 [0.050, 0.170] | 0.123 [0.025, 0.372] | 0.812 [0.257, 1.000] |
| Other masculinity threats | 32 | 58 | 0.552 [0.416, 0.680] | 0.063 [0.050, 0.165] | 0.131 [0.047, 0.279] | 0.785 [0.266, 1.000] |
| Preregistration | ||||||
| No | 207 | 414 | 0.500 [0.452, 0.548] | 0.091 [0.050, 0.174] | 0.361 [0.273, 0.448] | 0.526 [0.249, 1.000] |
| Yes | 22 | 38 | 0.579 [0.409, 0.733] | 0.345 [0.170, 0.644] | 0.539 [0.360, 0.678] | 0.100 [0.029, 0.257] |
| Publication period | ||||||
| 1979–2014 | 57 | 101 | 0.564 [0.462, 0.662] | 0.060 [0.050, 0.126] | 0.152 [0.048, 0.267] | 0.826 [0.364, 1.000] |
| 2015–2025 | 172 | 351 | 0.490 [0.437, 0.544] | 0.146 [0.051, 0.324] | 0.469 [0.373, 0.564] | 0.307 [0.110, 0.987] |
| Effect type | ||||||
| Main effect | 110 | 259 | 0.425 [0.364, 0.488] | 0.187 [0.061, 0.433] | 0.424 [0.321, 0.532] | 0.229 [0.069, 0.809] |
| Interaction effect | 119 | 193 | 0.617 [0.544, 0.685] | 0.081 [0.050, 0.151] | 0.335 [0.250, 0.421] | 0.596 [0.295, 1.000] |
| Outcome type | ||||||
| Affective | 49 | 73 | 0.671 [0.550; 0.774] | 0.081 [0.05; 0.153] | 0.335 [0.250; 0.420] | 0.125 [0.026; 0.406] |
| Behavioral | 51 | 116 | 0.440 [0.349; 0.535] | 0.076 [0.05; 0.154] | 0.257 [0.165; 0.359] | 0.642 [0.289; 1.000] |
| Cognitive | 129 | 263 | 0.490 [0.429; 0.552] | 0.113 [0.05; 0.279] | 0.306 [0.20; 0.416] | 0.414 [0.136; 1.000] |
Detailed clustered z-curve results.
Results from clustered z-curve analyses (cluster = sample) with bootstrap confidence intervals. ksig, number of significant focal tests (p < 0.05) included in the analysis; ktotal, total number of focal tests in that subset. ODR, observed discovery rate, EDR, estimated discovery rate, ERR, expected replication rate, Soric FDR, Soric false discovery risk.
The ERR estimate (≈0.38 at α = 0.05) suggests that exact replications using similar designs and sample sizes would be expected to succeed more often than not only in some subsets, but not reliably at the field level. Finally, the Soric FDR at α = 0.05 is non-trivially large and confidence interval was imprecisely estimated, indicating that a non-negligible share of statistically significant findings could be false positives.
As a retrospective sensitivity analysis, we also examined how z-curve indicators change when more stringent significance thresholds are applied to a literature that was overwhelmingly conducted and published under the conventional α = 0.05. These analyses characterize the evidential profile of the existing published record; they should not be interpreted as predictions about what would happen if future PMT research prospectively adopted α = 0.01 or another threshold. Such a change would likely alter study design, sample-size planning, publication incentives, and researcher behavior, potentially changing the resulting credibility indicators.
Within the existing literature, however, lowering the analysis threshold from α = 0.05 to α = 0.01 substantially reduces the estimated upper-bound FDR while maintaining a non-trivial ODR and an ERR consistent with meaningful expected replication. In contrast, moving to α = 0.005 or 0.001 restricts the analysis to the extreme tail of the z-distribution, leaving too few studies for stable estimation.
This is reflected in very low EDRs, wide confidence intervals, and FDR estimates whose intervals extend to 1, making them largely uninformative. Thus, the present analyses suggest that, among findings already published under the α = 0.05 research culture, results surviving the more stringent α = 0.01 threshold provide stronger evidential support than results lying just below the conventional 0.05 threshold. They should not be interpreted as evidence that prospectively adopting α = 0.01 would necessarily produce the same improvement in field-level credibility.
Manipulation types
For analysis, we combined the types of manipulations with small numbers of tests (see Table 3). The various “false feedback” manipulations (false gender-knowledge feedback and the three personality-feedback families) generally show moderate ODRs but noticeably lower EDRs, together with moderate ERRs and wide confidence intervals for the FDR. This combination points to selection bias and only modest expected replicability, but with substantial uncertainty in the exact proportion of true vs. false positives.
By contrast, “general threat to men” and “interpersonal threat” manipulations show both higher EDR and higher ERR, alongside lower FDR and narrower intervals. Although these estimates are based on fewer tests, they suggest comparatively stronger evidential value and higher expected replicability for these manipulation types.
In turn, the “writing task” and “other masculinity threats” categories combine relatively high ODRs with very low EDRs, low ERRs, and high FDRs. This pattern is consistent with pronounced selection bias and weak evidential value in these subsets, although the wide confidence intervals again call for cautious interpretation.
Preregistration
Non-preregistered tests show the pattern of ODR exceeding EDR, with only moderate ERR and a comparatively high and imprecise FDR, indicating substantial selection bias and a sizeable false discovery risk. Preregistered tests, in contrast, show a higher EDR, higher ERR, and a much lower Soric FDR, even though the confidence intervals are wide due to the smaller number of tests from preregistered studies. In other words, preregistered studies appear to produce more replicable evidence with a lower proportion of false positives.
Changes over time
Comparing pre-2015 and post-2015 studies points to a modest improvement in evidential value over time. As indicated in Table 3, earlier studies show high ODR combined with very low EDR and ERR, and a high FDR, which together indicate strong selection bias, low expected replicability, and a substantial risk that a sizeable fraction of significant results are false positives.
More recent studies display a somewhat lower ODR but clearly higher EDR and ERR and a lower FDR. This pattern is compatible with the idea that, in the wake of the replication crisis and broader reforms in methodology and reporting, precarious-manhood research has shifted toward designs that yield a somewhat more trustworthy evidential base—although selection bias remains visible and the upper confidence bounds for FDR are still very high.
Main effects vs. interaction effects
When comparing main-effect and interaction-effect tests, main effects show a lower ODR than interactions but relatively higher EDR and ERR and a lower FDR. Interaction tests, by contrast, have the highest ODR in the table but comparatively low EDR and ERR and a high FDR, with wide confidence intervals.
This pattern suggests that interaction findings in this literature are especially prone to selection bias and are less likely to replicate than main-effect findings. Given that many samples contributed both main and interaction tests, and that estimates for some subsets are imprecise, these differences should be interpreted cautiously—but they align with the general notion that interaction effects are harder to detect reliably and therefore more vulnerable to selective reporting.
Outcome types
Finally, in an exploratory, non-preregistered subgroup analysis by outcome domain, the pattern suggests that affective outcomes show the most “successful” published record (highest ODR) but do not exhibit a clearly stronger underlying evidential profile (EDR remains low), whereas behavioral outcomes appear most vulnerable to false discoveries (highest estimated FDR and lowest ERR). Cognitive outcomes fall between these extremes, with credibility indicators generally closer to the overall corpus.
Discussion
Precarious manhood theory proposes that manhood is a socially constructed status that is difficult to earn and easy to lose, and that men respond to threats to this status with compensatory, masculinity norm-consistent attitudes and behavior (e.g., aggression, risk-taking, homonegativity; ; ). This idea has become a popular organizing framework for explaining men's behavior in contexts of social change, shifting gender norms, and audience-driven identity regulation (). Reviews of masculinity-related threat processes similarly treat threatened masculine identity as a theoretically central pathway to men's aggression and violence and related defensive responses ().
Recent evidence gives a mixed cumulative picture. On the one hand, effect-size meta-analysis suggests that masculinity threat has a detectable average effect and that effects vary across threat triggers, outcomes, publicness, and outcome order (). On the other hand, recent preregistered replication work has failed to reproduce some influential masculinity-threat effects in larger and more representative samples, particularly for political-attitude outcomes (). These findings are not necessarily contradictory. A literature can contain detectable average effects, context-dependent effects, inflated estimates, and false-positive findings at the same time. The present study therefore addressed a different question from conventional effect-size meta-analysis: how credible and replicable is the published significant-results record of experimental PMT research when selection for significance is modeled directly using z-curve (; )?
Field-level credibility and expected replicability
At the level of the full corpus, the z-curve pattern is most consistent with a literature in which statistical significance has operated as a substantial selection mechanism. The observed rate of statistically significant results was substantially higher than the rate implied by the fitted z-curve model (). This inference is model-conditional and should be read as a characterization of the published significant-results distribution rather than a reconstruction of the full, unseen research process, but it implies that the experimental PMT literature likely mixes comparatively informative tests with findings whose published status is contingent on selection for statistical significance.
Two cautions follow directly from what z-curve does—and does not—identify. First, z-curve evaluates whether the distribution of significant results is consistent with a mixture of probabilities of obtaining statistically significant results under selection, but it does not pinpoint which specific questionable research practices, if any, produced that distribution (). The observed pattern may reflect publication bias, selective outcome reporting, undisclosed analytic flexibility, incomplete reporting of non-significant results, or some combination of these processes. Second, ODR should not be interpreted in isolation. A literature can show substantial selection even when the observed proportion of significant focal tests is not extremely high, because ODR is jointly shaped by the underlying probability of significance and by selective visibility of results. Similarly, FDR should not be read as the observed rate of false positives. Instead, it is an upper-bound FDR: the maximum proportion of false discoveries.
A further implication is that published effect-size estimates in the PMT literature may be upwardly biased. When statistically significant findings are selectively visible, effect-size estimates based primarily on the published record can overstate underlying effects, particularly in literatures characterized by modest discovery rates and substantial selection for significance. This concern also applies to quantitative syntheses. While the recent meta-analysis by ) represents a valuable effort to synthesize PMT experiments to date, its pooled estimate may nevertheless remain inflated to the extent that the available primary literature itself contains selectively visible statistically significant findings. This possibility should be regarded as a methodological caution rather than a conclusion about that particular meta-analysis, because z-curve evaluates the evidential profile of the underlying literature rather than the validity of any individual synthesis. Accordingly, we caution against using the pooled estimate reported by ) as a benchmark for planning future PMT experiments without accounting for selection-related effect size inflation in the underlying literature. For this reason, following ), we did not conduct a conventional effect-size meta-analysis and instead focused on selection-adjusted credibility and replicability indicators.
Sensitivity to the significance threshold
A practically important implication of our α-level analyses is that credibility inferences depend strongly on how much evidential weight is placed on results just below 0.05. Methodological work argues that when selection for significance operates, p-values tend to pile up just below conventional thresholds, and that results in the 0.05–0.01 range should therefore be treated with caution when evaluating existing literatures ().
Our results align with that logic: within the existing published literature, findings that remain statistically significant at α = 0.01 are associated with substantially lower estimated upper-bound false-discovery risk than findings evaluated at α = 0.05. Accordingly, readers are encouraged interpret existing single-study PMT claims supported primarily by p-values between 0.01 and 0.05 more cautiously unless they are accompanied by strong design features (e.g., preregistration, transparent reporting, precise sample-size justification) or successful direct replication. Importantly, this conclusion is retrospective. It should not be interpreted as implying that prospectively changing the field-wide significance threshold to α = 0.01 would necessarily yield comparable improvements, because such a change would also alter researcher behavior, study design, publication incentives, and the selection process itself.
Subgroups
Several moderator patterns are theoretically and methodologically informative to consider. Differences across subgroups may reflect differences in effect sizes, sample size, measurement precision, analytic flexibility, selective reporting, or how directly a given manipulation and outcome test PMT's proposed threat-and-restoration process. Z-curve cannot distinguish among these explanations on its own.
Main effects vs. interaction effects
Interaction tests showed a higher ODR but lower EDR and lower ERR than main effects, along with higher estimated false-discovery risk at α = 0.05. This pattern is consistent with an experimental design reality: interactions typically require larger samples than main effects to be estimated with adequate precision, and small-sample factorial designs make it easier for the published record to overrepresent significant interaction findings (). In PMT specifically, where moderators are often theoretically central (e.g., ideology, gender norm endorsement), this pattern implies that moderator-heavy claims should be treated as among the most uncertainty-prone parts of the published record unless they are built on designs explicitly powered for interactions.
Publication period
Post-2015 studies showed higher ERR and higher EDR (with lower FDR) than the pre-2015 subset, although selection bias remained. This pattern parallels “two-crisis” accounts emphasizing that psychology's credibility improvements have been uneven and incomplete: transparency norms have strengthened, but incentives for clean narratives and publishable results still shape the publication reality (). Crucially, this also complicates simple narratives in either direction: improvements in later work can strengthen confidence in some PMT effects while still leaving the earlier record comparatively uncertain.
Manipulation families
Some manipulation types (e.g., interpersonal threat; general threat to men) yielded comparatively higher ERR and lower estimated false-discovery risk, whereas other categories (e.g., writing tasks; heterogeneous “other threats”) showed particularly low EDR/ERR and very high uncertainty. At minimum, this indicates that the PMT experimental toolbox is not psychometrically equivalent; some manipulations may be better specified, produce larger effects, or be implemented with better-powered designs. However, z-curve is not equipped to distinguish whether these differences reflect (a) true effect-size heterogeneity across paradigms, (b) different levels of selection, or (c) both. This underscores the importance of strengthening construct-to-manipulation links rather than treating all threats as interchangeable demonstrations of the same theoretical claim. Future PMT studies should specify what aspect of masculinity is threatened, why the manipulation should be experienced as threatening by the target sample, which audience is implied or present, and why the selected outcome should follow from that threat.
Outcome types
We also examined clustered z-curve estimates separately for affective, behavioral, and cognitive outcomes. Across these three families, the basic pattern resembled the overall picture: ODRs were consistently higher than EDRs and ERRs, and all three outcome types showed evidence of selection for statistical significance rather than uniformly high probabilities of obtaining statistically significant results. This does not mean that affective, behavioral, and cognitive outcomes are theoretically equivalent. Affective outcomes such as anxiety, anger, shame, or public discomfort assess immediate responses to threatened manhood, whereas behavioral and political outcomes often require additional assumptions about whether the outcome is culturally coded as masculine and whether the setting gives participants an opportunity to restore masculinity through that response.
Preregistration
The preregistered subset showed a higher EDR, higher ERR, and markedly lower estimated false-discovery risk than the non-preregistered subset. This result corresponds to broader arguments that preregistration and Registered Reports can reduce undisclosed analytic flexibility, clarify confirmatory vs. exploratory inference, and weaken incentives for selective reporting (). In PMT, where outcome scoring and exclusion rules are often consequential, preregistration is particularly valuable as a way to increase the interpretability and severity of tests by making analytic decisions more transparent and reducing ambiguity about which analyses were planned in advance. However, this subgroup comparison should be interpreted cautiously because preregistered studies are also a smaller and more recent subset of the literature and may differ from non-preregistered studies in sample size, transparency norms, and reporting practices.
Selection for significance is not the only source of uncertainty in cumulative evidence. Even when researchers act in good faith and report their analyses transparently, different defensible analytic decisions can lead to different conclusions. Evidence from recent “many-analysts” research shows that, even when multiple analysts address the same research question using the same dataset, reasonable analytic decisions can generate a distribution of effect estimates and conclusions rather than a single stable answer ().
In ), most re-analyses reached the same qualitative conclusion as the original study, but a substantial minority yielded null or inconclusive outcomes, and effect estimates often varied beyond narrow tolerance bands. This pattern implies that single-path analyses in the social and behavioral sciences should not be assumed to be robust to alternative, defensible specifications.
PMT studies are also exposed to this issue because common outcome domains (e.g., aggression proxies, risk-taking tasks, affect measures, social judgments) often permit multiple defensible scoring rules, exclusion criteria, covariate sets, and modeling strategies. In such settings, analytic flexibility can interact with publication selection to amplify the visibility of results that narrowly cross conventional thresholds.
Consistent with this concern, the present z-curve results suggest that at least some findings—especially those supported by p-values near 0.05—may reflect the combined influence of selection for significance and specification sensitivity. This statement should not be read as evidence that any particular PMT study involved questionable research practices. The more precise conclusion is that parts of the PMT record are vulnerable to selection and specification sensitivity when many defensible analytic choices are available and statistically significant results are more visible. Preregistration and Registered Reports are specifically intended to mitigate this mechanism by requiring researchers to document key analytic decisions in advance, thereby reducing undisclosed flexibility and clarifying which analyses are confirmatory vs. exploratory ().
Implications for theory-building in precarious manhood research
At a theoretical level, our results do not imply that PMT is “false” or that masculinity threat never produces compensatory behavior. Instead, they suggest that the published experimental record most likely overstates evidential strength, and that a non-trivial share of statistically significant claims—particularly those supported primarily by p-values in the 0.05–0.01 range and interaction tests based on small sample sizes—should be treated as uncertain.
This distinction matters because PMT is used not only to explain laboratory outcomes but also to inform interventions and public-facing claims about men's health behavior, violence, and responses to gender equality initiatives (e.g., ; ). A credibility review is therefore a theory-maintenance task: it clarifies which parts of the evidential base can carry interpretive weight and where the field most needs high-power replication.
Recommendations for future precarious manhood theory experiments
Before turning to recommendations, we clarify their scope. The present z-curve analysis evaluates the credibility of the published significant-results record; it does not adjudicate among specific PMT mechanisms, manipulation families, or outcome domains. Because evidence of selection was present across much of the literature, and because subgroup estimates were not precise enough to identify any clearly robust segment of the record, we cannot offer strong theory-specific recommendations about which PMT pathways should be retained, revised, or abandoned. The recommendations below therefore focus on the methodological conditions needed to make future experiments more informative: explicit design planning, transparent reporting, preregistration, direct and multilab replications, and stronger manipulation validation.
First, researchers should replace “rule-of-thumb” sample-size rationales with explicit justifications that link design, smallest effect size of interest, and error costs, even when resources are limited (for practical guidelines, refer to ). Second, α should be justified rather than defaulted to 0.05; where false-positive costs are substantial, formal justification and, in some cases, a lower α may be warranted ().
Third, wider uptake of preregistration and Registered Reports is likely to improve evidential value, as suggested by the differences between preregistered and non-preregistered subsets in our dataset (see ). Fourth, we encourage authors to prioritize transparency over narratives by reporting null outcomes rather than selectively emphasizing “clean” results. Null findings from well-designed studies should be treated as informative constraints on PMT, not as non-results.
Fifth, replicate important findings using samples from multiple countries, including non-WEIRD (western, educated, industrial, rich, democratic) contexts. Replication resources should prioritize (a) widely cited claims (for an informative example, see ), (b) widely used manipulation types, and (c) interaction-heavy hypotheses that have relied on small per-cell sample sizes—the conditions under which selection pressures and limited power are most likely to distort the published record (; ).
Relatedly, we join ) in urging researchers to devote greater methodological attention to manipulation construction and validation: what, specifically, constitutes a “threat” to manhood, for whom, and under which cultural and situational conditions? Manipulation validation should not be treated as a routine check only; it should establish that the manipulation changes the intended psychological state in the target sample and that the selected outcome is a plausible response to that state in the specific context. To strengthen causal interpretability and reduce ambiguity in construct-to-manipulation links, we refer readers to practical guidance on causal reasoning and study design (see ; ).
Recent work on the concept of unmanliness indicates that the content of “unmanliness” varies across countries (); as a result, threat paradigms developed and repeatedly used in U.S. samples may not transfer easily to other cultural settings. Given the dominance of U.S. (and, to a lesser extent, European) samples in PMT experiments, it remains an open empirical question whether—and in what form—manhood threats elicit compensatory responses in non-WEIRD contexts.
Limitations
As with any synthesis, this review inherits the assumptions of the method. Z-curve models selection as a function of the probability of obtaining statistically significant results and is designed to infer discovery and replication properties from the distribution of significant results under heterogeneity (; ). Accordingly, z-curve is most informative about the implications of selective visibility for the significant-results record, but it does not identify the mechanisms that produced that record. If questionable research practices generate statistically significant outcomes through mechanisms that are not well-captured by this selection process, some parameter estimates may be biased; moreover, z-curve cannot identify which specific questionable research practices, if any, produced the observed distribution. Estimates may also be biased if selection operates not only around the conventional significance threshold but also among already significant results, for example if larger, cleaner, or more theoretically appealing significant effects are more likely to be reported. Dependence among extracted tests is another concern, although our clustered procedures were designed to reduce the influence of non-independence.
A further limitation is that several subgroup subsets remain relatively small, leading to wide uncertainty intervals and constraining the precision of subgroup conclusions. Because the PMT literature continues to expand, the present estimates should be treated as time-bound, and repeating the analysis in future would be informative for assessing whether selective reporting patterns and credibility metrics have shifted. Several plausible moderators could also not be meaningfully evaluated with the available record: age and covariate variables were inconsistently reported and difficult to harmonize; country comparisons were limited by the small number of non-U.S. studies; and sample size is closely intertwined with other design and analytic features, so we are cautious interpreting z-curve outputs as isolating the influence of N per se.
We also deliberately did not include published meta-analyses themselves as units of analysis. This decision reflects the logic of z-curve, which operates on primary test statistics rather than on secondary pooled estimates. Meta-analyses involve additional synthesis decisions, including inclusion criteria, effect-size extraction rules, dependence handling, and publication-bias corrections. Excluding meta-analyses therefore allowed us to focus on the primary experimental record, thereby complementing recent meta-analytic work.
Finally, we excluded tests based on very small samples to avoid distortions arising from poor approximations when converting common test statistics to z-values. This decision likely improved model stability, but it also necessarily omits a small segment of the evidential record.
Conclusion
Precarious manhood remains an influential framework for generating hypotheses about men's responses to threats to masculine status. The present z-curve analysis of the experimental PMT literature indicates, however, that the published record shows substantial selection for statistical significance and only moderate expected replicability at α = 0.05 under the fitted model. The estimated false-discovery risk further indicates that the significant-results record leaves non-trivial room for false discoveries. Consistent with this pattern, the evidential value of published findings is uneven: results with p-values < 0.01 provide stronger evidential support within this selected record, whereas findings closer to the 0.05 threshold are more consistent with selection effects and inflated estimates.
Selection for significance further implies that reported effect-size estimates are likely to be upwardly biased. This concern extends to quantitative synthesis, because meta-analyses that rely primarily on the published record can inherit inflation when non-significant or ambiguous results are selectively missing. Reducing distortion in cumulative evidence requires more than increasing sample size: it also requires systematic and transparent reporting of all theoretically relevant outcomes, including null findings and unanticipated outcomes.
Improving cumulative inference in this literature will require (a) explicit sample-size justification and design choices that are aligned with the estimands under test, particularly for interaction hypotheses; (b) error-rate decisions rather than default reliance on α = 0.05; and (c) broader uptake of preregistration and Registered Reports to strengthen the severity of tests and reduce selective reporting and publication bias. At the level of publication incentives, well-designed studies with sufficiently large samples that yield null results should be treated as informative contributions that constrain plausible effect sizes, rather than as “non-results.”
Statements
Author contributions
VK: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Visualization, Writing – original draft, Writing – review & editing. OG: Methodology, Investigation, Writing – review & editing, Conceptualization, Writing – original draft.
Funding
The author(s) declared that financial support was received for this work and/or its publication. Open access funding by Swiss Federal Institute of Technology in Lausanne (EPFL).
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that Generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frsps.2026.1864827/full#supplementary-material
References
1
AczelB.SzasziB.ClellandH. T.KovacsM.HolzmeisterF.Van RavenzwaaijD.et al. (2026). Investigating the analytical robustness of the social and behavioural sciences. Nature652, 135–142. doi: 10.1038/s41586-025-09844-9
2
AdlerS. J.RöselerL.SchönigerM. K. (2023). A toolbox to evaluate the trustworthiness of published findings. J. Bus. Res.167:114189. doi: 10.1016/j.jbusres.2023.114189
3
BartošF.SchimmackU. (2022). Z-curve 2.0: estimating replication rates and discovery rates. Meta-Psychology 6:MP.2021.2720. doi: 10.15626/MP.2021.2720
4
BorensteinM.HedgesL. V.HigginsJ. P. T.RothsteinH. R. (2009). Introduction to Meta-analysis. Chichester: John Wiley and Sons. doi: 10.1002/9780470743386
5
BossonJ. K. (2025). Gender identity and aggression. Annu. Rev. Psychol.76, 635–661. doi: 10.1146/annurev-psych-020124-115456
6
BossonJ. K.JurekP.VandelloJ. A.Kosakowska-BerezeckaN.OlechM.BestaT.et al. (2021). Psychometric properties and correlates of precarious manhood beliefs in 62 nations. J. Cross Cult. Psychol.52, 231–258. doi: 10.1177/0022022121997997
7
BrunnerJ.SchimmackU. (2020). Estimating population mean power under conditions of heterogeneity and selection for significance. Meta-Psychology 4:MP.2018.874. doi: 10.15626/MP.2018.874
8
CarterE. C.SchönbrodtF. D.GervaisW. M.HilgardJ. (2019). Correcting for bias in psychology: a comparison of meta-analytic nethods. Adv. Methods Pract. Psychol. Sci.2, 115–144. doi: 10.1177/2515245919847196
9
FanelliD. (2010). “Positive” results increase down the hierarchy of the sciences. PLoS ONE5:e10068. doi: 10.1371/journal.pone.0010068
10
FremlingL.StrauelC.BognarE. (2026). Z-curve analysis of studies involving moderation published in leading health psychology journals. Health Psychol.45, 167–177. doi: 10.1037/hea0001534
11
GothreauC.HaasN. (2026). A replication and extension of Willer et al. (2013), overdoing gender: a test of the masculine overcompensation thesis. J. Exp. Polit. Sci.1–11. doi: 10.1017/XPS.2025.10027. [Epub ahead of print].
12
HongS.ReedW. R. (2021). Using Monte Carlo experiments to select meta-analytic estimators. Res. Synth. Methods12, 192–215. doi: 10.1002/jrsm.1467
13
KrivoshchekovV.GulevichO.BlagovI. (2023). Traditional masculinity and male violence against women: a meta-analytic examination. Psychol. Men Masculinities24, 346–364. doi: 10.1037/men0000426
14
LakensD. (2022). Sample size justification. Collabra Psychol.8:33267. doi: 10.1525/collabra.33267
15
LakensD. (2025). Concerns about replicability across two crises in social psychology. Int. Rev. Soc. Psychol.38:5. doi: 10.5334/irsp.1036
16
LakensD.CaldwellA. R. (2021). Simulation-based power analysis for factorial analysis of variance designs. Adv. Methods Pract. Psychol. Sci.4:251524592095150. doi: 10.1177/2515245920951503
17
LakensD.MesquidaC.RastiS.DitroiloM. (2024). The benefits of preregistration and registered reports. Evid. Based Toxicol.2:2376046. doi: 10.1080/2833373X.2024.2376046
18
LorenzL. L.HütherL.SteffensM. C.NiedlichC.WesnitzerH.KachelS. (2026). Masculinity threat in heterosexual men: a comprehensive meta-analysis of experimental research with recommendations for future theory building and research practice. Pers. Soc. Psychol. Rev. doi: 10.1177/10888683261433109. [Epub ahead of print].
19
LundbergI.JohnsonR.StewartB. M. (2021). What is your estimand? Defining the target quantity connects statistical evidence to theory. Am. Sociol. Rev.86, 532–565. doi: 10.1177/00031224211004187
20
MaierM.LakensD. (2022). Justify your alpha: a primer on two practical approaches. Adv. Methods Pract. Psychol. Sci. 5. doi: 10.1177/25152459221080396
21
McShaneB. B.BöckenholtU.HansenK. T. (2016). Adjusting for publication bias in meta-analysis: an evaluation of selection methods and some cautionary notes. Perspect. Psychol. Sci.11, 730–749. doi: 10.1177/1745691616662243
22
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science349:aac4716. doi: 10.1126/science.aac4716
23
PekJ.WuH.LiuY.DusenberyK. J.WegenerD. T. (2026). What does a Z-curve analysis tell us? Cogn. Emot. 1–16. doi: 10.1080/02699931.2026.2613139
24
RogalinC. L.AddisonS. M. (2025). Masculinities under the microscope: a systematic review of experimental methods. Psychol. Men Masculinities. doi: 10.1037/men0000537
25
RohrerJ. M. (2024). Causal inference for psychologists who think that causal inference is not for them. Soc. Personal. Psychol. Compass18:e12948. doi: 10.1111/spc3.12948
26
RöselerL. (2023). Predicting replication rates with z-curve: a brief exploratory validation study using the replication database. MetaArXiv. doi: 10.31222/osf.io/ewb2t
27
ScheelA. M.SchijenM. R. M. J.LakensD. (2021). An excess of positive results: comparing the standard psychology literature with registered reports. Adv. Methods Pract. Psychol. Sci.4, 1–12. doi: 10.1177/25152459211007467
28
SchimmackU. (2020). A meta-psychological perspective on the decade of replication failures in social psychology. Can. Psychol.61, 364–376. doi: 10.1037/cap0000246
29
SchimmackU.BartošF. (2023). Estimating the false discovery risk of (randomized) clinical trials in medical journals based on published p-values. PLoS ONE18:e0290084. doi: 10.1371/journal.pone.0290084
30
SchimmackU.SotoM. D. (2026a). A response to Pek et al.'s commentary on Z-curve: clarifying the assumptions of selection models. Cogn. Emot.1–5. doi: 10.1080/02699931.2026.2678998
31
SchimmackU.SotoM. D. (2026b). How credible are published significant results: estimating average true power with P-Curve or Z-curve?Commun. Methods Meas.20, 188–196. doi: 10.1080/19312458.2026.2658505
32
SiddawayA. P.WoodA. M.HedgesL. V. (2019). How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annu. Rev. Psychol.70, 747–770. doi: 10.1146/annurev-psych-010418-102803
33
SotoM. D.SchimmackU. (2025). Credibility of results in emotion science: a Z-curve analysis of results in the journals Cognition and Emotion and Emotion. Cogn. Emot.39, 1803–1819. doi: 10.1080/02699931.2024.2443016
34
SotolaL. K. (2023). How can I study from below, that which is above? Comparing replicability estimated by z-curve to real large-scale replication attempts. Meta-Psychology 7:MP.2022.3299. doi: 10.15626/MP.2022.3299
35
SotolaL. K.CredéM. (2022). On the predicted replicability of two decades of experimental research on system justification: a Z-curve analysis. Euro. J. Social Psychol. 52, 895–909. doi: 10.1002/ejsp.2858
36
SotolaL. K.CredéM. (2023). Estimating the replicability of statistically significant moderation effects in personality research using z-curve analysis. J. Res. Pers.107:104435. doi: 10.1016/j.jrp.2023.104435
37
StanalandA.GaitherS.Gassman-PinesA. (2023). When is masculinity “fragile”? An expectancy-discrepancy-threat model of masculine identity. Personality and Social Psychol. Rev.27, 359–377. Scopus. doi: 10.1177/10888683221141176
38
StanleyT. D. (2017). Limitations of PET-PEESE and other meta-analysis methods. Soc. Psychol. Personal. Sci.8, 581–591. doi: 10.1177/1948550617693062
39
TynerA. H.AbatayoA. L.DaleyM.FieldS.FoxN.HaberN. A.et al. (2026). Investigating the replicability of the social and behavioural sciences. Nature652, 143–150. doi: 10.1038/s41586-025-10078-y
40
van LaarC.HolmanT.GrootjansS. (2026). Men's self and identity and their relations to others in a changing world. Annu. Rev. Psychol.77, 1–15. doi: 10.1146/annurev-psych-013125-041821
41
VandelloJ. A.BossonJ. K. (2013). Hard won and easily lost: a review and synthesis of theory and research on precarious manhood. Psychol. Men Masculinity14, 101–113. doi: 10.1037/a0029826
42
VandelloJ. A.BossonJ. K.CohenD.BurnafordR. M.WeaverJ. R. (2008). Precarious manhood. J. Pers. Soc. Psychol.95, 1325–1339. doi: 10.1037/a0012453
43
WillerR.RogalinC. L.ConlonB.WojnowiczM. T. (2013). Overdoing gender: a test of the masculine overcompensation thesis. Am. J. Sociol.118, 980–1022. doi: 10.1086/668417
44
WongY. J.WangS.-Y. (2022). Toward an integrative psychology of masculinities. Psychol. Men Masculinities23, 285–298. doi: 10.1037/men0000387
45
WuW.DuanJ.ReedW. R.TiptonE. (2026). What can we learn from 1,000 meta-analyses across 10 different disciplines?Res. Synth. Methods17, 123–156. doi: 10.1017/rsm.2025.10035
46
ŻadkowskaM.SzlendakT.KossakowskiR.Kosakowska-BerezeckaN. (2025). What makes a man unmanly? The global concept of ‘unmanliness'. Human. Social Sci. Commun.12:348. doi: 10.1057/s41599-025-04667-5
Summary
Keywords
false positive risk, masculinity threat, meta-analysis, precarious manhood, Z-curve
Citation
Krivoshchekov V and Gulevich O (2026) How credible is the experimental evidence on precarious manhood? A z-curve analysis. Front. Soc. Psychol. 4:1864827. doi: 10.3389/frsps.2026.1864827
Received
24 April 2026
Revised
02 July 2026
Accepted
28 July 2026
Published
18 August 2026
Volume
4 - 2026
Edited by
Sarah J. Gervais, University of Nebraska-Lincoln, United States
Reviewed by
Marcel Van Assen, Tilburg University, Netherlands
Nathaniel Schermerhorn, University of Essex, United Kingdom
Updates
Copyright
© 2026 Krivoshchekov and Gulevich.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Vladislav Krivoshchekov, vladislav.krivoshchekov@epfl.ch
ORCID: Vladislav Krivoshchekov orcid.org/0000-0003-1213-0884; Olga Gulevich orcid.org/0000-0003-3806-5064
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.