Objective Measurement of Subjective Beliefs: Improving the Usefulness of Elicitation and Assessment Methods

  • 325

    Total views and downloads

Participate in this topic

Submit your idea

You will be redirected to our submission process.

Submit your manuscript summarySubmit your manuscript

About this Research Topic

Submission deadlines

  1. Manuscript Summary Submission Deadline 11 August 2025 | Manuscript Submission Deadline 29 November 2025

  2. This Research Topic is still accepting articles.

    1. Check author guidelines

Background

Subjective judgments are central to decision-making across various high-stakes domains, such as public policy, geopolitical forecasting, human-computer interaction, climate change, and healthcare. For example, decisions to implement economic policies and to launch clinical trials heavily rely on such judgments, where the accuracy of these beliefs can greatly influence outcomes. Thus, the informativeness of these judgments is paramount. Informativeness, in this context, refers to the extent to which judgments accurately reflect beliefs and/or the likelihood of all the different plausible outcomes. However, elicitation of such judgments is challenging because the elicitation mechanism is configured to match the specific task and/or domain, and there is no standard way to conduct an elicitation. While some fields benefit from regular validation of measures, such as personality and educational assessments, others face substantial validation gaps. This broad spectrum of approaches often leads to difficulty in evaluation and comparison, underscoring the need for validated measures of psychological beliefs and judgments tailored to specific outcomes.

This Research Topic aims to synthesize insights from diverse fields, including psychology, economics, policymaking, and decision science. The objective is to develop methods that are not only valid and reliable but also generalizable and actionable across different content domains. Valid methods are methods that elicit subjective judgments that accurately reflect a person’s underlying beliefs. Reliable methods elicit judgments that are internally and logically consistent and coherent. Generalizable methods are those that can be applied across a range of settings and content domains. Actionable methods are those that produce judgments that can be understood and utilized by stakeholders to make well-informed decisions.

By systematically testing and comparing existing and novel methodologies, the collection hopes to establish a clear linkage between elicitation techniques, assessments of informativeness, judgment accuracy, and grounded behavioral outcomes. For instance, to help a trader decide how much to invest in a risky option, what is the best way to capture and communicate outcome uncertainty? How do different methods affect the way judgments form, and do they increase or reduce biases like anchoring, ambiguity aversion, base rate neglect, or confirmation bias? Are the judgments internally coherent and logically consistent within and across methods, and how should their accuracy be evaluated once the outcome is known? This topic hopes to identify methods and measures that could improve judgment quality and support better decision-making in substantive areas of interest, including policy, program evaluation, health, climate, and more.

To advance judgment elicitation and assessment, this Research Topic will examine diverse methodological frameworks drawing on insights from judgment and decision-making (JDM) across behavioral science, cognitive and quantitative psychology, decision science, computational social science, economics, and political science. The collection welcomes contributions including, but not limited to, the following:

Evaluating Innovative Elicitation Methods
• Novel methodologies for judgment elicitation and effectiveness.
• Advances in existing elicitation methods, scoring, and assessment techniques to ensure reliability and reduce bias.
• Linking incentivized scoring rules to valid psychological measurement.
• Exploring underlying psychometric structure and measurement dimensionality.
• Identifying and addressing complex characteristics and/ confounds in judgment elicitation
• Measurement of second-order cognition about subjective judgments (i.e., meta-cognition, confidence, etc)

Real-world Applications
• Applying elicitation methods in diverse fields (e.g. epidemiology, climate change, and medical decision-making).
• Human biases in legal and managerial decisions.
• Individual differences in judgment and information evaluation.
• Trust in human and/ technological advice across domains.

Measurement in Computational Social Science, including Artificial Intelligence (AI) and Machine Learning (ML) Approaches
• Integrating Explainable AI (XAI) and its effect on decision-making
• Detection methods for hidden patterns and biases in large-scale data, including challenges like Simpson’s paradox and causal inference.
• Emerging disciplines and technologies related to human-computer interaction as well as the potential implications of technologies like AI in enhancing or complicating subjective judgments.
• Computational modelling of beliefs and cognitive biases (individual or collective decision making).
• AI-driven techniques for detecting systematic biases in judgment elicitation.

Topic Editor, Daniel Benjamin, and Topic Coordinator, Mark Himmelstein, receive financial support for research from the Forecasting Research Institute (FRI). The other Topic Editors report no competing interests related to this Research Topic.

Article types and fees

This Research Topic accepts the following article types, unless otherwise specified in the Research Topic description:

  • Brief Research Report
  • Case Report
  • Conceptual Analysis
  • Data Report
  • Editorial
  • General Commentary
  • Hypothesis and Theory
  • Methods
  • Mini Review

Articles that are accepted for publication by our external editors following rigorous peer review incur a publishing fee charged to Authors, institutions, or funders.

Article types

This Research Topic accepts the following article types, unless otherwise specified in the Research Topic description:

  • Brief Research Report
  • Case Report
  • Conceptual Analysis
  • Data Report
  • Editorial
  • General Commentary
  • Hypothesis and Theory
  • Methods
  • Mini Review
  • Opinion
  • Original Research
  • Perspective
  • Registered Report
  • Review
  • Systematic Review
  • Technology and Code

Keywords: decision making, elicitation, assessment, belief measurement, Judgment, Methods

Important note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.

Topic editors

Topic coordinators

Frequently asked questions

  • Frontiers' Research Topics are collaborative hubs built around an emerging theme.Defined, managed, and led by renowned researchers, they bring communities together around a shared area of interest to stimulate collaboration and innovation.

    Unlike section journals, which serve established specialty communities, Research Topics are pioneer hubs, responding to the evolving scientific landscape and catering to new communities.

  • The goal of Frontiers' publishing program is to empower research communities to actively steer the course of scientific publishing. Our program was implemented as a three-part unit with fixed field journals, flexible specialty sections, and dynamically emerging Research Topics, connecting communities of different sizes and maturity.

    Research Topics originate from the scientific community. Many of our Research Topics are suggested by existing editorial board members who have identified critical challenges or areas of interest in their field.

  • As an editor, Research Topics will help you build your journal, as well as your community, around emerging, cutting-edge research. As research trailblazers, Research Topics attract high-quality submissions from leading experts all over the world.

    A thriving Research Topic can potentially evolve into a new specialty section if there is sustained interest and a growing community around it.

  • Each Research Topic must be approved by the specialty chief editor, and it falls under the editorial oversight of our editorial boards, supported by our in-house research integrity team. The same standards and rigorous peer review processes apply to articles published as part of a Research Topic as for any other article we publish.

    In 2023, 80% of the Research Topics we published were edited or co-edited by our editorial board members, who are already familiar with their journal's scope, ethos, and publishing model. All other topics are guest edited by leaders in their field, each vetted and formally approved by the specialty chief editor.

  • Publishing your article within a Research Topic with other related articles increases its discoverability and visibility, which can lead to more views, downloads, and citations. Research Topics grow dynamically as more published articles are added, causing frequent revisiting, and further visibility.

    As Research Topics are multidisciplinary, they are cross-listed in several fields and section journals – increasing your reach even more and giving you the chance to expand your network and collaborate with researchers in different fields, all focusing on expanding knowledge around the same important topic.

    Our larger Research Topics are also converted into ebooks and receive social media promotion from our digital marketing team.

     

  • Frontiers offers multiple article types, but it will depend on the field and section journals in which the Research Topic will be featured. The available article types for a Research Topic will appear in the drop-down menu during the submission process.

    Check available article types here 

  • Yes, we would love to hear your ideas for a topic. Most of our Research Topics are community-led and suggested by researchers in the field. Our in-house editorial team will contact you to talk about your idea and whether you’d like to edit the topic. If you’re an early-stage researcher, we will offer you the opportunity to coordinate your topic, with the support of a senior researcher as the topic editor. 

    Suggest your topic here 

  • A team of guest editors (called topic editors) lead their Research Topic. This editorial team oversees the entire process, from the initial topic proposal to calls for participation, the peer review, and final publications.

    The team may also include topic coordinators, who help the topic editors send calls for participation, liaise with topic editors on abstracts, and support contributing authors. In some cases, they can also be assigned as reviewers.

  • As a topic editor (TE), you will take the lead on all editorial decisions for the Research Topic, starting with defining its scope. This allows you to curate research around a topic that interests you, bring together different perspectives from leading researchers across different fields and shape the future of your field. 

     

    You will choose your team of co-editors, curate a list of potential authors, send calls for participation and oversee the peer review process, accepting or recommending rejection for each manuscript submitted.

  • As a topic editor, you're supported at every stage by our in-house team. You will be assigned a single point of contact to help you on both editorial and technical matters. Your topic is managed through our user-friendly online platform, and the peer review process is supported by our industry-first AI review assistant (AIRA).

  • If you’re an early-stage researcher, we will offer you the opportunity to coordinate your topic, with the support of a senior researcher as the topic editor. This provides you with valuable editorial experience, improving your ability to critically evaluate research articles and enhancing your understanding of the quality standards and requirements for scientific publishing, as well as the opportunity to discover new research in your field, and expand your professional network.

  • Yes, certificates can be issued on request. We are happy to provide a certificate for your contribution to editing a successful Research Topic.

  • Research Topics thrive on collaboration and their multi-disciplinary approach around emerging, cutting-edge themes, attract leading researchers from all over the world.

  • As a topic editor, you can set the timeline for your Research Topic, and we will work with you at your pace. Typically, Research Topics are online and open for submissions within a few weeks and remain open for participation for 6 – 12 months. Individual articles within a Research Topic are published as soon as they are ready.

    Find out more about our Research Topics

  • Our fee support program ensures that all articles that pass peer review, including those published in Research Topics, can benefit from open access – regardless of the author's field or funding situation.

    Authors and institutions with insufficient funding can apply for a discount on their publishing fees. A fee support application form is available on our website.

  • In line with our mission to promote healthy lives on a healthy planet, we do not provide printed materials. All our articles and ebooks are available under a CC-BY license, so you can share and print copies.

     

Manuscripts can be submitted to this Research Topic via the main journal or any other participating journal.

Submit your idea

You will be redirected to our submission process.

Submit your manuscript summarySubmit your manuscript

Impact

  • 325Topic views
View impact