Probabilistic Visualization of Complex Arguments to Resolve Analytic Disagreements

Organization
Office of the Director of National Intelligence (ODNI)
Reference Code
ICPD-2022-08
How to Apply

Create and release your Profile on Zintellect – Postdoctoral applicants must create an account and complete a profile in the on-line application system.  Please note: your resume/CV may not exceed 2 pages.

Complete your application – Enter the rest of the information required for the IC Postdoc Program Research Opportunity. The application itself contains detailed instructions for each one of these components: availability, citizenship, transcripts, dissertation abstract, publication and presentation plan, and information about your Research Advisor co-applicant.

Additional information about the IC Postdoctoral Research Fellowship Program is available on the program website located at: https://orise.orau.gov/icpostdoc/index.html.

If you have questions, send an email to ICPostdoc@orau.org.  Please include the reference code for this opportunity in your email. 

Application Deadline
2/28/2022 6:00:00 PM Eastern Time Zone
Description

Research Topic Description, including Problem Statement:

Individuals and organizations often disagree in good faith. These disagreements can arise from different evidence, different probabilities assigned to that evidence, different evaluations of the importance of that evidence in the reasoning, and even in the methods for reasoning. Previous research has shown that visualizing (mapping) an argument can deepen understanding of the issues and help people who disagree reach a correct common understanding. But that research has focused on relatively simple arguments about hypothetical situations in which the subjects are not personally vested intellectually or emotionally. This research will explore ways to use similar approaches to complex, real-life reasoning, where subjects often are personally vested.

Research should include developing collaborative, easy-to-use software and techniques to visualize the logical structure of complex real-life arguments. Despite the arguments’ complexity, the visualizations should be intuitive enough that people will understand them relatively easily and will find them helpful, even illuminating. Visualizations that only a highly trained person can understand are of little interest to this program. To achieve the requisite simplicity, the techniques should help people focus on key contested assumptions, explicit or implicit. The techniques for producing the visualizations may be automated. If not, then the process should be intuitive and easy to learn.

Since many disagreements come from differing estimates of the initial conditions (for example, different prior probabilities), the visualizations should make clear the (subjective) probabilities of key claims and of likelihoods/likelihood ratios, and any underlying arguments for those probabilities.

Example Approaches:

  • Develop easy-to-use argument visualization collaborative software that focuses on clarifying points of disagreement.
  • Develop easy-to-use collaborative software that encourages discussion and resolution of different likelihood ratios.
  • Bayesian approaches to argument mapping.
  • Develop techniques to identify evidence with the greatest discriminating power between the hypotheses.

Relevance to the Intelligence Community:

Analyzing and resolving disagreements among IC analysts or Government agencies.

  • For example, an analyst may assign greater weight to intelligence collected by their home agency than to other types of intelligence.

Analyzing and resolving disagreements where one or both reports are obtained from open-source reporting.

  • For example, the open-source report is likely to lack evidence available to the IC. Conversely, the open-source report may rely on evidence that the IC analyst is not aware of or has weighted differently. Understanding the importance of this evidence is worthwhile.
  • The open-source argument may serve as a contrary opinion for the IC analyst to consider and rebut.

Probabilistic visualization of the analyst’s argument can prompt analysts to explore elements that are not sufficiently supported and to consider alternatives, even alternative conclusions.

References:

  1. Mercier, H. & D. Sperber (2017). The enigma of reason. Harvard University Press.
  2. Cullen, S., Fan, J., van der Brugge, E. et al. Improving analytical reasoning and argument understanding: a quasi-experimental field study of argument visualization. npj Science Learn 3, 21 (2018). https://doi.org/10.1038/s41539-018-0038-5.
  3. Twardy, C. Argument maps improve critical thinking. In Teaching Philosophy 27 (2):95-116 (2004).
  4. Davies, M., Barnett, A., and van Gelder, T, Using Computer-Assisted Argument Mapping to Teach Reasoning to Students, in J. Anthony Blair (ed.), Studies in Critical Thinking (2nd Edition), Windsor, ON, Canada: Windsor Studies in Argumentation. pp 115-121 (2021).
  5. Saletta M, Kruger A, Primoratz T, Barnett A, van Gelder T, et al. (2020) The role of narrative in collaborative reasoning and intelligence analysis: A case study. PLOS ONE 15(1): e0226981. https://doi.org/10.1371/journal.pone.0226981

Key Words: Argument Mapping, Bayes Mapping, Dialogue Mapping, Argument Visualization, Enthymeme, Pragmatics of Reasoning, Tacit Knowledge, Human-Computer Interaction, Collaborative Software

Qualifications

Postdoc Eligibility

  • U.S. citizens only
  • Ph.D. in a relevant field must be completed before beginning the appointment and within five years of the application deadline
  • Proposal must be associated with an accredited U.S. university, college, or U.S. government laboratory
  • Eligible candidates may only receive one award from the IC Postdoctoral Research Fellowship Program

Research Advisor Eligibility

  • Must be an employee of an accredited U.S. university, college or U.S. government laboratory
  • Are not required to be U.S. citizens
Eligibility Requirements
  • Citizenship: U.S. Citizen Only
  • Degree: Doctoral Degree.
  • Discipline(s):
    • Communications and Graphics Design (2 )
    • Computer, Information, and Data Sciences (16 )
    • Earth and Geosciences (21 )
    • Engineering (27 )
    • Environmental and Marine Sciences (14 )
    • Life Health and Medical Sciences (46 )
    • Mathematics and Statistics (10 )
    • Other Non-S&E (2 )
    • Other Physical Sciences (12 )
    • Other S&E-Related (1 )
    • Physics (16 )
    • Social and Behavioral Sciences (27 )