Multi-Camera Full-Moton Video 3D Scene Reconstruction and Understanding using Edge Computing Accelerators Fellowship

Organization
Office of the Director of National Intelligence (ODNI)
Reference Code
ICPD-2024-17
How to Apply

Create and release your Profile on Zintellect – Postdoctoral applicants must create an account and complete a profile in the on-line application system.  Please note: your resume/CV may not exceed 3 pages.

Complete your application – Enter the rest of the information required for the IC Postdoc Program Research Opportunity. The application itself contains detailed instructions for each one of these components: availability, citizenship, transcripts, dissertation abstract, publication and presentation plan, and information about your Research Advisor co-applicant.

Additional information about the IC Postdoctoral Research Fellowship Program is available on the program website located at: https://orise.orau.gov/icpostdoc/index.html.

If you have questions, send an email to ICPostdoc@orau.org.  Please include the reference code for this opportunity in your email. 

Application Deadline
2/28/2024 6:00:00 PM Eastern Time Zone
Description

Research Topic Description, including Problem Statement:

Problem Statement: Given real-time Full-Motion Video data from multiple cameras with different view angles covering 360° view of the scene, solve the problem of multi-camera video fusion, 3D scene reconstruction or rendering, and 3D scene understanding to gain situational awareness of the people, objects and their relative motion using the edge computing AI hardware accelerators.


Topic Description: Automated analysis of Full-Motion Video (FMV) in real-time or near-real time is critical to make command-and-control decisions in multiple domains of intelligence. By using a distribution of cameras, for example body-worn cameras, ground-based autonomous vehicle cameras, security cameras, etc., a 360° view can be achieved for faster and more informed decisions. To process the vast amount of real-time video data from multiple cameras, a collection of efficient pre-processing algorithms is required for image registration and scan matching from multiple views angles. Further, to gain actionable intelligence, near real-time 3D rendering of the scenes, and the analysis of the reconstructed 3D scenes using computer vision algorithms for object detection, object tracking, localization, path planning, and scene understanding is necessary. It is also important to consider the power requirements of the AI hardware accelerators for deploying the algorithms as edge computing application on autonomous vehicles or body-worn cameras. For a user-friendly deployment on-field, it is also desirable to integrate voice or prompt-based question and answering for FMV understanding and situational awareness of dynamic agents.

Example Approaches: 

Neural Radiance Fields (NeRF) for novel views of complex 3D scenes based on partial set of 2D images, Hybrid Convolutional Neural Network (CNN)- Recurrent Neural Network (RNN) models, State Space Models for Time-Series, Mixture Density Network for estimating 6 DoF pose, Attention-based Scan Matching, Large Language Models (LLMs), and ChatGPT.

Relevance to the Intelligence Community: 

Develop/Enhance Automated Capabilities for In-Scene Analysis of Full Motion Video.

Key Words: Video Understanding, Multi-Camera Fusion, Time-Series Data Preprocessing, 3D Scene Reconstruction, Computer Vision, Edge AI Accelerator, Video Question - Answering, Large Language Models, Prompt Engineering.

Qualifications

Postdoc Eligibility

  • U.S. citizens only
  • Ph.D. in a relevant field must be completed before beginning the appointment and within five years of the appointment start date
  • Proposal must be associated with an accredited U.S. university, college, or U.S. government laboratory
  • Eligible candidates may only receive one award from the IC Postdoctoral Research Fellowship Program

Research Advisor Eligibility

  • Must be an employee of an accredited U.S. university, college or U.S. government laboratory
  • Are not required to be U.S. citizens
Eligibility Requirements
  • Citizenship: U.S. Citizen Only
  • Degree: Doctoral Degree.
  • Discipline(s):
    • Chemistry and Materials Sciences (12 )
    • Communications and Graphics Design (4 )
    • Computer, Information, and Data Sciences (17 )
    • Earth and Geosciences (21 )
    • Engineering (27 )
    • Environmental and Marine Sciences (14 )
    • Life Health and Medical Sciences (45 )
    • Mathematics and Statistics (11 )
    • Other Non-Science & Engineering (2 )
    • Physics (16 )
    • Science & Engineering-related (1 )
    • Social and Behavioral Sciences (30 )