Interactive Digital Twins through Multimodal AI

Purpose and goal

We combine high-end research with new products and services to create a multimodal AI system: an interactive Digital Twin which allow user interaction with complex outdoor scenarios and events. This project advances the concept av Digital Twins by merging Swedish industrial partners, leading research, and stakeholder needs into an interactive system, featuring 3D models, human reconstructions, voice-controlled language models, and accuracy assessments. We demonstrate high level capability, combining partners’ contributions with added value, and expect win-win situation.

Expected effects and result

1) Strengthen cooperation among partners with complementary and advanced AI competencies. 2) Contribute to growth in Swedish industries, such as new services and products for Swedish authorities, international markets, and startups. 3) Identify new research problems with high societal relevance, using Visual Crime Scene as a platform for further Research & Development. 4) Visualising AI´s effects for decision-makers, such as the judiciary and the public. 5) Enhance collaboration between regional authorities and international actors.

Planned approach and implementation

We have assigned the responsibilities and divided timelines across Work Packages (WP) 1-5: WP1 (2024-09-012026-03-31): Project Management, Santa Anna. WP2 (2024-10-01 to 2024-12-31): Design and System Setup, led by Intel, assisted by Lutra. WP3 (2024-09-01 to 2025-09-30): WASP WARA Research Arena Validation, led by CVL/LiU, assisted by Polisen and CVL. WP4 (2024-10-01 to 2026-01-31): Integration, Tests, and Systems Verification led by Lutra, assisted by Intel. WP5 (2025-11-01 to 2026-03-31): Exploiting and Showcasing the Testbed, Santa Anna. Other actors are also assigned.