www.ptreview.co.uk
15
'26
Written on Modified on
NEC and Keio University Develop Rapid AI-Driven 3D Modeling Technology
NEC Corporation has developed an AI-based system to generate detailed 3D models in one minute from standard smartphone footage by optimizing point density.
www.nec.com
NEC Corporation, in collaboration with the Keio AI Research Center, has developed a technology that uses proprietary artificial intelligence to generate highly detailed 3D models in as little as one minute. The system operates solely on video footage captured with general-purpose cameras, such as those integrated into smartphones, and automatically removes transient or unnecessary subjects from the final render. The technology is engineered to precisely replicate on-site conditions without requiring expensive specialized hardware or disrupting active work environments. NEC plans to commercialize the technology within fiscal year 2027, targeting digital twin applications across the infrastructure, utilities, and construction industries.
Overcoming Video Constraints and Spatial Modeling Hurdles
Modern infrastructure operators and construction companies increasingly share live video feeds with remote supervisors to manage labor shortages and minimize inspection travel expenses. However, traditional video recording makes it difficult for remote teams to quickly isolate specific viewpoints or dynamically adjust angles to inspect obscure details.
While digital twins built on 3D models offer free-viewpoint inspection, broad adoption has been constrained by technical hurdles. Standard photogrammetry and laser scanning methods require expensive specialized sensors, necessitate halting on-site operations during filming to avoid capturing transient workers, and demand extensive off-site processing times to render the final model.
To resolve these operational bottlenecks, the collaborative project paired Gaussian Splatting—a spatial rendering technique increasingly used for background generation in film and animation—with custom neural network processing. The combination permits rapid, high-fidelity spatial reconstruction using mobile device cameras without halting ongoing operations.
Spatial Optimization and Object-Removal Features
The technology incorporates two key software innovations to optimize processing efficiency and ensure model accuracy:
- Visual Complexity Analysis & Adaptive Particle Distribution: The system automatically evaluates the geometric and visual complexity of the source video frame-by-frame. In complex or highly textured areas, the software densely clusters the 3D Gaussian particles. In simple or uniform regions, such as flat walls and floors, the particle distribution is significantly thinned out. This adaptive optimization maintains full visual detail while minimizing the aggregate particle count, reducing computational overhead and cutting model generation times by 90% compared to conventional Gaussian Splatting methods.
- Transient Subject Removal & Background Inpainting: During the 3D reconstruction process, the AI automatically detects and filters out temporary subjects—such as moving personnel, vehicles, and transient materials. To prevent voids in the completed 3D scene, the algorithm infers and inpaints the missing background structures based on surrounding spatial data. This produces a static, unobstructed 3D model representing the permanent facility layout.
The resulting 3D models are compatible with ordinary computers or tablets. This allows field engineers and remote coordinators to immediately assess site conditions, conduct virtual inspections, and accelerate decision-making during operational anomalies.
Additional Context
This section details technical specifications not included in the original news release.
Traditional 3D reconstruction pipelines rely on Structure-from-Motion (SfM) and Multi-View Stereo (MVS) to build dense polygon meshes, which requires substantial processing time. Gaussian Splatting accelerates this process by representing 3D space as a collection of continuous, semi-transparent 3D ellipsoids (Gaussians) rather than rigid, discrete polygonal points. Each Gaussian particle is mathematically defined by its spatial position, scale, rotation, color, and opacity.
The NEC and Keio University platform refines this technique by integrating an active visual-complexity analyzer before the optimization step. Conventional splatting evenly distributes particles across a scene and relies on a slow pruning-and-splitting algorithm to refine details.
In contrast, this technology employs a real-time edge-detection and texture-gradient mapping algorithm to estimate localized spatial frequency. The system uses these density maps to allocate the initial Gaussian seeds. This pre-optimized placement allows the neural engine to bypass redundant particle iterations on flat surfaces. It concentrates GPU processing power solely on complex surfaces, compressing the entire rendering sequence into a one-minute operational envelope.
Edited by Romila DSilva, Induportals Editor, with AI assistance.
Additional Context
This section details technical specifications not included in the original news release.
Traditional 3D reconstruction pipelines rely on Structure-from-Motion (SfM) and Multi-View Stereo (MVS) to build dense polygon meshes, which requires substantial processing time. Gaussian Splatting accelerates this process by representing 3D space as a collection of continuous, semi-transparent 3D ellipsoids (Gaussians) rather than rigid, discrete polygonal points. Each Gaussian particle is mathematically defined by its spatial position, scale, rotation, color, and opacity.
The NEC and Keio University platform refines this technique by integrating an active visual-complexity analyzer before the optimization step. Conventional splatting evenly distributes particles across a scene and relies on a slow pruning-and-splitting algorithm to refine details.
In contrast, this technology employs a real-time edge-detection and texture-gradient mapping algorithm to estimate localized spatial frequency. The system uses these density maps to allocate the initial Gaussian seeds. This pre-optimized placement allows the neural engine to bypass redundant particle iterations on flat surfaces. It concentrates GPU processing power solely on complex surfaces, compressing the entire rendering sequence into a one-minute operational envelope.
Edited by Romila DSilva, Induportals Editor, with AI assistance.
