Stereo Image to 3D Mesh Renderer

This is a Python-based tool that takes two simultaneous photos (Stereo) and calculates the 3D structure of the scene. The goal was to move beyond simple images and create a solid 3D object that can be opened, measured, and inspected in professional software like Blender or Windows 3D Viewer.

Overview

Below is the input images compared to the output:

Key Features

• Stereo Image Processing: Uses OpenCV to compute depth maps from stereo image pairs.
• Point Cloud Generation: Converts depth maps into 3D point clouds by creating millions of (x, y, z) coordinate point, each with its own colour from the original photo.
• Mesh Generation: Utilises Poisson Surface Reconstruction to create a mesh from the point cloud, which can be exported as a .ply file.
• Object Usage: The resulting mesh can be imported into 3D software for further editing, measurement, or visualisation.

The final output is in a standard 3D format that can be easily used in various applications.

Usage

1. Ensure you have uv installed to create a virtual environment.
2. Install the required dependencies using:

uv sync

3. To run the full reconstruction process, use:

uv run main.py --view

To process a custom dataset and save the output, use:

uv run main.py --data data/raw/your-dataset --out data/processed/your-dataset --view

To visualise an existing mesh without processing new images, use:

uv run main.py --skip-run --view

Dependencies

• uv: For creating a virtual environment
• numpy
• opencv-python
• open3d

Practical Applications

• 3D Scanning: This tool can be used to create 3D models of real-world objects, which can be useful for 3D printing, virtual reality, or digital archiving.
• Cultural Heritage: Museums and archaeologists can use this to create digital replicas of artifacts, allowing for preservation and remote access.

Future Improvements

• Multi-Angle View - Currently, the tool generates a mesh from a single stereo pair, which provides a limited view of the object. Future work could involve capturing multiple stereo pairs from different angles to create a more complete 3D model.
• Enhanced Mesh Quality: Implementing more advanced mesh generation techniques to improve the quality and detail of the resulting mesh, as it currently has some noise.
• GPU Acceleration: Leveraging GPU processing to speed up the depth map computation and mesh generation, which can be time-consuming for high-resolution images.
• Integrate with Geographic Information Systems (GIS): Adding functionality to georeference the generated meshes for use in GIS applications, which would be beneficial for multiple fields, including ecology, archaeology, or architecture.