Assignment 3: Neural Volume Rendering and Surface Rendering

Prerequisites

Before starting, ensure you have the required materials dataset downloaded. Follow these steps:

sudo apt install git-lfs
git lfs install

git clone https://huggingface.co/datasets/learning3dvision/nerf_materials
cd nerf_materials
unzip materials.zip -d <path_to_your_data_folder>

---

A. Neural Volume Rendering

1. Differentiable Volume Rendering

1.3 Ray Sampling

1.4 Point Sampling

1.5 Volume Rendering

2. Optimizing a Basic Implicit Volume

2.1 Random Ray Sampling

• Box Center: (0.25, 0.25, -0.00)
• Box Side Lengths: (2.00, 1.50, 1.50)

3. Optimizing a Neural Radiance Field (NeRF)

4. NeRF Extras

4.1 View Dependence

• Lego Dataset
  

• Materials Dataset
  

Increased View Dependence vs. Generalization

Increased view dependence captures more detail and incorporates lighting effects, such as highlights on reflective materials. However, it may reduce the model's ability to generalize to unseen viewpoints due to overfitting.

---

B. Neural Surface Rendering

5. Sphere Tracing

Sphere tracing iteratively samples points along each ray to find surface intersections. A threshold of 0.001 (eps) units is used to approximate the intersection point. A mask (mask) tracks intersections efficiently using boolean operations, avoiding computationally expensive iterations.

6. Optimizing a Neural SDF

• Input Point Cloud
  

• Output Geometry
  

7. VolSDF

Alpha and Beta Parameters

• Alpha: Controls structure and opacity through density. Higher alpha results in well-defined edges and higher saturation, while lower alpha leads to diffused appearances.
• Beta: Controls edge sharpness and smoothness. Higher beta produces smoother edges, while lower beta captures finer details.

Visualizations

• Default Values (Beta = 0.05):
  

• High Beta (Beta = 0.3):
  

• Low Beta (Beta = 0.03):
  

Key Insights

• High Beta: Easier to train due to faster convergence but results in diffused surfaces.
• Low Beta: Captures finer details and produces more accurate surfaces.

Best Output (Alpha = 20, Beta = 0.05, n_layers_distance = 8, n_hidden_neurons_distance = 256)

8. Neural Surface Extras

8.2 Fewer Training Views

• VolSDF Output (20 Training Views):
  

• NeRF Output (20 Training Views):
  

Observation: VolSDF captures more detail and sharpness compared to NeRF, as it models volume density using a signed distance function, making it more robust in few-view conditions.