Improved documentation and usability for new contributors

DeepLense_Physics_Informed_Super_Resolution_Anirudh_Shankar/README.md

@@ -21,21 +21,21 @@ Find below the list of contents of this description:
 11. [Appendix](#11-appendix)
 
 ## 1. Replication and set-up
-The Git repository can be accessed from here, as part of the parent ML4Sci repository.
+This project lives inside the parent ML4Sci DeepLense repository.
 
 It contains all the Python Notebooks used in training and testing, the trained model weights, dataset simulation scripts, set-up instructions and some examples.
 
-Requirements are divided into two, for each of the tasks:
+The dependencies are split into two groups based on the workflow:
 
-**1. Image simulation**: Use the following in an environment of your choice to install the libraries required for simulating the datasets: 
+**1. Image simulation**: Use the following command in your environment to install the libraries required for simulating datasets:
 
-`pip install simulations.txt` 
+`pip install -r simulations.txt`
 
 The Simulations directory contains the code used to create the dataset, and is adopted from [Michael Toomey’s work](https://github.com/mwt5345/DeepLenseSim/tree/main/Model_I), one of my project mentors.
 
-**2. Super-resolution and related tasks (basically, everything else)**: Use the following in an environment of your choice to install the libraries required for everything else presented in this project:
+**2. Super-resolution and related tasks (everything else)**: Use the following command to install the libraries required for the training and evaluation notebooks:
 
-`pip install requirements.txt`
+`pip install -r requirements.txt`
 
 ## 2. What is gravitational lensing ?
 
@@ -77,7 +77,7 @@ Training is done on low-resolution lensing images alone, as described in the [fo
 ## 4. Datasets
 
 Three datasets are used, each of which mimics a telescope:
-* Model 1 dataset: mimics an artifically constructed telescope
+* Model 1 dataset: mimics an artificially constructed telescope
 * Model 2 dataset: mimics Euclid
 * Model 3 dataset: mimics the Hubble Space Telescope
 
@@ -100,7 +100,7 @@ The model is trained in an unsupervised fashion using a multi-faceted physics-ba
   * Sub-low-resolution scale (0.5x): At dimensions that are further lower than the low-resolution input images
 * Mean squared error (MSE) between the Sérsic profile and the reconstructed source image
 * MSE between the interpolated images and the images produced by the model
-* Intensity constraints between the Sérsic source and the upscaled lensing images. This is elabored on in the [Appendix](#11-appendix)
+* Intensity constraints between the Sérsic source and the upscaled lensing images. This is elaborated on in the [Appendix](#11-appendix)
 * The deflection angle is ensured to be > 0, as this is a physical constraint on the system owing to the non-negativity of the mass distribution
 * A variation density loss (VDL) that restricts the local variability of the deflection angle. This loss ensures that the produced deflection angles remain physical, and without any artifacts or aphysical fluctuations. More in the [Appendix](#11-appendix).
 
@@ -122,7 +122,7 @@ Models are trained on the training dataset described in [Section 4](#4-datasets)
 
 Performance of the models are evaluated using the following metrics:
 * **MSE**: The MSE between the SR and true HR images acts as a simple estimate of closeness of individual pixels in both images.
-* **Structural Similarity Index Measure (SSIM)**: This metric also evalutates how different the relation between pixels and their neighbours are, in the SR and true HR images.
+* **Structural Similarity Index Measure (SSIM)**: This metric also evaluates how different the relationships between neighbouring pixels are in the SR and true HR images.
 * **Peak Signal-to-Noise Ratio (PSNR)**: This metric acts as a measure of the amount of noise present in the image. A higher value corresponds to better image quality (lesser noise).
 
 Model 1:
@@ -147,7 +147,7 @@ Model 3:
 | CDM (sub-halos) | 0.001684 | 0.809 | 28.636 |
 
 ### Performance on the model-2 dataset
-As seen above, the performance of the model on the model-2 dataset (limited to the SSIM) appears to be significantly worse. Model-2, mimicing Euclid, has a much lower PSF (point spread function) when compared to the other models. This causes broader and more blurred local features in the images, with which both the model. The worse performance can thus be associated for the most part with the quality issues imposed by the model-2 PSF.
+As seen above, the performance of the model on the Model 2 dataset, especially in terms of SSIM, appears to be significantly worse. Model 2, which mimics Euclid, has a much lower PSF (point spread function) than the other models. This leads to broader and blurrier local features in the images, making reconstruction harder for the model. Most of the performance gap can therefore be attributed to the image-quality limitations imposed by the Model 2 PSF.
 
 ## 7. Auxiliary studies
 
@@ -185,15 +185,15 @@ This direction tests the performance of the model when trained with a much small
 
 ![Performance with sparsity](Readme/sparse.jpg)
 
-The model trained on fewer samples again, show at most a 10% degradation in the SSIM and PSNR scores. Again, this could imply minute structural improvements, but overall similariy in performance.
+The model trained on fewer samples again shows at most a 10% degradation in the SSIM and PSNR scores. This suggests small structural differences, but broadly similar overall performance.
 
 This result also falls in line with several PINN studies, suggesting that a strength of PINNs is their ability to function effectively with sparse datasets.
 
 ### Quality verification
 
-This final direction aims at ensuring the quality of images produed by the SR architecture proposed. While the SSIM and PSNR scores can ensure perceptual quality, their true quality that one can see directly, must also be ensured.
+This final direction aims to ensure the quality of the images produced by the proposed SR architecture. While SSIM and PSNR help quantify perceptual quality, the visual and downstream usefulness of the outputs must also be checked directly.
 
-For this purpose, a downstream classification into the three DM sub-structure classes is performed by similar classification networks on the two sets of images: the LR images, and the SR architecture's outputs. There was initially a 10% worse perofrmance by the SR model in downsteam classification accuracy, and small adjustments to the architecture was made to preserve SR image quality.
+For this purpose, a downstream classification into the three DM sub-structure classes is performed using similar classification networks on two sets of images: the LR images and the SR model outputs. There was initially a roughly 10% drop in downstream classification accuracy for the SR outputs, so small adjustments were made to the architecture to preserve SR image quality.
 
 ![DC model adjustments](Readme/Alpha_2.png)
 
@@ -385,4 +385,4 @@ Model 3:
 |-----------------|----------|-------|--------|
 | No_substructure | 0.004156 | 0.786 | 25.519 |
 |  Axion (vortex) | 0.003884 | 0.790 | 26.537 |
-| CDM (sub-halos) | 0.003883 | 0.786 | 25.986 |
+| CDM (sub-halos) | 0.003883 | 0.786 | 25.986 |

DeepLense_Physics_Informed_Super_Resolution_Anirudh_Shankar/vanilla/Evaluation.ipynb

@@ -1,5 +1,14 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Physics-Informed Super-Resolution Evaluation\n",
+    "\n",
+    "This notebook evaluates the vanilla physics-informed super-resolution model on the DeepLense test split. It loads a trained model, reconstructs higher-resolution lensing images, and reports image-quality metrics for each dark matter substructure class.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -27,13 +36,22 @@
     "print(device)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Load Configuration\n",
+    "\n",
+    "This section imports the experiment settings and selects which trained model checkpoint to evaluate. The `model_number` controls the telescope-style dataset variant used during testing.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 2,
    "metadata": {},
    "outputs": [],
    "source": [
-    "with open('config.json') as config_file:\n",
+    "with open('../config.json') as config_file:\n",
     "    config = json.load(config_file)\n",
     "\n",
     "model_number = 1"
@@ -82,6 +100,15 @@
     "        return torch.sum(diff_x)/(diff_x.shape[2]*diff_x.shape[3]) + torch.sum(diff_y)/(diff_y.shape[2]*diff_y.shape[3])"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Prepare Evaluation Datasets\n",
+    "\n",
+    "The notebook builds test datasets for the three substructure categories and their corresponding high-resolution targets. These pairs are used to compare the model output against the reference images.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 5,
@@ -97,6 +124,15 @@
     "HR_cdm = data.LensingDataset('../Simulations/test/data_model_%d/'%model_number,['cdm_HR'],5000)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Load the Trained Model\n",
+    "\n",
+    "The super-resolution model predicts deflection-angle information that is then used to reconstruct sharper lensing images. Here we restore the saved weights for the selected dataset configuration.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 6,
@@ -119,6 +155,15 @@
     "alpha_model.load_state_dict(torch.load('Weights_%d.pt'%(model_number), weights_only=True))"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Visualize a Sample Reconstruction\n",
+    "\n",
+    "Before aggregating metrics across the full test set, the notebook defines a helper to inspect one reconstructed example. This makes it easier to confirm that the model output looks reasonable.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 7,
@@ -239,6 +284,19 @@
     "give_image(dataset_cdm, HR_cdm, alpha_model, len(dataset_cdm), plot=True)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Define Evaluation Metrics\n",
+    "\n",
+    "The reported metrics capture complementary aspects of image quality:\n",
+    "\n",
+    "- `MSE` measures average pixel-level error.\n",
+    "- `SSIM` measures perceptual and structural similarity.\n",
+    "- `PSNR` summarizes reconstruction quality relative to noise.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 9,
@@ -265,6 +323,15 @@
     "history_cdm = {'loss':[],'SSIM':[], 'PSNR':[], 'vdl':[]}"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Run Evaluation Across All Classes\n",
+    "\n",
+    "The next cells iterate through the test samples, compute reconstruction metrics, and store per-class histories for later reporting.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 10,
@@ -274,9 +341,9 @@
      "name": "stderr",
      "output_type": "stream",
      "text": [
-      "Evaluating CDM sub-structure images: 100%|██████████| 5000/5000 [00:35<00:00, 141.76it/s]\n",
-      "Evaluating no sub-structure images: 100%|██████████| 5000/5000 [00:32<00:00, 154.52it/s]\n",
-      "Evaluating axion DM sub-structure images: 100%|██████████| 5000/5000 [00:33<00:00, 148.73it/s]\n"
+      "Evaluating CDM sub-structure images: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5000/5000 [00:35<00:00, 141.76it/s]\n",
+      "Evaluating no sub-structure images: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5000/5000 [00:32<00:00, 154.52it/s]\n",
+      "Evaluating axion DM sub-structure images: 100%|\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5000/5000 [00:33<00:00, 148.73it/s]\n"
      ]
     }
    ],
@@ -339,6 +406,15 @@
     "print(f\"cdm: Evaluation completed with \\nMSE: {np.mean(history_cdm['loss'])} ({np.std(history_cdm['loss'])})\\nSSIM: {np.mean(history_cdm['SSIM'])} ({np.std(history_cdm['SSIM'])})\\nPSNR:  {np.mean(history_cdm['PSNR'])} ({np.std(history_cdm['PSNR'])})\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Summarize Results\n",
+    "\n",
+    "The final table presents the average scores for each dark matter substructure class so the model can be compared across datasets.\n"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 12,
@@ -438,4 +514,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 2
-}
+}

DeepLense_Physics_Informed_Super_Resolution_Anirudh_Shankar/vanilla/Train_3.ipynb

@@ -39,7 +39,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "with open('config.json') as config_file:\n",
+    "with open('../config.json') as config_file:\n",
     "    config = json.load(config_file)"
    ]
   },

DeepLense_Physics_Informed_Super_Resolution_Anirudh_Shankar/vanilla/data.py

@@ -1,5 +1,6 @@
 import torch
 import numpy as np
+from pathlib import Path
 
 class LensingDataset(torch.utils.data.Dataset):
     def __init__(self, directory, classes, num_samples, aux='_sim_'):
@@ -12,10 +13,11 @@ def __init__(self, directory, classes, num_samples, aux='_sim_'):
         :param aux: Used to indicate whether the dataset contains image sumulations ('_sim_') or deflection angles ('_alpha_')
         """
         super(LensingDataset, self).__init__()
-        self.directory = directory
+        self.directory = Path(directory)
         self.classes = classes
         self.num_samples = num_samples
         self.aux = aux
+
     def __len__(self):
         """
         :return: Returns the length of the dataset
@@ -30,8 +32,23 @@ def __getitem__(self, index):
         :return: LR image, min-max normalized
         """
         selected_class = self.classes[index//self.num_samples]
-        class_index = index%self.num_samples
-        image = torch.tensor(np.array([np.load(self.directory+selected_class+'/%s'%selected_class+self.aux+'%d.npy'%(class_index))]))
+        sample_index = index%self.num_samples
+        image_path = self.directory / selected_class / f"{selected_class}{self.aux}{sample_index}.npy"
+
+        if not image_path.exists():
+            raise FileNotFoundError(f"Expected sample not found: {image_path}")
+
+        # Keep the returned tensor in the original single-channel format: [1, H, W].
+        image = torch.tensor(np.array([np.load(image_path)]))
+
         if self.aux == '_sim_':
-            image = (image - torch.min(image))/(torch.max(image)-torch.min(image))
-        return image
+            image_min = torch.min(image)
+            image_max = torch.max(image)
+
+            # Avoid dividing by zero if a sample is constant-valued.
+            if torch.isclose(image_max, image_min):
+                return torch.zeros_like(image)
+
+            image = (image - image_min)/(image_max - image_min)
+
+        return image