Implement 3-Layer GMPL → MUIO Conversion Pipeline (Parser, Interpreter, Transformer)

[parthdagia05]

Closes : #2

Feature: 3-Layer GMPL to MUIO JSON Transformation Pipeline ⚙️

Closes: #

Overview

This PR introduces a complete, modular, and fully validated 3-layer pipeline to convert standard GMPL/OSeMOSYS .dat files into MUIO-compatible JSON structures.

The implementation is designed as a standalone, additive conversion engine. It does not modify existing application behavior and requires no external dependencies, making it a safe and testable foundation for future UI import workflows.

---

Architecture

The pipeline follows a strictly decoupled 3-phase execution model to ensure structural integrity and semantic accuracy.

Data Flow:
GMPL .dat ➔ GMPLParser (Syntax) ➔ SliceInterpreter (Semantics) ➔ MuioTransformer (Schema) ➔ MUIO JSON

---

Phase Details

1. GMPLParser (Structural Extraction)

File: GMPLParser.py
Responsible for parsing raw GMPL syntax into structured Python objects without altering data semantics.

• Extraction: Handles set definitions, parameter blocks, and multi-slice tables.
• Edge Case Resilience: Manages glued tokens (e.g., 999:=), headerless parameters, mixed whitespace (tabs/spaces), and trailing decimals (0.).
• Validation: Verified against UTOPIA (54 params) and MUIO-style (46 params) fixtures.

2. SliceInterpreter (Semantic Expansion)

File: SliceInterpreter.py
Expands parsed slice blocks into normalized tuple dictionaries: { param_name: { dimension_tuple: value } }.

• Capabilities: Full wildcard expansion (*), single/double wildcard resolution, and region padding.
• Normalization: Aligns conversion matrices and technology-to-storage parameters into standardized dimension tuples.

Parameter	Expansion Type	Tuple Result
CapitalCost	2D	(Region, Tech)
InputActivityRatio	5D	(Region, Year, Tech, Fuel, Mode)
TechnologyToStorage	4D	(Region, Tech, Storage, Mode)

3. MuioTransformer (MUIO JSON Generation)

File: MuioTransformer.py
Transforms normalized tuples into the exact JSON schema required by the MUIO backend.

• Deterministic ID Generation: Generates stable, sorted IDs to ensure repeated runs produce identical results.
• Logic Mapping: Renames FUEL ➔ COMMODITY and injects required MUIO sets (e.g., UDC, STORAGEINTRADAY).
• Schema Coverage: Generates 22 distinct file groups (R, RT, RYT, RYTCM, etc.) within a scenario envelope (SC_0).

Set Pattern	Generated ID Example
TECHNOLOGY	T_0, T_1
COMMODITY	C_0, C_1
EMISSION	E_0, E_1

---

Validation Suite

Comprehensive tests have been implemented to ensure zero regressions across the pipeline. These can be executed locally via:

PYTHONPATH=API python3 API/Classes/Case/validate_parser.py
PYTHONPATH=API python3 API/Classes/Case/validate_interpreter.py
PYTHONPATH=API python3 API/Classes/Case/validate_transformer.py
All tests currently pass for both canonical (UTOPIA) and sparse (MUIO) test fixtures.

## Safety & Compatibility
Fully Additive: No existing files were modified. All logic resides in new modules.

No Side Effects: The engine performs no filesystem writes and has no dependency on Flask or global Config objects.

Environment Agnostic: Runs in any Python 3 environment without extra requirements.

Note: This PR serves as the foundational infrastructure. A future update will integrate these modules into the main UI case-upload workflow.


***

### Key Formatting Highlights:
* **Code Blocks & Tables:** Used to make technical data (like ID patterns and validation commands) scannable.
* **Data Flow Visualization:** Replaced the simple list with a clear "➔" flow to show the ETL (Extract, Transform, Load) nature of the work.
* **Checklist/Status:** Used bold headers and emojis to distinguish the "Phase" sections clearly for a reviewer.

---