Feature/optimizer module and registry

docs/source/get-started/installation.md

@@ -30,7 +30,7 @@ The following [LLM](../build-workflows/llms/index.md) API providers are supporte
 
 ## Packages
 
-To keep the library lightweight, many of the first-party plugins supported by NeMo Agent Toolkit are located in separate distribution packages. For example, the `nvidia-nat-langchain` distribution contains all the LangChain-specific and LangGraph-specific plugins, and the `nvidia-nat-mem0ai` distribution contains the Mem0-specific plugins.
+The default `nvidia-nat` install includes `nvidia-nat-core` and `nvidia-nat-config-optimizer` (workflow config and prompt optimization). To keep the library lightweight, many other first-party plugins are in separate distribution packages. For example, the `nvidia-nat-langchain` distribution contains all the LangChain-specific and LangGraph-specific plugins, and the `nvidia-nat-mem0ai` distribution contains the Mem0-specific plugins.
 
 To install these first-party plugin libraries, you can use the full distribution name (for example, `nvidia-nat-langchain`) or use the `nvidia-nat[langchain]` extra distribution. The following extras are supported:
 
@@ -43,6 +43,7 @@ To install these first-party plugin libraries, you can use the full distribution
 - `nvidia-nat[mcp]` or `nvidia-nat-mcp` - [Model Context Protocol (MCP)](https://modelcontextprotocol.io/)
 - `nvidia-nat[mem0ai]` or `nvidia-nat-mem0ai` - [Mem0](https://mem0.ai/)
 - `nvidia-nat[mysql]` or `nvidia-nat-mysql` - [MySQL](https://www.mysql.com/)
+- `nvidia-nat[optimizer]` or `nvidia-nat-config-optimizer` - Workflow config and prompt optimizer (included by default with `nvidia-nat`; add this extra if you installed only `nvidia-nat-core` and want to use `nat optimize`)
 - `nvidia-nat[openpipe-art]` or `nvidia-nat-openpipe-art` - [Agent Reinforcement Trainer](https://art.openpipe.ai/getting-started/about) Conflicts with `nvidia-nat[adk]` and `nvidia-nat[crewai]`.
 - `nvidia-nat[opentelemetry]` or `nvidia-nat-opentelemetry` - [OpenTelemetry](https://opentelemetry.io/)
 - `nvidia-nat[phoenix]` or `nvidia-nat-phoenix` - [Arize Phoenix](https://arize.com/docs/phoenix)

docs/source/improve-workflows/optimizer.md

@@ -18,6 +18,10 @@ limitations under the License.
 
 This document provides a comprehensive overview of how to use the NeMo Agent Toolkit Optimizer to tune your NeMo Agent Toolkit [workflows](../build-workflows/about-building-workflows.md).
 
+## Prerequisites
+
+The optimizer is included when you install `nvidia-nat`. If you installed only `nvidia-nat-core`, install the config optimizer to use `nat optimize`: run `pip install nvidia-nat-config-optimizer` or `pip install nvidia-nat`. See the [Install Guide](../get-started/installation.md) for details.
+
 ## Introduction
 
 ### What is Parameter Optimization?

examples/dynamo_integration/react_benchmark_agent/src/react_benchmark_agent/DEVELOPER_NOTES.md

@@ -347,7 +347,7 @@ This configuration enables the optimizer to tune Dynamo router parameters for la
 | `OptimizableField`, `SearchSpace` | `nat/data_models/optimizable.py` | Hyper-parameter metadata and Optuna integration |
 | `evaluators.avg_llm_latency._type: avg_llm_latency` | `nat/eval/runtime_evaluator/register.py` | `AverageLLMLatencyConfig` evaluator |
 | `optimizer.eval_metrics` | `nat/data_models/optimizer.py` | `OptimizerConfig.eval_metrics` field |
-| Optimizer runtime | `nat/profiler/parameter_optimization/parameter_optimizer.py` | `optimize_parameters()` function |
+| Optimizer runtime | `nat/optimizer/parameter_optimizer.py` | `optimize_parameters()` function |
 
 #### Optimizable Parameters
 

examples/evaluation_and_profiling/email_phishing_analyzer/tests/test_email_phishing_analyzer.py

@@ -49,7 +49,7 @@ async def test_run_full_workflow():
 async def test_optimize_full_workflow(capsys):
     from nat.data_models.config import Config
     from nat.data_models.optimizer import OptimizerRunConfig
-    from nat.profiler.parameter_optimization.optimizer_runtime import optimize_config
+    from nat.optimizer.optimizer_runtime import optimize_config
     from nat_email_phishing_analyzer.register import EmailPhishingAnalyzerConfig
 
     config_file: Path = locate_example_config(EmailPhishingAnalyzerConfig, "config_optimizer.yml")

packages/nvidia_nat_config_optimizer/README.md

@@ -0,0 +1,24 @@
+# nvidia-nat-config-optimizer
+
+Workflow config and prompt optimization for [NVIDIA NeMo Agent Toolkit](https://github.com/NVIDIA/NeMo-Agent-Toolkit). Provides genetic-algorithm and numeric (Optuna) optimizers for workflow configs and prompts. Scoped to config-level optimization (hyperparameters, prompts); excludes runtime/inference optimizations.
+
+Install with NAT: `pip install nvidia-nat` (config optimizer is included by default), or `pip install nvidia-nat-core nvidia-nat-config-optimizer`.
+
+Config-optimizer-only (minimal deps): `pip install nvidia-nat-config-optimizer` (requires `nvidia-nat-core` for eval contracts).
+
+## Development / testing
+
+From **repo root** (install test deps, then run optimizer tests):
+
+```bash
+uv sync --extra test
+uv run pytest packages/nvidia_nat_config_optimizer/tests/ -v
+```
+
+For Pareto/visualization tests (matplotlib), install the optimizer with the visualization extra first:
+
+```bash
+cd packages/nvidia_nat_config_optimizer && uv sync --extra test --extra visualization && uv run pytest tests/ -v
+```
+
+Or run the full repo test suite (all packages): `python ci/scripts/run_tests.py`

packages/nvidia_nat_config_optimizer/pyproject.toml

@@ -0,0 +1,82 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES.
+# All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[build-system]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools>=64", "setuptools-scm>=8", "setuptools_dynamic_dependencies>=1.0.0"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["nat.*"]
+
+[tool.setuptools_scm]
+git_describe_command = "git describe --long --first-parent"
+root = "../.."
+
+[project]
+name = "nvidia-nat-config-optimizer"
+dynamic = ["version", "dependencies", "optional-dependencies"]
+requires-python = ">=3.11,<3.14"
+description = "Workflow config and prompt optimizer for NVIDIA NeMo Agent Toolkit"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+keywords = ["ai", "rag", "agents", "optimization"]
+classifiers = [
+  "Programming Language :: Python",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+]
+authors = [{ name = "NVIDIA Corporation" }]
+maintainers = [{ name = "NVIDIA Corporation" }]
+
+[project.urls]
+documentation = "https://docs.nvidia.com/nemo/agent-toolkit/latest/"
+source = "https://github.com/NVIDIA/NeMo-Agent-Toolkit"
+
+[tool.setuptools_dynamic_dependencies]
+dependencies = [
+  "nvidia-nat-core == {version}",
+  "numpy~=2.3",
+  "optuna~=4.4.0",
+  "pandas~=2.2",
+  "pydantic~=2.11",
+  "PyYAML~=6.0",
+]
+
+[tool.setuptools_dynamic_dependencies.optional-dependencies]
+visualization = [
+  "matplotlib~=3.9",
+]
+test = [
+  "nvidia-nat-test == {version}",
+]
+
+[tool.uv]
+build-constraint-dependencies = ["setuptools>=64", "setuptools-scm>=8", "setuptools_dynamic_dependencies>=1.0.0"]
+managed = true
+config-settings = { editable_mode = "compat" }
+
+[tool.setuptools]
+include-package-data = true
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"
+
+[tool.uv.sources]
+nvidia-nat-core = { path = "../nvidia_nat_core", editable = true }
+nvidia-nat-test = { path = "../nvidia_nat_test", editable = true }

packages/nvidia_nat_config_optimizer/src/nat/optimizer/__init__.py

@@ -0,0 +1,15 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES.
+# All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

packages/nvidia_nat_config_optimizer/src/nat/optimizer/evolutionary_base.py

@@ -0,0 +1,124 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES.
+# All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use it except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Abstract base class for evolutionary prompt optimizers (GA and related strategies)."""
+
+import asyncio
+from abc import ABC
+from abc import abstractmethod
+from typing import Any
+
+from nat.data_models.config import Config
+from nat.data_models.optimizable import SearchSpace
+from nat.data_models.optimizer import OptimizerConfig
+from nat.data_models.optimizer import OptimizerRunConfig
+from nat.eval.evaluate import EvaluationRun
+from nat.eval.evaluate import EvaluationRunConfig
+from nat.optimizer.ga_individual import Individual
+from nat.optimizer.update_helpers import apply_suggestions
+
+
+class BaseEvolutionaryPromptOptimizer(ABC):
+    """
+    Base class for evolutionary prompt optimizers.
+
+    Provides evaluation infrastructure: apply_suggestions + EvaluationRun,
+    concurrent population evaluation, and a _post_evaluate_single hook for
+    subclasses. Fitness computation and persistence are implementation-specific.
+    """
+
+    @abstractmethod
+    async def run(
+        self,
+        *,
+        base_cfg: Config,
+        full_space: dict[str, SearchSpace],
+        optimizer_config: OptimizerConfig,
+        opt_run_config: OptimizerRunConfig,
+    ) -> None:
+        """Run the evolutionary optimization loop."""
+        ...
+
+    # ---------- evaluation ---------- #
+
+    async def _evaluate_single_given_trial(
+        self,
+        ind: Individual,
+        cfg_trial: Config,
+        optimizer_config: OptimizerConfig,
+        opt_run_config: OptimizerRunConfig,
+    ) -> None:
+        """Run EvaluationRun for an already-built trial config; fill ind.metrics. Subclasses may extend via _post_evaluate_single."""
+        eval_cfg = EvaluationRunConfig(
+            config_file=cfg_trial,
+            dataset=opt_run_config.dataset,
+            result_json_path=opt_run_config.result_json_path,
+            endpoint=opt_run_config.endpoint,
+            endpoint_timeout=opt_run_config.endpoint_timeout,
+            override=opt_run_config.override,
+        )
+        metric_cfg = optimizer_config.eval_metrics or {}
+        eval_metrics = [v.evaluator_name for v in metric_cfg.values()]
+        reps = max(1, getattr(optimizer_config, "reps_per_param_set", 1))
+
+        all_results: list[list[tuple[str, Any]]] = []
+        for _ in range(reps):
+            res = (await EvaluationRun(config=eval_cfg).run_and_evaluate()).evaluation_results
+            all_results.append(res)
+
+        metrics: dict[str, float] = {}
+        for metric_name in eval_metrics:
+            scores: list[float] = []
+            for run_results in all_results:
+                for name, result in run_results:
+                    if name == metric_name:
+                        scores.append(result.average_score)
+                        break
+            metrics[metric_name] = float(sum(scores) / len(scores)) if scores else 0.0
+        ind.metrics = metrics
+        await self._post_evaluate_single(ind, all_results, optimizer_config, opt_run_config)
+
+    async def _post_evaluate_single(
+        self,
+        ind: Individual,
+        all_results: list[list[tuple[str, Any]]],
+        optimizer_config: OptimizerConfig,
+        opt_run_config: OptimizerRunConfig,
+    ) -> None:
+        """Override in subclasses to add post-evaluation logic (e.g. oracle feedback). Default no-op."""
+        pass
+
+    async def _evaluate_population(
+        self,
+        population: list[Individual],
+        base_cfg: Config,
+        optimizer_config: OptimizerConfig,
+        opt_run_config: OptimizerRunConfig,
+        max_concurrency: int = 8,
+    ) -> None:
+        """Evaluate all individuals (concurrently). Semaphore wraps only apply_suggestions."""
+        unevaluated = [ind for ind in population if not ind.metrics]
+        if unevaluated:
+            sem = asyncio.Semaphore(max_concurrency)
+
+            async def _eval_one(ind: Individual) -> None:
+                async with sem:
+                    cfg_trial = apply_suggestions(base_cfg, ind.prompts)
+                await self._evaluate_single_given_trial(
+                    ind, cfg_trial, optimizer_config, opt_run_config
+                )
+
+            await asyncio.gather(*[_eval_one(ind) for ind in unevaluated])

packages/nvidia_nat_config_optimizer/src/nat/optimizer/ga_config.py

@@ -0,0 +1,79 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES.
+# All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use it except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+GA-specific optimizer config subtype.
+
+Base types live in nat.data_models.optimizer (core). This package subclasses
+BaseOptimizerConfig with the GA-specific nest (.prompt) and implementation-
+specific fields (oracle feedback) that may not exist in other GA implementations.
+"""
+
+from typing import Literal
+
+from pydantic import Field
+
+from nat.data_models.optimizer import BaseOptimizerConfig
+from nat.data_models.optimizer import PromptGAOptimizationConfig
+
+
+class GAOptimizerConfig(BaseOptimizerConfig):
+    """
+    Optimizer config for this GA prompt implementation.
+
+    Extends the shared base with .prompt (population, generations, init/recombine, etc.)
+    and with oracle-feedback fields that are specific to this implementation—
+    another GA might not use oracle feedback at all, so these live on the subtype.
+    """
+    prompt: PromptGAOptimizationConfig = Field(default_factory=PromptGAOptimizationConfig)
+
+    # Oracle feedback: implementation-specific to this GA (inject failure reasoning into mutations).
+    oracle_feedback_mode: Literal["never", "always", "failing_only", "adaptive"] = Field(
+        description="When to inject failure reasoning into mutations.",
+        default="never",
+    )
+    oracle_feedback_worst_n: int = Field(
+        description="Number of worst-scoring items to extract reasoning from.",
+        default=5,
+        ge=1,
+    )
+    oracle_feedback_max_chars: int = Field(
+        description="Maximum characters for oracle feedback in mutation prompt.",
+        default=4000,
+        ge=1,
+    )
+    oracle_feedback_fitness_threshold: float = Field(
+        description="For 'failing_only' mode: normalized fitness threshold below which feedback is injected.",
+        default=0.3,
+        ge=0.0,
+        le=1.0,
+    )
+    oracle_feedback_stagnation_generations: int = Field(
+        description="For 'adaptive' mode: generations without improvement before enabling feedback.",
+        default=3,
+        ge=1,
+    )
+    oracle_feedback_fitness_variance_threshold: float = Field(
+        description="For 'adaptive' mode: fitness variance threshold for collapse detection.",
+        default=0.01,
+        ge=0.0,
+    )
+    oracle_feedback_diversity_threshold: float = Field(
+        description="For 'adaptive' mode: prompt duplication ratio threshold (0-1).",
+        default=0.5,
+        ge=0.0,
+        le=1.0,
+    )