Skip to content

Latest commit

 

History

History
186 lines (140 loc) · 9.16 KB

installation.md

File metadata and controls

186 lines (140 loc) · 9.16 KB

Install NVIDIA NeMo Agent Toolkit

This guide will help you set up your NVIDIA NeMo Agent Toolkit development environment.

Supported LLM APIs

The following LLM API providers are supported:

  • NIM (such as Llama-3.1-70b-instruct and Llama-3.3-70b-instruct)
  • OpenAI
  • AWS Bedrock
  • Azure OpenAI

Packages

To keep the library lightweight, many of the first-party plugins supported by NeMo Agent Toolkit are located in separate distribution packages. For example, the nvidia-nat-langchain distribution contains all the LangChain-specific and LangGraph-specific plugins, and the nvidia-nat-mem0ai distribution contains the Mem0-specific plugins.

To install these first-party plugin libraries, you can use the full distribution name (for example, nvidia-nat-langchain) or use the nvidia-nat[langchain] extra distribution. The following extras are supported:

  • nvidia-nat[adk] or nvidia-nat-adk - Google ADK Conflicts with nvidia-nat[openpipe-art] and nvidia-nat[ragaai].
  • nvidia-nat[agno] or nvidia-nat-agno - Agno
  • nvidia-nat[crewai] or nvidia-nat-crewai - CrewAI Conflicts with nvidia-nat[openpipe-art].
  • nvidia-nat[data-flywheel] or nvidia-nat-data-flywheel - NeMo DataFlywheel
  • nvidia-nat[eval] or nvidia-nat-eval - Evaluation orchestration package
  • nvidia-nat[langchain] or nvidia-nat-langchain - LangChain, LangGraph
  • nvidia-nat[llama-index] or nvidia-nat-llama-index - LlamaIndex
  • nvidia-nat[mcp] or nvidia-nat-mcp - Model Context Protocol (MCP)
  • nvidia-nat[mem0ai] or nvidia-nat-mem0ai - Mem0
  • nvidia-nat[mysql] or nvidia-nat-mysql - MySQL
  • nvidia-nat[openpipe-art] or nvidia-nat-openpipe-art - Agent Reinforcement Trainer Conflicts with nvidia-nat[adk] and nvidia-nat[crewai].
  • nvidia-nat[opentelemetry] or nvidia-nat-opentelemetry - OpenTelemetry
  • nvidia-nat[phoenix] or nvidia-nat-phoenix - Arize Phoenix
  • nvidia-nat[ragaai] or nvidia-nat-ragaai - RagaAI Catalyst Conflicts with nvidia-nat[adk] and nvidia-nat[strands].
  • nvidia-nat[redis] or nvidia-nat-redis - Redis
  • nvidia-nat[s3] or nvidia-nat-s3 - Amazon S3
  • nvidia-nat[security] or nvidia-nat-security - Security and red-team components (nat red-team)
  • nvidia-nat[semantic-kernel] or nvidia-nat-semantic-kernel - Microsoft Semantic Kernel
  • nvidia-nat[strands] or nvidia-nat-strands - Strands Agents. Conflicts with nvidia-nat[ragaai].
  • nvidia-nat[test] or nvidia-nat-test - NeMo Agent Toolkit testing package
  • nvidia-nat[vanna] or nvidia-nat-vanna - Vanna text-to-SQL with Databricks support
  • nvidia-nat[profiler] or nvidia-nat-profiler - Profiling and performance analysis components used by evaluation and sizing workflows
  • nvidia-nat[weave] or nvidia-nat-weave - Weights & Biases Weave
  • nvidia-nat[zep-cloud] or nvidia-nat-zep-cloud - Zep

Other Extras

  • nvidia-nat[async_endpoints] - Support for asynchronous endpoints when launching nat serve
  • nvidia-nat[gunicorn] - Support for launching nat serve with an alternative server; requires additional configuration file changes
  • nvidia-nat[most] - Extra containing all Framework integrations except for: nvidia-nat-ragaai, nvidia-nat-openpipe-art
  • nvidia-nat[pii-defense] - Additional dependencies for red-teaming defenses and middleware

Supported Platforms

Operating System Architecture Python Version Supported
Linux x86_64 3.11, 3.12, 3.13 ✅ Tested, Validated in CI
Linux aarch64 3.11, 3.12, 3.13 ✅ Tested, Validated in CI
macOS x86_64 3.11, 3.12, 3.13 ❓ Untested, Should Work
macOS aarch64 3.11, 3.12, 3.13 ✅ Tested
Windows x86_64 3.11, 3.12, 3.13 ❓ Untested, Should Work
Windows aarch64 3.11, 3.12, 3.13 ❌ Unsupported

Software Prerequisites

NVIDIA NeMo Agent Toolkit is a Python library that doesn't require a GPU to run by default. Before you begin using NeMo Agent Toolkit, ensure that you meet the following software prerequisites:

Additional Prerequisites for Development

Install from Package

The package installation is recommended for production use.

:::{note} To run any examples, you need to install the NeMo Agent Toolkit from source. :::

To install the latest stable version of NeMo Agent Toolkit, run the following command:

pip install nvidia-nat

NeMo Agent Toolkit has many optional dependencies which can be installed with the core package. Optional dependencies are grouped by framework and can be installed with the core package. For example, to install the LangChain/LangGraph plugin, run the following:

pip install "nvidia-nat[langchain]"

The full list of optional dependencies can be found here.

Install From Source

:::{warning} Using Conda environments is not recommended and may cause component resolution issues. Only create vanilla Python virtual environments through python -m venv or uv venv with no other active environments. For more information, see the Troubleshooting Guide. :::

Installing from source is required to run any examples provided in the repository or to contribute to the project.

  1. Clone the NeMo Agent Toolkit repository to your local machine.

    git clone -b main https://github.com/NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit
cd nemo-agent-toolkit

  2. Initialize, fetch, and update submodules in the Git repository.

    git submodule update --init --recursive

  3. Fetch the data sets by downloading the LFS files.

    git lfs install
git lfs fetch
git lfs pull

  4. Create a Python environment.

    uv venv --python 3.13 --seed .venv
source .venv/bin/activate

    :::{note} Python 3.11 and 3.12 are also supported simply replace 3.13 with 3.11 or 3.12 in the uv command above. :::

  5. Install the NeMo Agent Toolkit library. To install the NeMo Agent Toolkit library along with most of the optional dependencies. Including developer tools (--all-groups) and most of the dependencies needed for profiling and plugins (--extra most) in the source repository, run the following:

    uv sync --all-groups --extra most

    Alternatively to install just the core NeMo Agent Toolkit without any optional plugins, run the following:

    uv sync

    At this point individual plugins, which are located under the packages directory, can be installed with the following command uv pip install -e ".[<plugin_name>]". For example, to install the LangChain/LangGraph plugin, run the following:

    uv pip install -e ".[langchain]"

    :::{note} Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the examples/getting_started/simple_web_query/README.md guide will install the nvidia-nat-langchain plugin if you haven't already done so. :::

    In addition to plugins, install the profiler package when you plan to run profiling workflows with nat eval:

    uv pip install -e ".[profiler]"

  6. Verify that you've installed the NeMo Agent Toolkit library.

    nat --help
nat --version

    If the installation succeeded, the nat command will log the help message and its current version.

Next Steps

  • Follow the Quick Start Guide to get started running workflows with NeMo Agent Toolkit.