Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion pytest.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
[tool:pytest]
testpaths = test
testpaths =
tests
test
python_files = test_*.py *_test.py
python_classes = Test*
python_functions = test_*
Expand Down
142 changes: 95 additions & 47 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -1,126 +1,174 @@
# DeepWiki Tests

This directory contains all tests for the DeepWiki project, organized by type and scope.
This directory contains the main DeepWiki test suite, organized by type and scope. The repository also has a legacy `test/` directory with focused regression tests.

## Directory Structure

```
tests/
├── unit/ # Unit tests - test individual components in isolation
│ ├── test_google_embedder.py # Tests for Google AI embedder client
│ └── test_google_embedder_fix.py # Tests for embedding response parsing fix
├── integration/ # Integration tests - test component interactions
│ └── test_full_integration.py # Full pipeline integration test
├── api/ # API tests - test HTTP endpoints
│ └── test_api.py # API endpoint tests
└── run_tests.py # Test runner script
|-- unit/ # Unit tests - test individual components in isolation
| |-- test_google_embedder.py
| `-- test_all_embedders.py
|-- integration/ # Integration tests - test component interactions
| `-- test_full_integration.py
|-- api/ # API tests - test HTTP endpoints
| `-- test_api.py
`-- run_tests.py # Category-based test runner script

test/
`-- test_extract_repo_name.py # Legacy focused regression tests
```

## Running Tests

### All Tests
### Pytest Discovery

`pytest.ini` is configured to discover tests from both `tests/` and the legacy `test/` directory.

```bash
python -m pytest
```

If you are using the Poetry environment from `api/pyproject.toml` without activating it, run pytest from the repository root with explicit paths:

```bash
poetry -C api run python -m pytest ../tests ../test
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When running from the repository root, the paths to the test directories should be tests and test relative to the current directory. Using ../ would incorrectly point to directories outside the project root.

Suggested change
poetry -C api run python -m pytest ../tests ../test
poetry -C api run python -m pytest tests test

```

### Main Test Suite

```bash
python -m pytest tests
```

### Legacy Focused Tests

```bash
python -m pytest test
```

### Category-Based Test Runner

```bash
python tests/run_tests.py
```

### Unit Tests Only
#### Unit Tests Only

```bash
python tests/run_tests.py --unit
```

### Integration Tests Only
#### Integration Tests Only

```bash
python tests/run_tests.py --integration
```

### API Tests Only
#### API Tests Only

```bash
python tests/run_tests.py --api
```

### Individual Test Files

```bash
# Unit tests
python tests/unit/test_google_embedder.py
python tests/unit/test_google_embedder_fix.py
python -m pytest tests/unit/test_google_embedder.py
python -m pytest tests/unit/test_all_embedders.py

# Integration tests
python tests/integration/test_full_integration.py
python -m pytest tests/integration/test_full_integration.py

# API tests
python tests/api/test_api.py
python -m pytest tests/api/test_api.py

# Legacy focused tests
python -m pytest test/test_extract_repo_name.py
```

## Test Requirements

### Environment Variables

- `GOOGLE_API_KEY`: Required for Google AI embedder tests
- `OPENAI_API_KEY`: Required for some integration tests
- `DEEPWIKI_EMBEDDER_TYPE`: Set to 'google' for Google embedder tests
- `DEEPWIKI_EMBEDDER_TYPE`: Set to `google` for Google embedder tests

### Dependencies
All test dependencies are included in the main project requirements:
- `python-dotenv`: For loading environment variables
- `adalflow`: Core framework for embeddings
- `google-generativeai`: Google AI API client
- `requests`: For API testing

Install the Python dependencies from the project root:

```bash
python -m pip install poetry==2.0.1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Hardcoding a specific version of Poetry (2.0.1) is restrictive. It is generally better to recommend installing the latest version or a version range (e.g., poetry>=2.0.0) to ensure compatibility with the project's configuration while allowing for updates.

Suggested change
python -m pip install poetry==2.0.1
python -m pip install poetry

poetry install -C api
```

The test dependencies are declared in `api/pyproject.toml`.

## Test Categories

### Unit Tests
- **Purpose**: Test individual components in isolation
- **Speed**: Fast (< 1 second per test)
- **Dependencies**: Minimal external dependencies
- **Examples**: Testing embedder response parsing, configuration loading

### Integration Tests
- **Purpose**: Test how components work together
- **Speed**: Medium (1-10 seconds per test)
- **Dependencies**: May require API keys and external services
- **Examples**: End-to-end embedding pipeline, RAG workflow
- Purpose: Test individual components in isolation
- Speed: Fast
- Dependencies: Minimal external dependencies
- Examples: Embedder response parsing and configuration loading

### Integration Tests

- Purpose: Test how components work together
- Speed: Medium
- Dependencies: May require API keys and external services
- Examples: End-to-end embedding pipeline and RAG workflow

### API Tests
- **Purpose**: Test HTTP endpoints and WebSocket connections
- **Speed**: Medium-slow (5-30 seconds per test)
- **Dependencies**: Requires running API server
- **Examples**: Chat completion endpoints, streaming responses

- Purpose: Test HTTP endpoints and WebSocket connections
- Speed: Medium-slow
- Dependencies: Requires a running API server
- Examples: Chat completion endpoints and streaming responses

## Adding New Tests

1. **Choose the right category**: Determine if your test is unit, integration, or API
2. **Create the test file**: Place it in the appropriate subdirectory
3. **Follow naming convention**: `test_<component_name>.py`
4. **Add proper imports**: Use the project root path setup pattern
5. **Document the test**: Add docstrings explaining what the test does
6. **Update this README**: Add your test to the appropriate section
1. Choose the right category: unit, integration, API, or focused regression.
2. Place main suite tests under `tests/` and legacy focused regression tests under `test/` only when matching existing coverage.
3. Follow the naming convention: `test_<component_name>.py`.
4. Add the project root to `sys.path` when needed.
5. Add docstrings or clear test names explaining what the test covers.
6. Update this README when adding new test files or commands.

## Troubleshooting

### Import Errors

If you get import errors, ensure the test file includes the project root path setup:

```python
from pathlib import Path
import sys

# Add the project root to the Python path
project_root = Path(__file__).parent.parent.parent
sys.path.insert(0, str(project_root))
```

For tests directly under `test/`, adjust the parent traversal accordingly.

### API Key Issues

Make sure you have a `.env` file in the project root with the required API keys:

```
```bash
GOOGLE_API_KEY=your_google_api_key_here
OPENAI_API_KEY=your_openai_api_key_here
DEEPWIKI_EMBEDDER_TYPE=google
```

### Server Dependencies

For API tests, ensure the FastAPI server is running on the expected port:

```bash
cd api
python main.py
```
python -m api.main
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

For consistency with the rest of the documentation and to ensure the correct virtual environment is used, this command should be run via Poetry.

Suggested change
python -m api.main
poetry -C api run python -m api.main

```