The spirit of this project is to produce a framework that solves distributed serving of LLMs in a non-specific way to the inference engine running the weights. All features should be implemented with this in mind.
- Fork the repository
- Clone your fork:
git clone https://github.com/your-username/inference-stack.git - Install Helm and helm-unittest
-
Create a feature branch:
git checkout -b feature-name -
Make your changes to templates, values, or examples
-
Test your changes:
helm lint . helm unittest . helm template test-release . --values examples/single-vllm.yaml
-
Update tests if needed
-
Commit with conventional commits:
feat:,fix:,docs:, etc.
- Push your branch:
git push origin feature-name - Create a Pull Request with:
- Clear description of changes
- Test results showing lint and unittest pass
- Example usage if adding new features
templates/- Helm templatesexamples/- Example values filestests/- Unit testsvalues.yaml- Default configuration
Open an issue for questions or feature requests. Reach out at: hello@doubleword.ai!