End-to-End Python Template w/ Poetry

This is a Python project template that provides a starting point for building Python applications with essential development tools such as tests, linting, formatting, typing, coverage, and CI/CD setup using Poetry to manage dependencies.

 

Prerequisites

Before you begin, ensure you have met the following requirements:

• Poetry: Make sure you have Poetry, a Python dependency management and packaging tool, installed on your system. You can install it by following the instructions on the Poetry website. This will handle all of the environments for your application to eradicate the need of local versioning, ensuring all developers have a safe and immediate environment.

• Python: This project requires Python. You can download the latest version of Python from the official Python website.

 

Structure

This repository gives a fully functional template for starting new Python applications. It provides the following structure:

This repository offers a fully functional template for starting new Python applications. It provides the following directory and file structure:

• .github: This directory contains GitHub-specific files and configurations.

  • workflows: Inside this subdirectory, you can find GitHub Actions workflows. These workflows define automated tasks that run when specific events occur, such as pushing code to the repository. They are used to automate various development and testing processes.

• .coveragerc: This file is a configuration file for the coverage measurement tool. It specifies settings for code coverage reporting, which helps you understand how much of your code is tested. You can customize these settings to suit your project's coverage needs.

• src: This directory is where you can place your Python source code modules. It's a common convention to organize your code into packages and modules within the src directory. Your application's core logic and functionality will reside here. In the pyproject.toml the built package is derived from the underlying src directory. To modify this, change: packages = [{ include = "project_template", from = "src" }]

• tests: This directory is meant for your unit tests. Tests ensure that your code behaves as expected and helps maintain its quality.

• prospector.yaml: Prospector is a tool for analyzing Python code. This configuration file defines settings for code analysis, linting, and other checks that can help you maintain code quality. You can configure Prospector to enforce coding standards and catch potential issues in your codebase. This template focuses on using pylint to maintain code quality, and is validated in CI/CD.

• pyproject.toml: This is the configuration file for Poetry, a Python dependency management and packaging tool. It specifies project metadata and dependencies, making it easier to manage your project's dependencies. You can define project details, dependencies, and other metadata in this file.

• poetry.lock: This file is automatically generated by Poetry and records the exact versions of your project's dependencies, ensuring consistent builds across different environments. It's used to lock dependencies to specific versions to prevent unexpected changes.

• examples/main.py: This is often the entry point to your Python application. You can start your application by running this script. Depending on your project's purpose, main.py may contain the primary functionality of your application. This example displays the use of this built python library project_template

• makefile: A Makefile is used for defining tasks and commands for building and managing your project. It can simplify common development tasks and automate repetitive actions. You can define build, test, deployment, and other tasks in this file to streamline your workflow.

CI/CD w/ Github Workflows

In this repository, CI/CD processes are orchestrated using GitHub Actions workflows. The primary workflow is configured to run a series of checks on code changes, including testing, linting, and type checking, before merging or deploying any code.

Workflow Steps

The main CI/CD workflow includes the following steps:

1. Checkout Code: This step checks out the latest code from the repository, ensuring that the workflow operates on the latest changes.

2. Testing with pytest: The workflow runs pytest, a popular testing framework for Python. pytest executes unit tests defined in the tests directory, verifying that the code functions correctly. Any test failures trigger a notification, indicating potential issues in the code.

3. Linting with pylint: Pylint is used for static code analysis and style checking. It enforces coding standards, checks for code smells, and provides suggestions for improving code quality. Code that doesn't meet linting standards can be identified and corrected.

4. Type Checking with mypy: mypy is a static type checker for Python. It helps ensure type safety by analyzing code and flagging type-related issues. Type checking is especially valuable in large codebases, as it helps catch type-related errors before runtime.

Benefits of CI/CD

• Faster Feedback: CI/CD provides rapid feedback on code changes. Developers receive notifications about test failures or code issues shortly after pushing their changes, allowing for quick resolution.

• Code Quality: Automated checks, including testing, linting, and type checking, ensure code quality and adherence to coding standards.

• Safe Deployments: CD automates the deployment process, reducing the risk of human error during deployments. Only code that has passed all checks is deployed, minimizing downtime and issues in production.

• Consistency: CI/CD promotes consistency by running the same checks and processes for every code change, ensuring that the codebase remains reliable and stable.

By utilizing GitHub Actions for CI/CD, this repository maintains a high standard of code quality, fosters collaboration, and streamlines the development process, ultimately delivering more reliable and efficient Python applications.

This will run by default on use of this template

 

Getting Started

To use this template for your Python project, follow these steps:

 

1 - Clone the repository to your local machine (or use as template):

   git clone https://github.com/nachatz/python-template.git

 

2 - Evaluate the template

Run

make install

Ensure you have installed Poetry & Python

 

3 - Run the sample code

make run

 

4 - Update for your usecase

Navigate to the pyproject.toml and update all of the versions & dependencies you need (Python version, libraries, etc). It's highly recommended to utilize Poetry's API for this.

poetry add <PACKAGE>
poetry remove <PACKAGE>

Add any technology you only will use as a developer into the dev group to prevent it being packaged in release.

poetry add <PACKAGE> --with dev

 

5 - Build your application

Remove the contents of the src directory, or use it as a structural diagram. Use the root main.py as the entrypoint for navigating your APIs/libraries.

 

 

Make commands

This project includes a set of make commands to help you manage your development tasks. Here are the available commands:

• make install: Install project dependencies, including development extras.

• make fmt: Format code using Black. Use the FLAGS variable to pass additional formatting options. Example: make fmt FLAGS="--exclude some_folder".

• make mypy: Run MyPy type checking.

• make pylint: Run pylint for code analysis.

• make check: Run formatting and type checking (fmt and mypy).

• make validate: Run full validation, including checks, tests, and coverage.