ci cd pipeline

CI/CD Pipeline

The CI/CD pipeline for the Currency Conversion API project is designed to automate the process of building, testing, and deploying the application. It uses GitHub Actions to define the workflow and ensure code quality and consistency.

Workflow Name

The workflow is named "Build and Test Java Project with Maven".

Trigger Events

The workflow is triggered on push or pull request events to the master branch.

Jobs

Build Job

The build job runs on the latest Ubuntu environment and performs the following steps:

Check out the repository: This step fetches the entire repository history to ensure all branches and tags are available.
Check commit message: Determines if the current commit should skip changelog update based on the commit message.
Generate PlantUML diagrams: Generates PlantUML diagrams if necessary.
Set up JDK 17: Configures the job to use JDK 17.
Cache Maven packages: Caches Maven dependencies to speed up subsequent builds.
Set up Python: Configures the job to use Python 3.8.
Install Python dependencies: Installs necessary Python packages.
Install Groovy: Installs Groovy for script execution.
Get SHA of specified files: Retrieves the SHA of important files like CHANGELOG.md, .env, DIRECTORY.md, swagger.json, and swagger.yaml.
Check if files have changed: Compares the local and remote versions of the specified files to determine if they have changed.
Clean docs/site directory: Cleans the directory to prepare for fresh documentation generation.
Generate necessary files: Runs scripts to generate CHANGELOG.md, .env, DIRECTORY.md, swagger.json, swagger.yaml, and Java documentation in Markdown format.
Adjust Markdown Table with Python: Adjusts the generated Markdown table using a Python script.
Upload all files from docs/site: Commits and pushes changes in the docs/site directory if there are any.
Upload CHANGELOG.md: Uploads the updated CHANGELOG.md to the repository.
Upload .env: Uploads the updated .env file to the repository.
Upload DIRECTORY.md: Uploads the updated DIRECTORY.md to the repository.
Upload swagger.json: Uploads the updated swagger.json to the repository.
Upload swagger.yaml: Uploads the updated swagger.yaml to the repository.
Clean local repository: Cleans the local Maven repository to free up space.
Run tests: Executes the test suite to ensure code quality and correctness.

Example GitHub Actions Workflow File

name: Build and Test Java Project with Maven

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

permissions:
  contents: write

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - name: Check out the repository
      id: check-out-repository
      uses: actions/checkout@v2
      with:
        fetch-depth: 0

    - name: Check if is primary
      id: check-is_primary
      run: |
        COMMIT_MESSAGE=$(git log -1 --pretty=%B)
        if [[ "$COMMIT_MESSAGE" == AUTO_COMMIT:* || "$COMMIT_MESSAGE" == "Render PlantUML files" ]]; then
          echo "is_primary=false" >> $GITHUB_ENV
        else
          echo "is_primary=true" >> $GITHUB_ENV
        fi

    - name: Generate PlantUML diagrams
      id: generate-plantuml-diagrams
      if: env.is_primary == 'true'
      uses: grassedge/generate-plantuml-action@master
      with:
        path: sys/uml
        message: "Render PlantUML files"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Set up JDK 17
      id: setup-jdk
      if: env.is_primary == 'true'
      uses: actions/setup-java@v4
      with:
        distribution: 'adopt'
        java-version: '17'

    - name: Cache Maven packages
      id: cache-maven-package
      if: env.is_primary == 'true'
      uses: actions/cache@v4
      with:
        path: ~/.m2/repository
        key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
        restore-keys: ${{ runner.os }}-maven

    - name: Set up Python
      if: env.is_primary == 'true'
      uses: actions/setup-python@v2
      with:
        python-version: '3.8'

    - name: Install Python dependencies
      if: env.is_primary == 'true'
      run: pip install requests

    - name: Install Groovy
      id: install-groovy
      if: env.is_primary == 'true'
      run: |
        sudo apt-get install -y groovy

    - name: Cache Maven packages
      id: cache-maven-packages
      if: env.is_primary == 'true'
      uses: actions/cache@v4
      with:
        path: ~/.m2/repository
        key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
        restore-keys: ${{ runner.os }}-maven

    - name: Install dependencies
      id: install-depencencies
      if: env.is_primary == 'true'
      run: mvn clean install -Pci-cd -DskipTests

    - name: Get SHA of files if they exist
      if: env.is_primary == 'true'
      id: get-sha
      run: |
        get_sha() {
          local file_path=$1
          local var_name=$2
          local sha=$(curl -s -H "Authorization: Bearer $GITHUB_TOKEN" -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{ github.repository }}/contents/$file_path | jq -r .sha)
          if [ "$sha" != "null" ]; then
            echo "${var_name}=${sha}" >> $GITHUB_ENV
          else
            echo "${var_name}=" >> $GITHUB_ENV
          fi
        }

        get_sha "CHANGELOG.md" "changelog_sha"
        get_sha ".env" "env_sha"
        get_sha "DIRECTORY.md" "directory_sha"
        get_sha "openapi/swagger.json" "swagger_json_sha"
        get_sha "openapi/swagger.yaml" "swagger_yaml_sha"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Check if CHANGELOG.md, .env, DIRECTORY.md, swagger.json or swagger.yaml have changed
      if: env.is_primary == 'true'
      id: check-changes
      run: |
        check_change() {
          local file_path=$1
          local remote_file=$(curl -s -H "Authorization: Bearer ${{ secrets.TOKEN }}" -H "Accept: application/vnd.github.v3.raw" https://api.github.com/repos/${{ github.repository }}/contents/$file_path | base64 -d || echo "")
          local local_file=$(cat $file_path)
          if [ "$remote_file" == "$local_file" ]; then
            echo "No changes in $file_path"
            exit 0
          fi
        }

        check_change "CHANGELOG.md"
        check_change ".env"
        check_change "DIRECTORY.md"
        check_change "openapi/swagger.json"
        check_change "openapi/swagger.yaml"

    - name: Clean docs/site directory
      if: env.is_primary == 'true'
      run: |
        rm -rf docs/site

    - name: Generate necessary files
      id: generate-necessary-files
      if: env.is_primary == 'true'
      run: |
        chmod +x sys/generateChangelogAndProjectVersion.groovy
        groovy sys/generateChangelogAndProjectVersion.groovy
        chmod +x sys/directory.groovy
        groovy sys/directory.groovy
        mvn clean install swagger:generate
        mvn dokka:dokka

    - name: Adjust Markdown Table with Python
      id: adjust-markdown-table-index
      if: env.is_primary == 'true'
      run: |
        python sys/javadocmd.py
      env:
        PYTHONIOENCODING: UTF-8

    - name: Upload all files from docs/site
      if: env.is_primary == 'true'
      run: |
        ( cd docs/site
        git add .
        if git diff --staged --quiet; then
          echo "No changes to commit."
        else
          git config --global user.email "shelson@gmail.com"
          git config --global user.name "Shelson Ferrari"
          git commit -m "AUTO_COMMIT: Bulk update of documentation"
          git push
        fi )
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN3 }}

    - name: Upload CHANGELOG.md
      id: upload-changelog
      if: env.is_primary == 'true'
      run: |
        BASE64_CHANGELOG=$(base64 -w 0 CHANGELOG.md)
        curl -X PUT \
          -H "Authorization: Bearer ${{ secrets.TOKEN }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/contents/CHANGELOG.md \
          -d "$(jq -nc --arg content "$BASE64_CHANGELOG" --arg message "AUTO_COMMIT: Update CHANGELOG.md" --arg sha "$changelog_sha" --arg branch "${GITHUB_REF#refs/heads/}" '{message: $message, content: $content, sha: $sha, branch: $branch}')"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Upload .env
      id: upload-env
      if: env.is_primary == 'true'
      run: |
        BASE64_ENV=$(base64 -w 0 .env)
        curl -X PUT \
          -H "Authorization: Bearer ${{ secrets.TOKEN }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/contents/.env \
          -d "$(jq -nc --arg content "$BASE64_ENV" --arg message "AUTO_COMMIT: Update .env" --arg sha "$env_sha" --arg branch "${GITHUB_REF#refs/heads/}" '{message: $message, content: $content, sha: $sha, branch: $branch}')"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Upload DIRECTORY.md
      id: upload-directory
      if: env.is_primary == 'true'
      run: |
        BASE64_DIRECTORY=$(base64 -w 0 DIRECTORY.md)
        curl -X PUT \
          -H "Authorization: Bearer ${{ secrets.TOKEN }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/contents/DIRECTORY.md \
          -d "$(jq -nc --arg content "$BASE64_DIRECTORY" --arg message "AUTO_COMMIT: Update DIRECTORY.md" --arg sha "$directory_sha" --arg branch "${GITHUB_REF#refs/heads/}" '{message: $message, content: $content, sha: $sha, branch: $branch}')"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Upload swagger.json
      id: upload-swagger-json
      if: env.is_primary == 'true'
      run: |
        BASE64_SWAGGER_JSON=$(base64 -w 0 openapi/swagger.json)
        curl -X PUT \
          -H "Authorization: Bearer ${{ secrets.TOKEN }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/contents/openapi/swagger.json \
          -d "$(jq -nc --arg content "$BASE64_SWAGGER_JSON" --arg message "AUTO_COMMIT: Update swagger.json" --arg sha "$swagger_json_sha" --arg branch "${GITHUB_REF#refs/heads/}" '{message: $message, content: $content, sha: $sha, branch: $branch}')"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Upload swagger.yaml
      id: upload-swagger-yaml
      if: env.is_primary == 'true'
      run: |
        BASE64_SWAGGER_YAML=$(base64 -w 0 openapi/swagger.yaml)
        curl -X PUT \
          -H "Authorization: Bearer ${{ secrets.TOKEN }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/contents/openapi/swagger.yaml \
          -d "$(jq -nc --arg content "$BASE64_SWAGGER_YAML" --arg message "AUTO_COMMIT: Update swagger.yaml" --arg sha "$swagger_yaml_sha" --arg branch "${GITHUB_REF#refs/heads/}" '{message: $message, content: $content, sha: $sha, branch: $branch}')"
      env:
        GITHUB_TOKEN: ${{ secrets.TOKEN }}

    - name: Check if is PlantUML upload SVG
      id: check-is-plantuml
      run: |
        COMMIT_MESSAGE=$(git log -1 --pretty=%B)
        if [[ "$COMMIT_MESSAGE" == "Render PlantUML files" ]]; then
          echo "is_plantuml=true" >> $GITHUB_ENV
        else
          echo "is_plantuml=false" >> $GITHUB_ENV
        fi

    - name: Clean local repository
      id: clean-local-repository
      if: env.is_primary == 'true'
      run: mvn dependency:purge-local-repository

    - name: Run tests
      id: run-tests
      if: env.is_primary == 'true'
      run: mvn test -Pci-cd

Conclusion

The CI/CD pipeline for the Currency Conversion API project includes various automations to streamline the development process, ensure consistency, and generate necessary artifacts. By automating these tasks, we can focus on developing features and improving the application while maintaining high quality and reliability.

Wiki Menu

Wiki Main Page

1. Introduction to the Project

Overview: Presentation of the project, highlighting its purpose and the context in which it is embedded.
Project Objectives: Enumeration of the main objectives that the project aims to achieve.
Scope and Functionalities: Description of the main functionalities offered by the project and its scope of operation.

2. Configuration and Installation

Initial Configurations: Steps required to set up the development or production environment.
Installation Instructions: Detailed procedures for installing the project in different environments.
Docker Configuration: Specifications on how to configure and use Docker for the project.

3. Project Structure

Folder Structure: Description of the organization of the project directories.
Project Architecture: Explanation of the architecture used, including design patterns and technical decisions.

4. Development

Development Flow: Description of the development process adopted, including planning, coding, and review stages.
Apache Camel Integration: Guide on integrating Apache Camel into the project, including configuration and usage.
Contributors and Authors: Recognition of the contributors to the project.
Contributions: Guidelines on how to contribute to the project, including code standards and pull request requirements, tips and best practices.
Code of Conduct: Behavioral guidelines expected for the project community.

5. API and Documentation

OpenAPI Specification: Details about the OpenAPI specification used to document the API endpoints.
API Documentation with Swagger: Information on how to access and use the interactive API documentation generated by Swagger.
Javadoc Documentation: Information on the Javadoc documentation generated for the project.

6. Endpoints and Database

Endpoint Description: Details of the available API endpoints, including methods, parameters, and usage examples.
Database Management: Strategies and practices for efficient management of the database used by the project.

7. Testing

Testing Strategies: Approach and methods used to test the software, including unit, integration, and E2E tests.
Testing Tools: Description of the testing tools used in the project and how to configure them.

8. CI/CD and Automations

CI/CD Pipeline.
Automations and Artifact Generation: Description of the automations incorporated into the CI/CD, including documentation generation and build artifacts.

9. Configuration Files

.gitignore, .editorconfig and project.properties: Utility of these files to maintain code consistency and quality.
Maven Wrapper and application.properties: Explanation of how these files help standardize the development environment.
.env File and Travis CI Settings: Use of these files to configure the environment and CI integrations.

10. Best Practices

Code Standards and Security: Guidelines for maintaining code quality and security.
Monitoring and Logging Practices: Recommended techniques for monitoring and logging in the project.

11. Legal and Licensing

Licensing: Information about the rights and restrictions associated with the use of the software.
Terms of Use: Information about the terms and conditions for using the software.

12. Projections and Innovations

Future Plans: Discussion on functionalities and improvements considered for future versions of the project.
Improvement Proposals: Space for the community to suggest and debate improvements and innovations.

13. Attachments and Useful Links

External Links and References: Additional resources and external documentation relevant to the project.

14. Security

Security Policy: Details on the supported versions, reporting vulnerabilities, and general security practices.

ci cd pipeline

CI/CD Pipeline

Workflow Name

Trigger Events

Jobs

Build Job

Example GitHub Actions Workflow File

Conclusion

Wiki Menu

1. Introduction to the Project

2. Configuration and Installation

3. Project Structure

4. Development

5. API and Documentation

6. Endpoints and Database

7. Testing

8. CI/CD and Automations

9. Configuration Files

10. Best Practices

11. Legal and Licensing

12. Projections and Innovations

13. Attachments and Useful Links

14. Security

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally