Skip to content

Commit 09f30bc

Browse files
gliegardgliegard
authored
Merge pull request #6 from IGNF/dev
dev -> main pour version 1.0.2
2 parents 55b6f4e + 83da415 commit 09f30bc

23 files changed

Lines changed: 259 additions & 888 deletions

‎.github/workflows/cicd_deploy.yml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
2+
name: cicd_deploy
3+
4+
on:
5+
# Also run when the pull request merges (which generates a push)
6+
# So that we can tag the docker image appropriately.
7+
push:
8+
tags: [ 'v*.*.*' ]
9+
10+
env:
11+
IMAGE_NAME: ${{ github.repository }}
12+
REGISTRY: ghcr.io
13+
TEST_TAG: ${{ github.repository }}:test
14+
15+
jobs:
16+
deploy_docker:
17+
runs-on: ubuntu-latest
18+
19+
permissions:
20+
contents: read
21+
packages: write
22+
23+
steps:
24+
- name: Checkout branch
25+
uses: actions/checkout@v4
26+
27+
- name: Build the Docker image
28+
id: build
29+
uses: docker/build-push-action@v6
30+
with:
31+
load: true
32+
tags: ${{ env.TEST_TAG }}
33+
34+
# run the test on the docker image
35+
- name: Run tests in docker image
36+
run: >
37+
docker run
38+
${{ env.TEST_TAG }}
39+
python -m pytest -s
40+
--log-cli-level=DEBUG
41+
--log-format="%(asctime)s %(levelname)s %(message)s"
42+
--log-date-format="%Y-%m-%d %H:%M:%S"
43+
-m "not functional_test"
44+
45+
# Login against a Docker registry except on PR
46+
# https://github.com/docker/login-action
47+
- name: Log into registry ${{ env.REGISTRY }}
48+
if: github.event_name != 'pull_request'
49+
uses: docker/login-action@v3.4
50+
with:
51+
registry: ${{ env.REGISTRY }}
52+
username: ${{ github.actor }}
53+
password: ${{ secrets.GITHUB_TOKEN }}
54+
55+
# Extract metadata (tags, labels) for Docker
56+
# https://github.com/docker/metadata-action
57+
- name: Extract Docker metadata
58+
if: github.event_name != 'pull_request'
59+
id: meta
60+
uses: docker/metadata-action@v5
61+
with:
62+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
63+
64+
# Build a Docker image with Buildx (don't on PR)
65+
# https://github.com/docker/build-push-action
66+
- name: Build and push Docker image
67+
if: github.event_name != 'pull_request'
68+
id: build-and-push
69+
uses: docker/build-push-action@v6
70+
with:
71+
context: .
72+
push: true
73+
tags: ${{ steps.meta.outputs.tags }}
74+
labels: ${{ steps.meta.outputs.labels }}

‎.github/workflows/cicd_full.yml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
2+
name: cicd_full
3+
4+
on:
5+
# Run tests for pull-requests on master
6+
pull_request:
7+
branches:
8+
- main
9+
# Also run when the pull request merges (which generates a push)
10+
# So that we can tag the docker image appropriately.
11+
push:
12+
branches:
13+
- dev
14+
15+
env:
16+
IMAGE_NAME: ${{ github.repository }}
17+
REGISTRY: ghcr.io
18+
TEST_TAG: ${{ github.repository }}:test
19+
20+
jobs:
21+
deploy_docker:
22+
runs-on: ubuntu-latest
23+
24+
permissions:
25+
contents: read
26+
packages: write
27+
28+
steps:
29+
- name: Checkout branch
30+
uses: actions/checkout@v4
31+
32+
- name: Build the Docker image
33+
id: build
34+
uses: docker/build-push-action@v6
35+
with:
36+
load: true
37+
tags: ${{ env.TEST_TAG }}
38+
39+
# run the test on the docker image
40+
- name: Run tests in docker image
41+
run: >
42+
docker run
43+
${{ env.TEST_TAG }}
44+
python -m pytest -s
45+
--log-cli-level=DEBUG
46+
47+
test_local:
48+
runs-on: ubuntu-latest
49+
permissions:
50+
contents: read
51+
packages: write
52+
53+
steps:
54+
- name: Checkout branch
55+
uses: actions/checkout@v4
56+
57+
# See https://github.com/marketplace/actions/setup-micromamba
58+
- name: setup-micromamba
59+
uses: mamba-org/setup-micromamba@v2
60+
with:
61+
environment-file: environment.yml
62+
environment-name: ctview # activate the environment
63+
cache-environment: true
64+
cache-downloads: true
65+
generate-run-shell: true
66+
67+
- name: Run tests with pytest
68+
shell: micromamba-shell {0}
69+
run: python -m pytest -s --log-cli-level=DEBUG

‎.github/workflows/cicd_light.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
name: cicd_light
2+
3+
on:
4+
# Run tests for non-draft pull request on dev
5+
pull_request:
6+
branches:
7+
- dev
8+
9+
10+
jobs:
11+
test_light:
12+
runs-on: ubuntu-latest
13+
permissions:
14+
contents: read
15+
packages: write
16+
17+
steps:
18+
- name: Checkout branch
19+
uses: actions/checkout@v4
20+
21+
# See https://github.com/marketplace/actions/setup-micromamba
22+
- name: setup-micromamba
23+
uses: mamba-org/setup-micromamba@v2
24+
with:
25+
environment-file: environment.yml
26+
environment-name: ctview # activate the environment
27+
cache-environment: true
28+
cache-downloads: true
29+
generate-run-shell: true
30+
31+
- name: Run tests with pytest
32+
shell: micromamba-shell {0}
33+
run: python -m pytest -s --log-cli-level=DEBUG
34+
35+
36+

‎Dockerfile‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ WORKDIR /ctView
1111

1212
# # Set up the Conda environment: cf https://github.com/mamba-org/micromamba-docker
1313
COPY environment.yml /tmp/env.yaml
14-
COPY requirements.txt /tmp/requirements.txt
1514
RUN chown $MAMBA_USER:$MAMBA_USER /tmp/env.yaml
1615
RUN micromamba install -y -n base -f /tmp/env.yaml && \
1716
micromamba clean --all --yes

‎Makefile‎

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
# Makefile to manage main tasks
2+
# cf. https://blog.ianpreston.ca/conda/python/bash/2020/05/13/conda_envs.html#makefile
3+
4+
# Oneshell means I can run multiple lines in a recipe in the same shell, so I don't have to
5+
# chain commands together with semicolon
6+
.ONESHELL:
17

28
install:
39
mamba env update -n ctview -f environment.yml
@@ -6,27 +12,27 @@ install-precommit:
612
pre-commit install
713

814
testing:
9-
./ci/test.sh
15+
python -m pytest -s ./test -v
1016

1117
##############################
1218
# Docker
1319
##############################
1420

15-
PROJECT_NAME=lidar_hd/ct_view
21+
REGISTRY=ghcr.io
22+
NAMESPACE=ignf
23+
IMAGE_NAME=ctview
1624
VERSION=`python -m ctview._version`
17-
REGISTRY=docker-registry.ign.fr
25+
FULL_IMAGE_NAME=${REGISTRY}/${NAMESPACE}/${IMAGE_NAME}:${VERSION}
1826

1927
docker-build:
20-
docker build -t ${PROJECT_NAME}:${VERSION} -f Dockerfile .
28+
docker build -t ${IMAGE_NAME}:${VERSION} -f Dockerfile .
2129

2230
docker-test:
23-
docker run --rm -it ${PROJECT_NAME}:${VERSION} python -m pytest -s
31+
docker run --rm ${IMAGE_NAME}:${VERSION} python -m pytest -s -m "not functional_test"
2432

2533
docker-remove:
26-
docker rmi -f `docker images | grep ${PROJECT_NAME} | tr -s ' ' | cut -d ' ' -f 3`
27-
docker rmi -f `docker images -f "dangling=true" -q`
34+
docker rmi -f `docker images | grep ${IMAGE_NAME}:${VERSION} | tr -s ' ' | cut -d ' ' -f 3`
2835

2936
docker-deploy:
30-
docker login docker-registry.ign.fr -u svc_lidarhd
31-
docker tag ${PROJECT_NAME}:${VERSION} ${REGISTRY}/${PROJECT_NAME}:${VERSION}
32-
docker push ${REGISTRY}/${PROJECT_NAME}:${VERSION}
37+
docker tag ${IMAGE_NAME}:${VERSION} ${FULL_IMAGE_NAME}
38+
docker push ${FULL_IMAGE_NAME}

‎README.md‎

Lines changed: 28 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,33 @@
11
# CtView
22

3-
Ce logiciel permet de créer différents rasters représentant les données contenues dans un nuage de points. Il a été initiallement développé pour les usages suivants :
3+
# English version (short)
4+
5+
## Overview
6+
CtView is a tool based on pdal/gdal that generates misc thematic maps (raster views) from las/laz files.
7+
8+
Its initial goal was to create
9+
- operator views for pointcloud classification control
10+
- raster metadata (density and classification maps)
11+
12+
The currently implemented rasters are:
13+
- density map
14+
- classification map: in case of a pixel that contains several classes, classification values can be combined in combination classes (provided as parameters), and a priority list is used to choose the class to display among the remaining classes
15+
16+
Rasters can be (or not) colorized or shadowed using a digital surface/terrain model.
17+
18+
## Usage
19+
20+
- Ctview has been developed using Linux Ubuntu and tested under linux and windows
21+
- CtView should be used as a command line tool (commands must be executed from ctview's root folder)
22+
- It is intended to run on a single file (to use it on a folder, an external loop is required)
23+
- The configuration is handled via yaml files with [hydra](https://hydra.cc/docs/intro/)
24+
- A few commands for installation and tests are provided in a [Makefile](Makefile) in the root folder
25+
- conda is used for environment building, pytest for testing
26+
- The entry point of the code is [ctview/main_ctview.py](ctview/main_ctview.py)
27+
28+
# Version française (plus complète)
29+
30+
Ce logiciel permet de créer différents rasters représentant les données contenues dans un nuage de points. Il a été initialement développé pour les usages suivants :
431
- Créer des vues opérateurs pour du contrôle de classification de nuages de points.
532
- Créer des métadonnées associées au nuage de points (carte de densité et cartes de classes)
633

@@ -134,9 +161,3 @@ exemple_control_fast.sh
134161
```
135162
exemple_metadata_fast.sh
136163
```
137-
138-
# POC Aymeric metadata carte de classe
139-
140-
```
141-
/var/data/store-echange/SV3D/Aymeric/RDI/DEMO_Raster_Class
142-
```

‎RELEASE_NOTES.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,12 @@
1+
# dev
2+
3+
# v1.0.2
4+
- Fix bug 2461, due to an error on ZIP compression while making buffered tile.
5+
The temporary buffered tile is now a LAS (instead of a LAZ), so there is no compression.
6+
- Adapt ci for github
7+
- host docker image on ghcr.io
8+
- update dependencies
9+
110
# v1.0.1
211
- Fix tile origin detection on tiles that are thinner than the buffer size
312
- Make density and class map computation optional by setting the corresponding output sub-dirs to "null"

‎TODO.md‎

Lines changed: 0 additions & 14 deletions
This file was deleted.

‎ci/test.sh‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎configs/config_control.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ class_map:
8888
- {value: 202, description: "Candidats Batiment", color: [255, 128, 128]}
8989
dxm_filter: # Paramètres pour le calcul du MNS (utilisé pour l'ombrage), dimension à filtrer et
9090
# valeur à garder, par défaut on utilise la dimension créée dans la phase de preprocessing
91-
# des produits dérivés
91+
# de las_digital_models
9292
dimension: dsm_marker
9393
keep_values: [1]
9494
post_processing: # Traitements à appliquer à la carte de classe

0 commit comments

Comments
 (0)