Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
a63987e
Spike; create master crop file via vips.
tonytw1 May 21, 2025
c845e1c
Naive resize crops to jpegs only step.
tonytw1 May 23, 2025
349d5f5
createCrops and resizeImageVips in same Future and arena to allow sha…
tonytw1 May 23, 2025
b64063d
Try to use 1 load of the source image across all resizes; do the resi…
tonytw1 May 23, 2025
d88df8e
createCrops takes the masterCrop as a VImage rather than a file.
tonytw1 May 23, 2025
d36d3c5
createMasterCrop moving to same arena.
tonytw1 May 23, 2025
166c52e
Remove colourModel check from createMasterCrop.
tonytw1 May 23, 2025
a73b39a
Logging. Show when we reload the master crop from disk.
tonytw1 May 23, 2025
92a3058
Correct arena close timing.
tonytw1 May 23, 2025
3ec25e1
createMasterCrop no Futures.
tonytw1 May 23, 2025
e9aab51
arena scope?
tonytw1 May 23, 2025
5bfd9b3
arena scope?
tonytw1 May 23, 2025
596fb7c
arena scope?
tonytw1 May 23, 2025
1b674b4
arena scope?
tonytw1 May 23, 2025
e0c958b
Spike; no reload of master crop image.
tonytw1 May 23, 2025
84bf7fd
Marking TODO; why local file for master crop.
tonytw1 May 24, 2025
f743f00
Log local resize file location.
tonytw1 May 24, 2025
250e4ff
PNG specific optimiseImage steps can move straight into the resizeIma…
tonytw1 May 24, 2025
335465b
Logging.
tonytw1 May 24, 2025
e377ba7
TODO Strip true is needed to stop exif auto correction until we can s…
tonytw1 May 24, 2025
a7b60e9
Revert "Revert "Master image converted to sRBG colour space.""
tonytw1 May 24, 2025
2acac71
Resize uses save file.
tonytw1 Feb 10, 2026
e327d80
Log crop type decision.
tonytw1 May 24, 2025
4de5344
Colour correct not effective if metadata stripped. Bake it in?
tonytw1 May 24, 2025
afab9d7
Refactor; master crop image file write pushes up out of the create ma…
tonytw1 May 24, 2025
76b3de3
createMasterCrop uses saveToFile and can now save PNGs.
tonytw1 Feb 10, 2026
eb202f3
Refactor. S3 store of master crop pushes up. MasterCrop is a simpler …
tonytw1 May 24, 2025
a2b8b9d
Refactor. File create for mastercrop pushes up to be next to the s3 s…
tonytw1 May 24, 2025
010ef6a
Master file create order; before arena close.
tonytw1 May 24, 2025
b8406b6
Comment; master crop if different from the the full dimensions crop; …
tonytw1 May 24, 2025
081f0b8
Refactor; split creation of crop files from publishing to S3.
tonytw1 May 25, 2025
9e6373b
quantise crops.
tonytw1 Feb 11, 2026
e6af963
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
1a0f01c
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
1ef36cb
Correct CMYK renders too light in crops.
tonytw1 May 25, 2025
07083a3
Unused parameters.
tonytw1 May 26, 2025
6dd8db3
Unused parameters.
tonytw1 May 26, 2025
dc35547
Bypass icc_transform for LAB images.
tonytw1 May 26, 2025
2e9a19c
Remove non used non vips functions.
tonytw1 Feb 28, 2026
bdca28f
Remove non used runConvert command.
tonytw1 May 26, 2025
0fd4e2a
Remove unused iccColourSpace val.
tonytw1 Jun 29, 2025
fbf4fae
Reapply png master quality.
tonytw1 Jun 29, 2025
a0d6d3c
Crop quality values can be int.
tonytw1 Jan 17, 2026
b20bd59
CropType decision can be deferred until after the master crop has bee…
tonytw1 Nov 24, 2025
cdde884
Refactor. Push the isGraphic? decision up out of cropType so that is …
tonytw1 Dec 29, 2025
383c732
Spike. vips based implementation of isGraphic? is likely to involve t…
tonytw1 Dec 29, 2025
d315d26
Test to exercise ImageOperations resize.
tonytw1 Feb 18, 2026
4519e31
[libvips] Testing around hasAlpha.
tonytw1 Feb 1, 2026
c193b19
Clarify master crop quality values.
tonytw1 Jan 17, 2026
6e77178
[libvips-cropping] Allow embedded icc profile to be used in crop icc …
tonytw1 Jan 19, 2026
8789c44
[libvips-cropping] Image operations resize takes the output file as a…
tonytw1 Jan 21, 2026
869853d
[libvips-cropping] Cropping of LAB fixed by making non icc_transform …
tonytw1 Jan 19, 2026
77f7756
[libvips-cropping] Show that it is possible to manually trim metadata…
tonytw1 Feb 12, 2026
aed71f4
[libvips-cropping] Bake the credit, copyright and supplier transmissi…
tonytw1 Jan 24, 2026
7e343c2
XMP Credit -> photoshop:Credit
tonytw1 Feb 24, 2026
82030eb
XMP Byline -> creator.
tonytw1 Feb 24, 2026
3b8c9db
[libvips-cropping] While manually stripping metadata might be useful,…
tonytw1 Feb 12, 2026
27db572
[libvips-cropping] Additional resize tests.
tonytw1 Jan 31, 2026
10a9f24
[libvips-cropping] Test for correct resize of png with alpha.
tonytw1 Feb 16, 2026
e38d6d9
[libvips-cropping] Provide an example of the vips can't render a LAB …
tonytw1 Jan 31, 2026
d3ddf8a
[libvips-cropping] Cleanup; Lower the input of apiSource to imageId f…
tonytw1 Jan 26, 2026
c1979d9
[libvips-cropping] resizeImageVips does not need the MasterCrop object
tonytw1 Mar 15, 2026
12cb734
[libvips-cropping] Metadata for crop call.
tonytw1 Feb 1, 2026
256ee85
[libvips-cropping] Crop metadata in XMP via vips removes exiftool as …
tonytw1 Jan 24, 2026
bbb8647
[libvips-cropping] createCrops moves to image operations for testing.
tonytw1 Jan 26, 2026
5b088dc
[libvips-cropping] Tests to exercise crops.
tonytw1 Jan 31, 2026
6bbf37d
[libvips-cropping] createCrops can execute it's resizeImage calls in …
tonytw1 Mar 15, 2026
7a01025
[libvips-cropping] Master crop save in parallel.
tonytw1 Jan 26, 2026
c8abe6f
[libvips-cropping] Logging.
tonytw1 Jan 26, 2026
fc7ab81
[libvips-cropping] Logging and imports.
tonytw1 Jan 26, 2026
e9382ec
[libvips-cropping] Do not send resizes until master has successfully …
tonytw1 Jan 27, 2026
a9ef182
[libvips-cropping] getImageInformation uses imageOperations.hasAlpha …
tonytw1 Feb 1, 2026
560c0f3
[libvips-cropping] Missing arena.close()
tonytw1 Feb 24, 2026
220d394
[libvips-cropping] Crops uses master hasAlpha image operation rather …
tonytw1 Feb 1, 2026
56fd25c
[libvips-cropping] Do not repeat calls to image_get_interpretation
tonytw1 Feb 24, 2026
3983ebd
Removes exiftool from base image.
tonytw1 Feb 20, 2026
b6e99a5
Remove pnquant from base image.
tonytw1 Jan 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ lazy val commonLib = project("common-lib").settings(
// declare explicit dependency on desired version of aws sdk v2 bedrock runtime
"software.amazon.awssdk" % "bedrockruntime" % awsSdkV2Version,
"software.amazon.awssdk" % "s3vectors" % awsSdkV2Version,
"com.adobe.xmp" % "xmpcore" % "6.1.11",
ws,
"org.testcontainers" % "testcontainers-elasticsearch" % "2.0.2" % Test,
),
Expand Down

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,6 @@ object ImageMagick extends GridLogging {
op
}

def runConvertCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[Unit] = {
Stopwatch.async(s"Using ${if(useImageMagick) "imagemagick" else "graphicsmagick"} for imaging conversion operation '$op'") {
Future {
new ConvertCmd(!useImageMagick).run(op)
}
}
}

def runIdentifyCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[List[String]] = {
Stopwatch.async(s"Using ${if (useImageMagick) "imagemagick" else "graphicsmagick"} for imaging identification operation '$op'") {
Future {
Expand Down
Binary file not shown.
25 changes: 25 additions & 0 deletions common-lib/src/test/resources/schaik.com_pngsuite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Test PNG images
===============

These images are taken from http://www.schaik.com/pngsuite/pngsuite_bas_png.html

basn0g08 - 8 bit (256 level) grayscale
basn2c08 - 3x8 bits rgb color
basn3p08 - 8 bit (256 color) paletted
basn6a08 - 3x8 bits rgb color + 8 bit alpha-channel

LICENCE
-------

At the time of downloading these images the licence file at http://www.schaik.com/pngsuite/PngSuite.LICENSE contained the following text:

```
PngSuite
--------

Permission to use, copy, modify and distribute these images for any
purpose and without fee is hereby granted.


(c) Willem van Schaik, 1996, 2011
```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
package com.gu.mediaservice.lib.imaging

import app.photofox.vipsffm.jextract.VipsRaw
import app.photofox.vipsffm.{VImage, Vips}
import app.photofox.vipsffm.Vips
import com.gu.mediaservice.lib.BrowserViewableImage
import com.gu.mediaservice.lib.logging.{LogMarker, MarkerMap}
import com.gu.mediaservice.model.{Bounds, Dimensions, ImageMetadata, Instance, Jpeg, Png, Tiff}
import org.scalatest.time.{Millis, Span}
import com.gu.mediaservice.model.{Dimensions, Instance, Tiff}
import com.gu.mediaservice.model._
import org.scalatest.concurrent.ScalaFutures
import org.scalatest.funspec.AnyFunSpec
import org.scalatest.matchers.should.Matchers
import org.scalatest.time.{Millis, Span}

import java.io.File
import java.lang.foreign.Arena
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.{Duration, SECONDS}

// This test is disabled for now as it doesn't run on our CI environment, because GraphicsMagick is not present...
class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
Expand All @@ -25,6 +22,12 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
implicit override val patienceConfig: PatienceConfig = PatienceConfig(timeout = Span(1000, Millis), interval = Span(25, Millis))
implicit val logMarker: LogMarker = MarkerMap()

private val metadata = ImageMetadata(
credit = Some("Tony McCrae"),
copyright = Some("Eel Pie Consulting Ltd"),
suppliersReference = Some("eelpie-123")
)

describe("thumbnail") {
it("should write thumbnail to output file") {
val image = fileAt("IMG_4403.jpg")
Expand Down Expand Up @@ -87,6 +90,118 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

describe("resize") {
it("should output resized image to file in chosen format") {
implicit val arena: Arena = Arena.ofShared()
val fullSizedImage = VImage.newFromFile(arena, fileAt("IMG_4403.jpg").getAbsolutePath)
val imageOperations = new ImageOperations("")

val outputFile = new File("/Users/tony/Desktop/out5.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(1000, 800), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly in sRGB") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out6.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly as PNG") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out7.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB 16 bit colour spaces correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB16.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out8.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render PNG with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("with-alpha.png")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/resized-png-with-alpha.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB TIFF with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("lab8-with-alpha.tif")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out13.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}
}

describe("alpha") {
it("should return false for RGB for a Jpeg with no alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("rgb-wo-profile.jpg").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(false)
}

it("should return true for PNG with alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("with-alpha.png").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(true)
}
}

describe("identifyColourModel") {
it("should return RGB for a JPG image with RGB image data and no embedded profile") {
val image = fileAt("rgb-wo-profile.jpg")
Expand Down Expand Up @@ -218,7 +333,91 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

// TODO: test cropImage and its conversions
describe("graphic detection") {
it("should return not graphic for true colour jpeg") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("exif-orientated-no-rotation.jpg").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(false)
arena.close()
}

it("should return is graphic for depth 2 tiff") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("flower.tif").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 4 png with alpha") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/tbbn0g04.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 8 indexed png") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/basn3p08.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

}

describe("cropping") {
val operations = new ImageOperations("")

it("should create unscaled master crop to resize from full sized images") {
implicit val arena: Arena = Arena.ofConfined
//val fullsizedImage = fileAt("Lab 16bpc (7d0b7c7b8e890d7e5d369093aa437bd833e20f71).tiff")
val fullsizedImage = fileAt("IMG_4403.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))
arena.close()
}

it("should create unscaled master crop from CMYK full sized image") {
implicit val arena: Arena = Arena.ofConfined
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master-from-cmyk.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))

arena.close()
}

it("should create files foreach crop size") {
implicit val arena: Arena = Arena.ofShared()
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 3000, 2000), metadata, None)
val landscapeCropSizingWidths = Seq(
Dimensions(140, 100),
Dimensions(320, 200),
Dimensions(800, 600),
Dimensions(1000, 1200),
Dimensions(2000, 3000),
)
implicit val i: Instance = Instance("id")

val crops = operations.createCrops(masterCrop, landscapeCropSizingWidths.toList, "test-image-id",
Bounds(0, 0, 1000, 1200),
Jpeg,
new File("/Users/tony/tmp/crops"),
75
)

arena.close()
}
}

def fileAt(resourcePath: String): File = {
new File(getClass.getResource(s"/$resourcePath").toURI)
Expand Down
2 changes: 0 additions & 2 deletions container-images/jdk-vips/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,6 @@ RUN rm /tmp/vips-8.18.0.tar.xz
RUN rm -r /tmp/vips-8.18.0/

RUN apt -y --no-install-suggests install \
pngquant \
libimage-exiftool-perl \
libjemalloc-dev \
graphicsmagick \
graphicsmagick-imagemagick-compat
Loading
Loading