dags: Refactor fetch_images into a helper module

[awilfox]

We can reuse this code in many different ways by making it a fully parameterised class instead of embedded in the DAG.

This also gives us a lot of flexibility in how we use it in the future.

Ref: AP-670

---

I still need to write tests for this, but I wanted to make sure the overall API I wrote was actually good and not just a diphenhydramine-induced fever dream.

[awilfox]

Self-reviewed to point out questions I had during implementation.

[awilfox]

v2:

• correct calculation with max width/height (ensure it is floating point, and use the present value instead of the original value, since it may already be scaled for byte size)

[anarchivist]

looks good. all my questions/comments are minor and nonblocking. r+

[awilfox]

v3:

• make md_cache a private attribute of the class instead of a public attribute.

Need to write tests, then this should be good to go.

[awilfox]

v4:

• Figured out that I could use the metadata fetch method on the image fetcher class in the DAG as well. This de-couples it from the inner TindClient, and also means we're able to re-use the cache so we only make one call instead of two!

[danschmidt5189]

Image size should scale roughly with the square of the change in side length, i.e. if you double the width you should naively expect the image size to quadruple. Ditto for shrinking: cutting the height of the image in half (while maintaining the aspect ratio) should reduce the image size to ~1/4 of the original (naively, ignoring resampling / compression). Obviously the reality is different but that would put a lower bound on the change.

[awilfox]

It should, but the IIIF endpoint sends a quality=2 (i.e. 100%) JPEG, so it balloons it quite heavily even with smaller dimensions than the original. A shame we cannot tweak that further.

[awilfox]

The only way to do this without naive calculations such as this would be to download a full-size version from IIIF so that we actually know the maximal byte size. But we would be doing that just to throw it away, using a lot of bandwidth and time in the process.

Happy to consider other ways to do this and discuss it further, but not sure it's worth it at this stage. Up to you.

[awilfox]

v5:

• Don't pass the 0 index from the fetch_image_to_record_directory task because it is already the default.
• Add get_file_metadata method to the FetchTind object so we can use it. Make get_first_file_metadata call it.
• Use functools @cache instead of manually implementing it.
• Only scale once, using the smallest of the potential factors.
• Simplify calculations in the second size check.
• Give another 5% cushion when re-downloading for over-sized result from IIIF API.

[jason-raitz]

r+ Good work!

[awilfox]

v6:

• Add unit tests.

[awilfox]

v7:

• Fix small test gaffe.
• Rebase on main.

[jason-raitz]

r+ I love how much simpler this makes the fetch_images DAG!
I'm going to have to rework some of #677 after this gets merged.