[bug]: Gemini model provider definitions not matching documentation

[DocStatic97]

Is there an existing issue for this problem?

• I have searched the existing issues

Install method

Manual

Operating system

Linux

GPU vendor

None (CPU)

GPU model

No response

GPU VRAM

No response

Version number

6.12.0.post1

Browser

No response

System Information

No response

What happened

Relative to the project's official documentation (as seen here)
https://invoke.ai/features/external-models/gemini/
The three Gemini image models are listed to support img2img, text2img and inpainting.

When going into canvas mode, both inpainting & img2img workflows error out saying the feature is not supported.

Trying to update the model definition via the WebUI sends an error that the model couldn't be updated.
Erroring out with this:
{"detail":[{"type":"extra_forbidden","loc":["body","default_settings","MainModelDefaultSettings","num_images"],"msg":"Extra inputs are not permitted","input":1},{"type":"extra_forbidden","loc":["body","default_settings","LoraModelDefaultSettings","width"],"msg":"Extra inputs are not permitted","input":1024},{"type":"extra_forbidden","loc":["body","default_settings","LoraModelDefaultSettings","height"],"msg":"Extra inputs are not permitted","input":1024},{"type":"extra_forbidden","loc":["body","default_settings","LoraModelDefaultSettings","num_images"],"msg":"Extra inputs are not permitted","input":1},{"type":"extra_forbidden","loc":["body","default_settings","LoraModelDefaultSettings","steps"],"msg":"Extra inputs are not permitted","input":null},{"type":"extra_forbidden","loc":["body","default_settings","LoraModelDefaultSettings","guidance"],"msg":"Extra inputs are not permitted","input":null},{"type":"missing","loc":["body","default_settings","ControlAdapterDefaultSettings","preprocessor"],"msg":"Field required","input":{"width":1024,"height":1024,"num_images":1,"steps":null,"guidance":null}},{"type":"extra_forbidden","loc":["body","default_settings","ControlAdapterDefaultSettings","width"],"msg":"Extra inputs are not permitted","input":1024},{"type":"extra_forbidden","loc":["body","default_settings","ControlAdapterDefaultSettings","height"],"msg":"Extra inputs are not permitted","input":1024},{"type":"extra_forbidden","loc":["body","default_settings","ControlAdapterDefaultSettings","num_images"],"msg":"Extra inputs are not permitted","input":1},{"type":"extra_forbidden","loc":["body","default_settings","ControlAdapterDefaultSettings","steps"],"msg":"Extra inputs are not permitted","input":null},{"type":"extra_forbidden","loc":["body","default_settings","ControlAdapterDefaultSettings","guidance"],"msg":"Extra inputs are not permitted","input":null},{"type":"extra_forbidden","loc":["body","default_settings","ExternalApiModelDefaultSettings","steps"],"msg":"Extra inputs are not permitted","input":null},{"type":"extra_forbidden","loc":["body","default_settings","ExternalApiModelDefaultSettings","guidance"],"msg":"Extra inputs are not permitted","input":null}]}

All in all:
Gemini provider seems to hardcode modes: ["txt2img"] and mask_format: "none" but docs claim img2img and inpainting are supported. These two fields need updating in the Gemini provider definition.

What you expected to happen

Either the img2img or the inpainting feature working as stated in the documentation or the update from the WebUI working without erroring out.

How to reproduce the problem

Add an external model, with a google ai API key.
Try to use anything needing img2img or inpainting in canvas mode
Try to update the model's settings to get these features working.

Additional context

No response

Discord username

No response

[Pfannkuchensack]

The docs are wrong because they got written before the final pr got merged. I will Update them.

[DocStatic97]

I see.
What is the actual compatibility for these models then?
Because img2img works when editing the starter_models.py file to add "img2img" for each gemini models.

[Pfannkuchensack]

I think the Problem here is that img2img with gemini is more like a edit mode and not the img2img as we know from SD/FLUX/XXX Models. And if you want to do that with gemini you add the images as ref images to use it.