图像编辑是否只能输出方形的图片？

[NiceRingNode]

请问作者图像编辑是否只能输出方形的图片？测试输入1000 X 801大小的图，若不指定宽高，输出仍然是1024 X 1024。
若指定宽高则会报错，如指定输出1000 X 801时，在internvlu_transformer.py的_prepare_hidden_inputs_anyres函数中会出现einops.EinopsError: Error while processing rearrange-reduction pattern "c (h ph) (w pw) -> (h w) (c ph pw)". Input tensor shape: torch.Size([16, 100, 125]). Additional info: {'ph': 2, 'pw': 2}. Shape mismatch, can't divide axis of length 125 in chunks of 2
论文里的示意图有非方形的图片，但是代码没有相应的实现，请问作者需要进行怎么样的修改使得图像编辑时输入和输出图像大小相等？

[bxhsort]

reszie_images = []
for img in input_images:
    width, height = img.size
    max_ = 0 
    if height > width:
        calculated_width = int(width / height * 1024) 
        calculated_width = 1024
    else:
        calculated_height = int(height / width * 1024)
        calculated_width = 1024
    width = calculated_width // 32 * 32
    height = calculated_height // 32 * 32
    img = img.resize((width, height), resample=Image.BICUBIC)
    reszie_images.append(img)

real_width, real_height = reszie_images[0].size

I think handling it this way is fine; previous image editing was approached in a similar manner as well.

[NiceRingNode]

@bxhsort Thanks a lot for your code and it works! An image with 1000 X 801 will be saved at 1024 X 800. That keeps image aspect ratio to some extent.

[bxhsort]

I'm glad I could help!

[NiceRingNode]

@bxhsort I found something wrong with this code. First,

if height > width:
    calculated_width = int(width / height * 1024) 
    calculated_width = 1024

The calculated_width appears twice and calculated_height has not been defined. There should be a calculated_height here.

Second, this code rescales the longer side of the image to 1024 pixels and adjusts the shorter side proportionally. This fixed resizing strategy is inflexible for images smaller than 1024 pixels (such as 300×200), and may lead to significant distortion.

Third, I found that when the image exceeds 1660×1312=2099200 pixels, the model raises a ZeroDivisionError. The threshold of 2099200 was determined through repeated testing. Therefore, I set the large size to 1440, since 1440×1440=2073600 < 2099200.

So I write a new version to address these issues.

def resize(input_images,max_size=1440):
    if not isinstance(input_images, list):
        input_images = [input_images]
    
    resized_images = []
    for img in input_images:
        w, h = img.size
        if max(w, h) > max_size:
            scale = max_size / max(w, h)
            w = int(w * scale)
            h = int(h * scale)

        new_w = round(w / 32) * 32
        new_h = round(h / 32) * 32
        img = img.resize((new_w, new_h), resample=Image.BICUBIC)
        resized_images.append(img)
    real_width, real_height = resized_images[0].size
    if len(resized_images) == 1:
        return resized_images[0], real_width, real_height
    else:
        return resized_images, real_width, real_height

If you find any issues, please feel free to share here.

[bxhsort]

I have a few thoughts on these two points.

First, regarding image resizing: based on my experience, resizing the image so that the longer side is fixed to 1024 pixels is a more reliable choice. Regardless of the original image resolution, keeping the longer side at 1024 tends to produce more consistent results. For images smaller than 1024, this strategy also helps maintain a more uniform visual scale.

As for the value 1440, my understanding is that the max_bs setting in the code is intended to support dynamic batch inference. When you modify the default value, the model can accept larger inputs. However, this does not necessarily mean the model can handle such resolutions well. In practice, this may be related to the training data distribution: if higher resolutions were not sufficiently covered during training, the model may fail to reconstruct the full image properly. In my tests, this often led to a result where only the central region was reconstructed, while the border areas became white.

I also considered using 512 and 768. However, in my large-scale batch experiments, neither of them performed as well as the adaptive resizing strategy with a longer side of 1024. In particular, when the longer side exceeds 1024, the issue described above becomes more likely to appear. Therefore, my suggestion is to set max_size to 1024 and remove the corresponding max_size check in the code.
To further clarify the resolution limit: I believe the author set the default max_bs=8 mainly due to GPU memory constraints, so the effective upper bound should be approximately: 8 × 512 × 512. In other words, this limit is more likely introduced for memory management during inference, rather than indicating that the model itself is robust to arbitrarily larger resolutions.

[nini0919]

Thank you for raising this question!

Currently, during image generation, the inference script first calculates an image_grid that is divisible by image_gen_processor.patch_size based on the input height and width. The size of the random noise (rand noise) is then determined based on this image_grid.

However, in the subsequent generation process, the input to the DiT model must further satisfy the requirement of being divisible by image_gen_processor.merge_size. In the current pipeline, the generation phase uses the image_grid multiplied by the VAE upsampling scale to restore the corresponding dimensions. But this result does not guarantee divisibility by image_gen_processor.merge_size, which can trigger errors during subsequent tensor rearrangement or merge operations.

The key points here are:

• patch_size actually corresponds to the VAE upsampling scale (8).
• merge_size is 2.

Therefore, while the existing script ensures that the input dimensions align with the patch_size, it is not enough to simultaneously satisfy the divisibility requirement of the merge_size during the subsequent DiT phase. This is the root cause of the current error.

To give an example that triggers this error, suppose the input resolution is 1000 × 801. The current process first calculates the image_grid based on patch_size=8. After aligning to the VAE upsampling scale, a certain dimension might become 125. Although this process satisfies the patch_size requirement, 125 cannot be divided by merge_size=2. Consequently, the rearrange or merge operations in the DiT directly throw an error, such as:

einops.EinopsError: Error while processing rearrange-reduction pattern "c (h ph) (w pw) -> (h w) (c ph pw)".
Input tensor shape: torch.Size([16, 100, 125]).
Additional info: {'ph': 2, 'pw': 2}.
Shape mismatch, can't divide axis of length 125 in chunks of 2.

In other words, the fundamental reason for this error is not that the input resolution itself is completely invalid, but rather that the script only ensures alignment with the patch_size without further ensuring that the restored dimensions also satisfy the merge_size divisibility requirement.
Additionally, I would like to clarify that condition images (condition inputs) are not affected by the height and width passed in here. That is to say, the size of the condition image does not need to perfectly match the size of the final generated image; strict adherence to the exact same resolution is not required.

To address this issue, our current solution is to implement an automatic dimension interception and restoration mechanism inside the pipeline. Taking the generation of a 1000 × 801 image as an example: internally, we first apply the formula new_width = 1000 // 16 * 16 (yielding 992) and new_height = 801 // 16 * 16 (yielding 800) to round down the input dimensions to a safe size of 992 × 800, which is perfectly divisible by 16. We then use this 992 × 800 dimension to generate the noise and run the entire inference process. Once the final image is generated, we simply resize it back to the user's originally specified 1000 × 801 resolution.

We will implement this solution and fix this issue soon. Thank you again for your feedback and discussion!