Skip to content

[UUM-133527] - Improve floating point precision when Probuilderizing small objects #658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thomas-tu wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[UUM-133527] - Improve floating point precision when Probuilderizing small objects #658

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Runtime/Core/VectorHash.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ namespace UnityEngine.ProBuilder
/// </summary>
static class VectorHash
{
public const float FltCompareResolution = 1000f;
public const float FltCompareResolution = 1000000f;
Copy link

@u-pr u-pr bot Mar 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high
Have you considered the impact of this new resolution on float precision limits and potential integer overflows?

Increasing FltCompareResolution to 1000000f introduces two major issues:

  1. Integer Overflow: Conversions in VectorHash and related structs (like IntVec3) multiply vertex coordinates by this resolution and convert them using System.Convert.ToInt32(). With a multiplier of 1,000,000, any coordinate with a magnitude greater than ~2147.48 will exceed int.MaxValue, triggering a runtime OverflowException and crashing the mesh generation process.
  2. Hash Stability: The purpose of this resolution is to mask floating-point imprecision so that spatially identical vertices hash to the same value. A 32-bit float only provides ~7 significant decimal digits. A scale of 1,000,000 demands exactness up to $10^{-6}$. For any coordinate $\ge 10$, this exceeds standard float precision limits. Minor math inaccuracies will cause virtually identical vertices to resolve to different integers, which will break vertex welding, duplicate detection, and spatial sorting.

I highly recommend reverting this to 1000f (which provides a reliable $0.001$ precision tolerance and safely supports coordinates up to ~2,000,000) to maintain tool stability.

🤖 Helpful? 👍/👎


static int HashFloat(float f)
{
Expand Down