Commit 9697d48
MeshHandler: Fix
fix(MeshHandler): fix `weightsData` dequantization error
In the Editor's serialization code, normalized weights are quantized as 5-bit UNORM values then packed as ints. This implies flooring - while the sum of the quantized values are guaranteed to be `31` this can lead to a loss of precision where the _de_quantized sum (div by 31 then sum) of weights is less than 1.0.
Such cases can be trivally reproduced by search, for example a tuple of (1,25,5) summed as (1/31+25/31+5/31) in _that exact order_ will yield 0.9999999999999999 instead of 1.0, and ((1/31+25/31+5/31) >= 1.0) will be falsey.
This PR implements the sum of weight with ints instead. Alternatively an epsilon value could be chosen, but this is what the player code does.weightsData dequantization error (#338)1 parent 5734513 commit 9697d48
1 file changed
Lines changed: 2 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
567 | 567 | | |
568 | 568 | | |
569 | 569 | | |
570 | | - | |
571 | 570 | | |
572 | 571 | | |
573 | 572 | | |
| |||
580 | 579 | | |
581 | 580 | | |
582 | 581 | | |
583 | | - | |
| 582 | + | |
584 | 583 | | |
585 | 584 | | |
586 | 585 | | |
587 | 586 | | |
588 | 587 | | |
589 | 588 | | |
590 | | - | |
| 589 | + | |
591 | 590 | | |
592 | 591 | | |
593 | 592 | | |
| |||
0 commit comments