Skip to content

Improve video frame conversion logic #203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MaxHeimbrock merged 6 commits into from
Mar 27, 2026
Merged

Improve video frame conversion logic #203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e3c5f1d
Mirror in cpu yuv path as well
MaxHeimbrock Mar 26, 2026
419a365
Getting rid of temp version files in git
MaxHeimbrock Mar 26, 2026
6c1b86c
Moving shaders into resources folder, then they will be included in b…
MaxHeimbrock Mar 26, 2026
5ab74e0
Removed debug statement
MaxHeimbrock Mar 27, 2026
ce13213
Convert sRGB to linear in the shader
MaxHeimbrock Mar 27, 2026
06a629e
Shader looks even better with some more tweaking, using limited range…
MaxHeimbrock Mar 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,6 @@ downloads~
# Ignore temporaries from GameCI
**/[Aa]rtifacts/
**/[Cc]odeCoverage/

# Temp generated SDK version file
**/LiveKitSdkVersionInfo.txt
8 changes: 8 additions & 0 deletions Resources.meta
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 Runtime/Shaders.meta → Resources/Shaders.meta
View file
Open in desktop
File renamed without changes.
8 changes: 6 additions & 2 deletions Runtime/Shaders/YuvToRgb.shader → Resources/Shaders/YuvToRgb.shader
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Shader "Hidden/LiveKit/YUV2RGB"

inline half3 yuvToRgb709Limited(half y, half u, half v)
{
// BT.709 limited range
// BT.709 limited range (Y: 16-235, UV: 16-240)
half c = y - half(16.0 / 255.0);
half d = u - half(128.0 / 255.0);
half e = v - half(128.0 / 255.0);
Expand All @@ -60,7 +60,11 @@ Shader "Hidden/LiveKit/YUV2RGB"
half y = tex2D(_TexY, uv).r;
half u = tex2D(_TexU, uv).r;
half v = tex2D(_TexV, uv).r;
return half4(yuvToRgb709Limited(y, u, v), 1.0h);
half3 rgb = yuvToRgb709Limited(y, u, v);
// YUV→RGB produces sRGB/gamma values; convert to linear so the
// hardware's linear→sRGB write to the sRGB RT gives correct output.
rgb = GammaToLinearSpace(rgb);
return half4(rgb, 1.0h);
}
ENDHLSL
}
Expand Down
0 Runtime/Shaders/YuvToRgb.shader.meta → Resources/Shaders/YuvToRgb.shader.meta
View file
Open in desktop
File renamed without changes.
7 changes: 5 additions & 2 deletions Runtime/Scripts/Video/YuvToRgbConverter.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ public bool EnsureOutput(int width, int height)
Output.Release();
UnityEngine.Object.Destroy(Output);
}
Output = new RenderTexture(width, height, 0, RenderTextureFormat.ARGB32);
Output = new RenderTexture(width, height, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.sRGB);
Output.Create();
changed = true;
}
Expand Down Expand Up @@ -133,7 +133,9 @@ private void CpuConvertToRenderTarget(VideoFrameBuffer buffer, int width, int he
{
tempTex.LoadRawTextureData((IntPtr)rgba.Info.DataPtr, (int)rgba.GetMemorySize());
tempTex.Apply();
Graphics.Blit(tempTex, Output);

// Also mirror horizontally by flipping the UV scale on the X axis and offsetting.
Graphics.Blit(tempTex, Output, new Vector2(-1f, 1f), new Vector2(1f, 0f));
}
finally
{
Expand All @@ -148,6 +150,7 @@ private void GpuConvertToRenderTarget()
_yuvToRgbMaterial.SetTexture("_TexY", _planeY);
_yuvToRgbMaterial.SetTexture("_TexU", _planeU);
_yuvToRgbMaterial.SetTexture("_TexV", _planeV);

Graphics.Blit(Texture2D.blackTexture, Output, _yuvToRgbMaterial);
}
}
Expand Down
1 change: 0 additions & 1 deletion Samples~/Meet/Assets/Resources/LiveKitSdkVersionInfo.txt
View file
Open in desktop

This file was deleted.

8 changes: 8 additions & 0 deletions Samples~/Meet/Assets/Resources/Shaders.meta
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 ...ime/Utils/ResizeVideo/ResizeShader.shader → ...ets/Resources/Shaders/ResizeShader.shader
View file
Open in desktop
File renamed without changes.
0 ...tils/ResizeVideo/ResizeShader.shader.meta → ...esources/Shaders/ResizeShader.shader.meta
View file
Open in desktop
File renamed without changes.
1 change: 0 additions & 1 deletion Samples~/Meet/ProjectSettings/GraphicsSettings.asset
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ GraphicsSettings:
- {fileID: 10753, guid: 0000000000000000f000000000000000, type: 0}
- {fileID: 10770, guid: 0000000000000000f000000000000000, type: 0}
- {fileID: 10783, guid: 0000000000000000f000000000000000, type: 0}
- {fileID: 4800000, guid: 2475752d0dcc74d1eb5e66b8317e87b6, type: 3}
m_PreloadedShaders: []
m_PreloadShadersBatchTimeLimit: -1
m_SpritesDefaultMaterial: {fileID: 10754, guid: 0000000000000000f000000000000000, type: 0}
Expand Down
Loading